Add vectorization in elementwise_util

[swolchok]

This is a first cut at #9241 . In this PR I've vectorized a small initial set of ops: atan2, clamp, fmod_Scalar, maximum, minimum, mul, pow, and sigmoid. In addition, the following ops should have gotten vectorized automatically because they already used generic lambdas: add, div, rsub, sub. I've left covering ops that use the unary_ufunc_* utilities in pattern.h for a follow-up push, because pattern.h and elementwise_util need some work before we can migrate pattern.h's utilities to be backed by elementwise_util.

This PR adds an interesting testing problem: in theory, all operators might need test cases long enough to tickle vectorization, because we might accidentally vectorize ops unexpectedly and break their lambdas due to anticipated differences in semantics. I address this issue by using Vectorized for the scalar prologue/epilogue in debug mode (we run tests in both debug and release) so that we can detect broken lambdas. I additionally intentionally introduced a bug in the vectorized path in elementwise_util and manually verified that we saw test failures for each vectorized op called out above.

[swolchok]

Stack from ghstack (oldest at bottom):

• Implement unary_ufunc functions using elementwise_util #9386
• Add mixed integer precision test for op_mul #11206
• Add SupportedTensorDtypes::BOOL #9584
• relax tolerances for all unary float ops #9585
• -> Add vectorization in elementwise_util #9432
• Add optimized_portable_kernels test #11205
• Add vectorized_math.h #11204

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9432

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 1 Unrelated Failure

As of commit 84170e4 with merge base 71025df ():

NEW FAILURE - The following job has failed:

• pull / test-llava-runner-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 13ce1b59de30a5a8c6d79f7ee03cbd73aefcd5bc77728be3148d6cbb5cd410a2 /exec failed with exit code 139

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• pull / test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job (gh) (detected as infra flaky with no log or failing log classifier)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[kimishpatel]

This is still in draft state. Are you planning to push more changes?

[swolchok]

This is still in draft state. Are you planning to push more changes?

I was waiting for CI to be green before bothering reviewers. Will publish shortly