Fix DeformConv2d's autocast test flakiness

[datumbox]

A series of recent unrelated PRs exposed the fact that DeformConv2d's autocast unit-test is flaky and it can fail if we hit a bad seed. To resolve the problem, PR #3032 introduced a temporary workaround that fixes the random_seed:

vision/test/test_ops.py

Lines 657 to 659 in 06ebee1

	@unittest.skipIf(not torch.cuda.is_available(), "CUDA unavailable")
	def test_autocast(self):
	set_rng_seed(0)

The above is not a great solution. We should investigate and fix the problem properly by adjusting the unit-test.

To reproduce the issue, remove the set_rng_seed call from the test and rerun it.

[NicolasHug]

I can't really tell why the test is failing for some seeds but I made some experiments:

• It only fails for half precision; the float32 dtype sections are fine
• increasing the tol from 1e-3 to 1.5e-3, seeds in [0, 79] pass (and seed 80 fails)
• increasing tol further to 2e-3 makes all seeds in [0, 100] succeed

Considering that all checks for float32 pass without change, perhaps the simplest fix is to increase the tolerance to 2e-3 for half precision?

[datumbox]

Thanks for the analysis @NicolasHug. Perhaps it's just an issue of using too small tolerance. I think 2e-3 is reasonable if it resolves the issue. I'm happy to review it if you want to send a PR.