[CI] PyPy tests failing lately and it is difficult to identify the root cause

[abravalheri]

I first noticed this behaviour when reverting the deprecations introduced in v78.0.0.
But lately we have seen PyPy failing in what seems to be unrelated issues.

Example extracted from #4931:

We can see an error in test_setupcfg.py:488 which does not really have a good explanation.
In the setuptools.tests.config.test_setupcfg:TestOptions.test_basic function, we use the fake_env, but by itself it does not add any classifier. So we have a warning about classifiers without having specified any classifier?

[abravalheri]

The fact that the error keeps changing seems to me that we have some sort of race condition,
specially when it comes to the tmp_path/tmpdir fixture. (It seems that the temp directory is "reminiscing" from one test to the other)