Enable --no-silence-site-packages in eval tests (#5370)

An unintended side-effect of silencing errors in site packages
and typeshed by default is that our tests no longer flag
changes to mypy that end up breaking typeshed in some way.

(For example, see https://github.com/python/mypy/pulls/5280 which
*really* should not be passing, at least as of time of writing.)

This pull request will add the `--no-silence-site-packages` flag
to any test suites that directly or indirectly test typeshed.

Specifically, I believe this PR ought to add the flag to any tests
triggered by `testsamples.py`, `testselfcheck.py`, and
`testpythoneval.py`.

Author: Michael0x2a

mypy/test/helpers.py

@@ -32,7 +32,8 @@
 def run_mypy(args: List[str]) -> None:
     __tracebackhide__ = True
     outval, errval, status = api.run(args + ['--show-traceback',
-                                             '--no-site-packages'])
+                                             '--no-site-packages',
+                                             '--no-silence-site-packages'])
     if status != 0:
         sys.stdout.write(outval)
         sys.stderr.write(errval)

mypy/test/testpythoneval.py

@@ -50,7 +50,12 @@ def test_python_evaluation(testcase: DataDrivenTestCase, cache_dir: str) -> None
     """
     assert testcase.old_cwd is not None, "test was not properly set up"
     # TODO: Enable strict optional for these tests
-    mypy_cmdline = ['--show-traceback', '--no-site-packages', '--no-strict-optional']
+    mypy_cmdline = [
+        '--show-traceback',
+        '--no-site-packages',
+        '--no-strict-optional',
+        '--no-silence-site-packages',
+    ]
     py2 = testcase.name.lower().endswith('python2')
     if py2:
         mypy_cmdline.append('--py2')