`socket_helper.transient_internet` helper fails when FTP server returns 500

[freakboy3742]

Bug report

Bug description:

We've seen a bunch of CI failures over the last few hours in the urllib2 FTP tests. These errors have been observed on iOS (1 2), macOS (1) and Linux (1). [edit: and Windows too (1) – @encukou]

The affected tests are:

• test_ftp_timeout - test.test_urllib2net.TimeoutTest.test_ftp_timeout
• test_ftp - test.test_urllib2net.OtherNetworkTests.test_ftp
• test_ftp_no_timeout - test.test_urllib2net.TimeoutTest.test_ftp_no_timeout
• test_ftp_basic - test.test_urllib2net.TimeoutTest.test_ftp_basic
• test_ftp_default_timeout - test.test_urllib2net.TimeoutTest.test_ftp_default_timeout

It's not yet clear how to reliably reproduce the error - it appears to be a transient issue caused by the upstream FTP server raising a 500 error (possibly as a result of #122795).

From a code perspective, the issue is that the error being raised internally is an instance of error_perm; this is used as the err.reason for the URLError raised by urllib2, which breaks the exception filter.

The error manifests as:

Traceback (most recent call last):
  File "/Users/buildbot/Library/Developer/XCTestDevices/AF99343E-DD9B-4A75-8EE3-3995A510EB7E/data/Containers/Bundle/Application/E807E918-8DD2-4F98-9247-402427533BB4/[iOSTestbed.app/python/lib/python3.13/test/support/socket_helper.py](http://iostestbed.app/python/lib/python3.13/test/support/socket_helper.py)", line 249, in transient_internet
    yield
  File "/Users/buildbot/Library/Developer/XCTestDevices/AF99343E-DD9B-4A75-8EE3-3995A510EB7E/data/Containers/Bundle/Application/E807E918-8DD2-4F98-9247-402427533BB4/[iOSTestbed.app/python/lib/python3.13/test/test_urllib2net.py](http://iostestbed.app/python/lib/python3.13/test/test_urllib2net.py)", line 343, in test_ftp_basic
    u = _urlopen_with_retry(self.FTP_HOST)
  File "/Users/buildbot/Library/Developer/XCTestDevices/AF99343E-DD9B-4A75-8EE3-3995A510EB7E/data/Containers/Bundle/Application/E807E918-8DD2-4F98-9247-402427533BB4/[iOSTestbed.app/python/lib/python3.13/test/test_urllib2net.py](http://iostestbed.app/python/lib/python3.13/test/test_urllib2net.py)", line 29, in wrapped
    return _retry_thrice(func, exc, *args, **kwargs)
  File "/Users/buildbot/Library/Developer/XCTestDevices/AF99343E-DD9B-4A75-8EE3-3995A510EB7E/data/Containers/Bundle/Application/E807E918-8DD2-4F98-9247-402427533BB4/[iOSTestbed.app/python/lib/python3.13/test/test_urllib2net.py](http://iostestbed.app/python/lib/python3.13/test/test_urllib2net.py)", line 25, in _retry_thrice
    raise last_exc
  File "/Users/buildbot/Library/Developer/XCTestDevices/AF99343E-DD9B-4A75-8EE3-3995A510EB7E/data/Containers/Bundle/Application/E807E918-8DD2-4F98-9247-402427533BB4/[iOSTestbed.app/python/lib/python3.13/test/test_urllib2net.py](http://iostestbed.app/python/lib/python3.13/test/test_urllib2net.py)", line 21, in _retry_thrice
    return func(*args, **kwargs)
  File "/Users/buildbot/Library/Developer/XCTestDevices/AF99343E-DD9B-4A75-8EE3-3995A510EB7E/data/Containers/Bundle/Application/E807E918-8DD2-4F98-9247-402427533BB4/[iOSTestbed.app/python/lib/python3.13/urllib/request.py](http://iostestbed.app/python/lib/python3.13/urllib/request.py)", line 189, in urlopen
    return [opener.open](http://opener.open/)(url, data, timeout)
           ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/AF99343E-DD9B-4A75-8EE3-3995A510EB7E/data/Containers/Bundle/Application/E807E918-8DD2-4F98-9247-402427533BB4/[iOSTestbed.app/python/lib/python3.13/urllib/request.py](http://iostestbed.app/python/lib/python3.13/urllib/request.py)", line 489, in open
    response = self._open(req, data)
  File "/Users/buildbot/Library/Developer/XCTestDevices/AF99343E-DD9B-4A75-8EE3-3995A510EB7E/data/Containers/Bundle/Application/E807E918-8DD2-4F98-9247-402427533BB4/[iOSTestbed.app/python/lib/python3.13/urllib/request.py](http://iostestbed.app/python/lib/python3.13/urllib/request.py)", line 506, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
                              '_open', req)
  File "/Users/buildbot/Library/Developer/XCTestDevices/AF99343E-DD9B-4A75-8EE3-3995A510EB7E/data/Containers/Bundle/Application/E807E918-8DD2-4F98-9247-402427533BB4/[iOSTestbed.app/python/lib/python3.13/urllib/request.py](http://iostestbed.app/python/lib/python3.13/urllib/request.py)", line 466, in _call_chain
    result = func(*args)
  File "/Users/buildbot/Library/Developer/XCTestDevices/AF99343E-DD9B-4A75-8EE3-3995A510EB7E/data/Containers/Bundle/Application/E807E918-8DD2-4F98-9247-402427533BB4/[iOSTestbed.app/python/lib/python3.13/urllib/request.py](http://iostestbed.app/python/lib/python3.13/urllib/request.py)", line 1558, in ftp_open
    raise URLError(exp) from exp
urllib.error.URLError: <urlopen error 500 OOPS: vsf_sysutil_bind>


Traceback (most recent call last):
  File "/Users/buildbot/Library/Developer/XCTestDevices/AF99343E-DD9B-4A75-8EE3-3995A510EB7E/data/Containers/Bundle/Application/E807E918-8DD2-4F98-9247-402427533BB4/[iOSTestbed.app/python/lib/python3.13/test/test_urllib2net.py](http://iostestbed.app/python/lib/python3.13/test/test_urllib2net.py)", line 349, in test_ftp_default_timeout
    with socket_helper.transient_internet(self.FTP_HOST):
    ...<6 lines>...
        self.assertEqual(u.fp.fp.raw._sock.gettimeout(), 60)
  File "/Users/buildbot/Library/Developer/XCTestDevices/AF99343E-DD9B-4A75-8EE3-3995A510EB7E/data/Containers/Bundle/Application/E807E918-8DD2-4F98-9247-402427533BB4/[iOSTestbed.app/python/lib/python3.13/contextlib.py](http://iostestbed.app/python/lib/python3.13/contextlib.py)", line 162, in __exit__
    self.gen.throw(value)
    ~~~~~~~~~~~~~~^^^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/AF99343E-DD9B-4A75-8EE3-3995A510EB7E/data/Containers/Bundle/Application/E807E918-8DD2-4F98-9247-402427533BB4/[iOSTestbed.app/python/lib/python3.13/test/support/socket_helper.py](http://iostestbed.app/python/lib/python3.13/test/support/socket_helper.py)", line 264, in transient_internet
    filter_error(err)
    ~~~~~~~~~~~~^^^^^
  File "/Users/buildbot/Library/Developer/XCTestDevices/AF99343E-DD9B-4A75-8EE3-3995A510EB7E/data/Containers/Bundle/Application/E807E918-8DD2-4F98-9247-402427533BB4/[iOSTestbed.app/python/lib/python3.13/test/support/socket_helper.py](http://iostestbed.app/python/lib/python3.13/test/support/socket_helper.py)", line 237, in filter_error
    (("ConnectionRefusedError" in err.reason) or
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: argument of type 'error_perm' is not iterable

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

[encukou]

Windows too: https://buildbot.python.org/#/builders/1079/builds/4538/steps/5/logs/stdio

[serhiy-storchaka]

I tried to fix this with the following change:

diff --git a/Lib/test/support/socket_helper.py b/Lib/test/support/socket_helper.py
index 87941ee1791..dcbd7e3ebd4 100644
--- a/Lib/test/support/socket_helper.py
+++ b/Lib/test/support/socket_helper.py
@@ -234,9 +234,12 @@ def filter_error(err):
             (isinstance(err, urllib.error.HTTPError) and
              500 <= err.code <= 599) or
             (isinstance(err, urllib.error.URLError) and
-                 (("ConnectionRefusedError" in err.reason) or
-                  ("TimeoutError" in err.reason) or
-                  ("EOFError" in err.reason))) or
+                 ((isinstance(err.reason, BaseException) and
+                   isinstance(err.reason.__cause__,
+                             (ConnectionRefusedError, TimeoutError, EOFError))) or
+                  ("ConnectionRefusedError" in str(err.reason)) or
+                  ("TimeoutError" in str(err.reason)) or
+                  ("EOFError" in str(err.reason)))) or
             n in captured_errnos):
             if not support.verbose:
                 sys.stderr.write(denied.args[0] + "\n")

But then the underlying issue was solved, and I do not know how to reproduce the failure.

This may only fix some symptoms, and I am not sure that it is the best fix. There may be a regression in the stdlib code after making an argument of URLError an exception instead of a string. Or the helper take this into account and unroll these exceptions like it already do with other exception chains. Without reproducer I cannot do experiments.

[serhiy-storchaka]

#32074 may be one of culprits. Before, the error message was formatted as 'ftp error: %r', after -- as f'ftp error: {exp}', i.e. it no longer contains the name of the original exception type.

[freakboy3742]

@serhiy-storchaka I was thinking taking of a similar approach, but hit the same issue - how do you validate the code is actually working?

However - @zware raised an interesting point on Discord: the fact this failed could be considered a good thing, because it drew our attention to the fact the pythontest server had a configuration issue that needed to be addressed. If the network failure had been silenced, the test would have been marked as a skip and the issue with the server likely would have gone unnoticed. That could have led to a situation where, despite having tests, CI for a release wasn't actually verifying that network-dependent features were working.

So - maybe not fixing this is the better approach?

[freakboy3742]

Looks like @jeremyhylton reported this separately a few days ago (#122909), and has a prototype patch that goes a bit deeper into url lib (#122913). Closing as a duplicate.