Skip to content

Emscripten: use better _Py_Version computation for worker module #129757

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

agriyakhetarpal
Copy link
Contributor

@agriyakhetarpal agriyakhetarpal commented Feb 6, 2025

Related to gh-127113

This is a small PR that improves the _Py_Version check for the Python worker module file for the Emscripten builds, which was last modified in https://github.com/python/cpython/pull/127113/files#diff-31f61a127a59386957acba86e9acedf0a96bc6ebe1ac159a48e39b1a291111b5R73-R75. I reviewed the corresponding change downstream in pyodide/pyodide#5406 and suggested this change, and @hoodmane suggested that I should upstream the change here.

The idea is that we were converting a number to a hex string, then padding that string, splitting it up, and then bringing it back by parsing it to an integer – these operations are unnecessary. We can instead use an unsigned right shift and apply a bitmask to grab the relevant bytes. For the format 0xMMmmpp00, this means that the major version is in bits 24–31, the minor version in bits 16–23, and the patch/micro version in bits 8–15. These bit operations should be more direct.

@ghost
Copy link

ghost commented Feb 6, 2025

All commit authors signed the Contributor License Agreement.
CLA signed

@bedevere-app
Copy link

bedevere-app bot commented Feb 6, 2025

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

@agriyakhetarpal
Copy link
Contributor Author

This is my first PR to CPython, and I have signed the CLA. The skip news label can be applied, as this change is not directed towards users.

@agriyakhetarpal agriyakhetarpal changed the title gh-127113: Emscripten: use better _Py_Version computation for worker module Emscripten: use better _Py_Version computation for worker module Feb 6, 2025
@freakboy3742 freakboy3742 removed the request for review from brettcannon February 7, 2025 04:46
Copy link
Contributor

@freakboy3742 freakboy3742 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed - that is a lot more elegant! Thanks for the patch!

@freakboy3742 freakboy3742 merged commit e2064d6 into python:main Feb 7, 2025
56 checks passed
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Arch Linux Perf 3.x has failed when building commit e2064d6.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1078/builds/4254) and take a look at the build logs.
  4. Check if the failure is related to this commit (e2064d6) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1078/builds/4254

Failed tests:

  • test_urllib2net

Failed subtests:

  • test_ftp_timeout - test.test_urllib2net.TimeoutTest.test_ftp_timeout
  • test_ftp_default_timeout - test.test_urllib2net.TimeoutTest.test_ftp_default_timeout
  • test_ftp - test.test_urllib2net.OtherNetworkTests.test_ftp
  • test_ftp_no_timeout - test.test_urllib2net.TimeoutTest.test_ftp_no_timeout
  • test_ftp_basic - test.test_urllib2net.TimeoutTest.test_ftp_basic

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 1540, in ftp_open
    fw = self.connect_ftp(user, passwd, host, port, dirs, req.timeout)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 1560, in connect_ftp
    return ftpwrapper(user, passwd, host, port, dirs, timeout,
                      persistent=False)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 1736, in __init__
    self.init()
    ~~~~~~~~~^^
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 1745, in init
    self.ftp.connect(self.host, self.port, self.timeout)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/ftplib.py", line 158, in connect
    self.sock = socket.create_connection((self.host, self.port), self.timeout,
                ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                         source_address=self.source_address)
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/socket.py", line 870, in create_connection
    raise exceptions[0]
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/socket.py", line 855, in create_connection
    sock.connect(sa)
    ~~~~~~~~~~~~^^^^
ConnectionRefusedError: [Errno 111] Connection refused


Traceback (most recent call last):
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/test/test_urllib2net.py", line 372, in test_ftp_timeout
    u = _urlopen_with_retry(self.FTP_HOST, timeout=60)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/test/test_urllib2net.py", line 28, in wrapped
    return _retry_thrice(func, exc, *args, **kwargs)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/test/test_urllib2net.py", line 24, in _retry_thrice
    raise last_exc
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/test/test_urllib2net.py", line 20, in _retry_thrice
    return func(*args, **kwargs)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 187, in urlopen
    return opener.open(url, data, timeout)
           ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 487, in open
    response = self._open(req, data)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 504, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
                              '_open', req)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 464, in _call_chain
    result = func(*args)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 1557, in ftp_open
    raise URLError(f"ftp error: {exp}") from exp
urllib.error.URLError: <urlopen error ftp error: [Errno 111] Connection refused>


Traceback (most recent call last):
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/test/test_urllib2net.py", line 260, in _test_urls
    f = urlopen(url, req, support.INTERNET_TIMEOUT)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/test/test_urllib2net.py", line 28, in wrapped
    return _retry_thrice(func, exc, *args, **kwargs)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/test/test_urllib2net.py", line 24, in _retry_thrice
    raise last_exc
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/test/test_urllib2net.py", line 20, in _retry_thrice
    return func(*args, **kwargs)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 487, in open
    response = self._open(req, data)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 504, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
                              '_open', req)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 464, in _call_chain
    result = func(*args)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 1557, in ftp_open
    raise URLError(f"ftp error: {exp}") from exp
urllib.error.URLError: <urlopen error ftp error: [Errno 111] Connection refused>


Traceback (most recent call last):
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/test/test_urllib2net.py", line 342, in test_ftp_basic
    u = _urlopen_with_retry(self.FTP_HOST)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/test/test_urllib2net.py", line 28, in wrapped
    return _retry_thrice(func, exc, *args, **kwargs)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/test/test_urllib2net.py", line 24, in _retry_thrice
    raise last_exc
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/test/test_urllib2net.py", line 20, in _retry_thrice
    return func(*args, **kwargs)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 187, in urlopen
    return opener.open(url, data, timeout)
           ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 487, in open
    response = self._open(req, data)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 504, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
                              '_open', req)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 464, in _call_chain
    result = func(*args)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 1557, in ftp_open
    raise URLError(f"ftp error: {exp}") from exp
urllib.error.URLError: <urlopen error ftp error: [Errno 111] Connection refused>


Traceback (most recent call last):
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/test/test_urllib2net.py", line 363, in test_ftp_no_timeout
    u = _urlopen_with_retry(self.FTP_HOST, timeout=None)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/test/test_urllib2net.py", line 28, in wrapped
    return _retry_thrice(func, exc, *args, **kwargs)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/test/test_urllib2net.py", line 24, in _retry_thrice
    raise last_exc
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/test/test_urllib2net.py", line 20, in _retry_thrice
    return func(*args, **kwargs)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 187, in urlopen
    return opener.open(url, data, timeout)
           ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 487, in open
    response = self._open(req, data)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 504, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
                              '_open', req)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 464, in _call_chain
    result = func(*args)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 1557, in ftp_open
    raise URLError(f"ftp error: {exp}") from exp
urllib.error.URLError: <urlopen error ftp error: [Errno 111] Connection refused>


Traceback (most recent call last):
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/test/test_urllib2net.py", line 351, in test_ftp_default_timeout
    u = _urlopen_with_retry(self.FTP_HOST)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/test/test_urllib2net.py", line 28, in wrapped
    return _retry_thrice(func, exc, *args, **kwargs)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/test/test_urllib2net.py", line 24, in _retry_thrice
    raise last_exc
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/test/test_urllib2net.py", line 20, in _retry_thrice
    return func(*args, **kwargs)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 187, in urlopen
    return opener.open(url, data, timeout)
           ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 487, in open
    response = self._open(req, data)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 504, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
                              '_open', req)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 464, in _call_chain
    result = func(*args)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 1557, in ftp_open
    raise URLError(f"ftp error: {exp}") from exp
urllib.error.URLError: <urlopen error ftp error: [Errno 111] Connection refused>


Traceback (most recent call last):
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 1540, in ftp_open
    fw = self.connect_ftp(user, passwd, host, port, dirs, req.timeout)
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 1584, in connect_ftp
    self.cache[key] = ftpwrapper(user, passwd, host, port,
                      ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
                                 dirs, timeout)
                                 ^^^^^^^^^^^^^^
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 1736, in __init__
    self.init()
    ~~~~~~~~~^^
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/urllib/request.py", line 1745, in init
    self.ftp.connect(self.host, self.port, self.timeout)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/ftplib.py", line 158, in connect
    self.sock = socket.create_connection((self.host, self.port), self.timeout,
                ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                                         source_address=self.source_address)
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/socket.py", line 870, in create_connection
    raise exceptions[0]
  File "/buildbot/buildarea/3.x.pablogsal-arch-x86_64.perfbuild/build/Lib/socket.py", line 855, in create_connection
    sock.connect(sa)
    ~~~~~~~~~~~~^^^^
ConnectionRefusedError: [Errno 111] Connection refused

@agriyakhetarpal agriyakhetarpal deleted the fix/better-py-version-check-for-emscripten-build branch February 7, 2025 07:15
srinivasreddy pushed a commit to srinivasreddy/cpython that referenced this pull request Feb 7, 2025
…ython#129757)

Use integer bit shifting instead of conversion to strings to compute Python version.
@freakboy3742
Copy link
Contributor

Buildbot failure is unrelated to this change; it looms like a network connectivity issue on the host machine.

cmaloney pushed a commit to cmaloney/cpython that referenced this pull request Feb 8, 2025
…ython#129757)

Use integer bit shifting instead of conversion to strings to compute Python version.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants