Skip to content

bpo-45229: Make datetime tests discoverable #28615

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Sep 29, 2021

Conversation

serhiy-storchaka
Copy link
Member

@serhiy-storchaka serhiy-storchaka commented Sep 29, 2021


def test_main():
run_unittest(*all_test_classes)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this isn't used anymore, the import of run_unittest isn't necessary either.

Copy link
Member Author

@serhiy-storchaka serhiy-storchaka Sep 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I missed this.

Copy link
Contributor

@ambv ambv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test_datetime on main: Ran 960 tests in 9.497s; OK (skipped=28)
test_datetime on this PR: Ran 960 tests in 9.468s; OK (skipped=28)

Same tests running:

--- /tmp/main_datetime.log	2021-09-29 21:26:34.000000000 +0200
+++ /tmp/28615_datetime.log	2021-09-29 21:28:25.000000000 +0200
@@ -1 +1 @@
-Wed Sep 29 21:26:23 CEST 2021
+Wed Sep 29 21:28:14 CEST 2021
@@ -3 +3 @@
-== CPython 3.11.0a0 (heads/main:45ca1c0413, Sep 29 2021, 21:24:05) [Clang 12.0.0 (clang-1200.0.32.29)]
+== CPython 3.11.0a0 (heads/test-datetime-discover:1131d9fb69, Sep 29 2021, 21:27:42) [Clang 12.0.0 (clang-1200.0.32.29)]
@@ -5 +5 @@
-== cwd: /Users/ambv/Dropbox/Python/cpython/build/test_python_40589æ
+== cwd: /Users/ambv/Dropbox/Python/cpython/build/test_python_49597æ
@@ -8,2 +8,2 @@
-0:00:00 load avg: 2.35 Run tests sequentially
-0:00:00 load avg: 2.35 [1/1] test_datetime
+0:00:00 load avg: 3.01 Run tests sequentially
+0:00:00 load avg: 3.01 [1/1] test_datetime
@@ -982 +982 @@
-Ran 960 tests in 9.497s
+Ran 960 tests in 9.468s

@miss-islington
Copy link
Contributor

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.9, 3.10.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

@serhiy-storchaka serhiy-storchaka deleted the test-datetime-discover branch September 29, 2021 20:28
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Sep 29, 2021
(cherry picked from commit d441437)

Co-authored-by: Serhiy Storchaka <[email protected]>
@miss-islington
Copy link
Contributor

Sorry, @serhiy-storchaka, I could not cleanly backport this to 3.9 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker d441437ee71ae174c008c23308b749b91020ba77 3.9

@bedevere-bot
Copy link

GH-28639 is a backport of this pull request to the 3.10 branch.

@bedevere-bot bedevere-bot removed the needs backport to 3.10 only security fixes label Sep 29, 2021
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this pull request Sep 30, 2021
(cherry picked from commit d441437)

Co-authored-by: Serhiy Storchaka <[email protected]>
@bedevere-bot bedevere-bot removed the needs backport to 3.9 only security fixes label Sep 30, 2021
@bedevere-bot
Copy link

GH-28645 is a backport of this pull request to the 3.9 branch.

@pablogsal
Copy link
Member

Unfortunately, this PR seems to have broken a bunch of buildbots. For instance:

https://buildbot.python.org/all/#/builders/75/builds/172/steps/5/logs/stdio

test_divide_and_round (test.datetimetester.TestModule_Pure) ... ok
test_divide_and_round (test.datetimetester.TestModule_Fast) ... skipped 'Only run for Pure Python implementation'
----------------------------------------------------------------------
Ran 2 tests in 0.000s
OK (skipped=1)
test_divide_and_round (test.datetimetester.TestModule_Pure) ... ERROR
.test test_datetime failed
test_divide_and_round (test.datetimetester.TestModule_Fast) ... skipped 'Only run for Pure Python implementation'
======================================================================
ERROR: test_divide_and_round (test.datetimetester.TestModule_Pure)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-rhel8-z.refleak/build/Lib/test/datetimetester.py", line 88, in test_divide_and_round
    dar = datetime_module._divide_and_round
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: module 'datetime' has no attribute '_divide_and_round'
----------------------------------------------------------------------
Ran 2 tests in 0.003s
FAILED (errors=1, skipped=1)
1 test failed again:
    test_datetime
== Tests result: FAILURE then FAILURE ==

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Fedora Stable Clang 3.x has failed when building commit b07fddd.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/441/builds/732) and take a look at the build logs.
  4. Check if the failure is related to this commit (b07fddd) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/441/builds/732

Failed tests:

  • test_multiprocessing_fork
  • test_multiprocessing_forkserver
  • test_multiprocessing_spawn

Failed subtests:

  • test_shared_memory_basics - test.test_multiprocessing_spawn.WithProcessesTestSharedMemory
  • test_shared_memory_basics - test.test_multiprocessing_forkserver.WithProcessesTestSharedMemory
  • test_shared_memory_basics - test.test_multiprocessing_fork.WithProcessesTestSharedMemory

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

412 tests OK.

10 slowest tests:

  • test_peg_generator: 2 min 49 sec
  • test_concurrent_futures: 2 min 40 sec
  • test_multiprocessing_spawn: 1 min 35 sec
  • test_nntplib: 1 min 21 sec
  • test_multiprocessing_forkserver: 1 min 13 sec
  • test_multiprocessing_fork: 1 min 6 sec
  • test_asyncio: 57.3 sec
  • test_signal: 47.2 sec
  • test_io: 38.7 sec
  • test_socket: 38.2 sec

3 tests failed:
test_multiprocessing_fork test_multiprocessing_forkserver
test_multiprocessing_spawn

12 tests skipped:
test_devpoll test_ioctl test_kqueue test_msilib test_startfile
test_tix test_tk test_ttk_guionly test_winconsoleio test_winreg
test_winsound test_zipfile64

3 re-run tests:
test_multiprocessing_fork test_multiprocessing_forkserver
test_multiprocessing_spawn

Total duration: 3 min 19 sec

Click to see traceback logs
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.clang/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
    ^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: '/psm_6bdda612'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.clang/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
    ^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: '/psm_21c743d3'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.clang/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
    ^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: '/psm_e86a3721'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.clang/build/Lib/test/_test_multiprocessing.py", line 3777, in test_shared_memory_basics
    sms = shared_memory.SharedMemory('test01_tsmb', create=True, size=512)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.clang/build/Lib/multiprocessing/shared_memory.py", line 103, in __init__
    self._fd = _posixshmem.shm_open(
               ^^^^^^^^^^^^^^^^^^^^^
FileExistsError: [Errno 17] File exists: '/test01_tsmb'

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Fedora Stable Clang Installed 3.x has failed when building commit b07fddd.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/350/builds/737) and take a look at the build logs.
  4. Check if the failure is related to this commit (b07fddd) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/350/builds/737

Failed tests:

  • test_multiprocessing_fork
  • test_multiprocessing_forkserver
  • test_multiprocessing_spawn

Failed subtests:

  • test_shared_memory_basics - test.test_multiprocessing_forkserver.WithProcessesTestSharedMemory
  • test_shared_memory_basics - test.test_multiprocessing_fork.WithProcessesTestSharedMemory
  • test_shared_memory_basics - test.test_multiprocessing_spawn.WithProcessesTestSharedMemory

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

408 tests OK.

3 tests failed:
test_multiprocessing_fork test_multiprocessing_forkserver
test_multiprocessing_spawn

16 tests skipped:
test_asdl_parser test_check_c_globals test_clinic test_devpoll
test_gdb test_ioctl test_kqueue test_msilib test_startfile
test_tix test_tk test_ttk_guionly test_winconsoleio test_winreg
test_winsound test_zipfile64

3 re-run tests:
test_multiprocessing_fork test_multiprocessing_forkserver
test_multiprocessing_spawn

Total duration: 2 min 50 sec

Click to see traceback logs
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.11/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
    ^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: '/psm_5a1b37ec'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.11/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
    ^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: '/psm_9c328efe'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.11/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
    ^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: '/psm_a28821d5'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.11/test/_test_multiprocessing.py", line 3777, in test_shared_memory_basics
    sms = shared_memory.SharedMemory('test01_tsmb', create=True, size=512)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.11/multiprocessing/shared_memory.py", line 103, in __init__
    self._fd = _posixshmem.shm_open(
               ^^^^^^^^^^^^^^^^^^^^^
FileExistsError: [Errno 17] File exists: '/test01_tsmb'

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x Fedora LTO + PGO 3.x has failed when building commit b07fddd.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/545/builds/783) and take a look at the build logs.
  4. Check if the failure is related to this commit (b07fddd) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/545/builds/783

Failed tests:

  • test_multiprocessing_fork
  • test_multiprocessing_forkserver
  • test_multiprocessing_spawn

Failed subtests:

  • test_shared_memory_basics - test.test_multiprocessing_spawn.WithProcessesTestSharedMemory
  • test_shared_memory_basics - test.test_multiprocessing_forkserver.WithProcessesTestSharedMemory
  • test_shared_memory_basics - test.test_multiprocessing_fork.WithProcessesTestSharedMemory

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

409 tests OK.

10 slowest tests:

  • test_concurrent_futures: 2 min 31 sec
  • test_multiprocessing_spawn: 1 min 20 sec
  • test_multiprocessing_forkserver: 1 min 9 sec
  • test_nntplib: 1 min 6 sec
  • test_multiprocessing_fork: 1 min 3 sec
  • test_signal: 47.5 sec
  • test_asyncio: 44.1 sec
  • test_socket: 37.7 sec
  • test_io: 31.9 sec
  • test_pydoc: 31.2 sec

3 tests failed:
test_multiprocessing_fork test_multiprocessing_forkserver
test_multiprocessing_spawn

15 tests skipped:
test_devpoll test_gdb test_ioctl test_kqueue test_msilib test_nis
test_readline test_startfile test_tix test_tk test_ttk_guionly
test_winconsoleio test_winreg test_winsound test_zipfile64

3 re-run tests:
test_multiprocessing_fork test_multiprocessing_forkserver
test_multiprocessing_spawn

Total duration: 4 min 15 sec

Click to see traceback logs
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.lto-pgo/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
    ^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: '/psm_51a8726a'


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.lto-pgo/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
    ^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: '/psm_b0dd5439'


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.lto-pgo/build/Lib/test/_test_multiprocessing.py", line 3777, in test_shared_memory_basics
    sms = shared_memory.SharedMemory('test01_tsmb', create=True, size=512)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.lto-pgo/build/Lib/multiprocessing/shared_memory.py", line 103, in __init__
    self._fd = _posixshmem.shm_open(
               ^^^^^^^^^^^^^^^^^^^^^
FileExistsError: [Errno 17] File exists: '/test01_tsmb'


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.lto-pgo/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
    ^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: '/psm_31cae0a6'

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Fedora Stable LTO 3.x has failed when building commit b07fddd.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/271/builds/669) and take a look at the build logs.
  4. Check if the failure is related to this commit (b07fddd) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/271/builds/669

Failed tests:

  • test_multiprocessing_fork
  • test_multiprocessing_forkserver
  • test_multiprocessing_spawn

Failed subtests:

  • test_shared_memory_basics - test.test_multiprocessing_spawn.WithProcessesTestSharedMemory
  • test_shared_memory_basics - test.test_multiprocessing_forkserver.WithProcessesTestSharedMemory
  • test_shared_memory_basics - test.test_multiprocessing_fork.WithProcessesTestSharedMemory

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

412 tests OK.

10 slowest tests:

  • test_peg_generator: 3 min 45 sec
  • test_concurrent_futures: 2 min 31 sec
  • test_gdb: 1 min 44 sec
  • test_multiprocessing_spawn: 1 min 32 sec
  • test_nntplib: 1 min 21 sec
  • test_multiprocessing_forkserver: 1 min 12 sec
  • test_multiprocessing_fork: 1 min 4 sec
  • test_asyncio: 54.1 sec
  • test_signal: 47.5 sec
  • test_socket: 37.9 sec

3 tests failed:
test_multiprocessing_fork test_multiprocessing_forkserver
test_multiprocessing_spawn

12 tests skipped:
test_devpoll test_ioctl test_kqueue test_msilib test_startfile
test_tix test_tk test_ttk_guionly test_winconsoleio test_winreg
test_winsound test_zipfile64

3 re-run tests:
test_multiprocessing_fork test_multiprocessing_forkserver
test_multiprocessing_spawn

Total duration: 4 min 26 sec

Click to see traceback logs
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
    ^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: '/psm_717541bc'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
    ^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: '/psm_c27b3f3e'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto/build/Lib/test/_test_multiprocessing.py", line 3777, in test_shared_memory_basics
    sms = shared_memory.SharedMemory('test01_tsmb', create=True, size=512)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto/build/Lib/multiprocessing/shared_memory.py", line 103, in __init__
    self._fd = _posixshmem.shm_open(
               ^^^^^^^^^^^^^^^^^^^^^
FileExistsError: [Errno 17] File exists: '/test01_tsmb'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
    ^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: '/psm_fe719e16'

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x Fedora LTO 3.x has failed when building commit b07fddd.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/55/builds/730) and take a look at the build logs.
  4. Check if the failure is related to this commit (b07fddd) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/55/builds/730

Failed tests:

  • test_multiprocessing_fork
  • test_multiprocessing_forkserver
  • test_multiprocessing_spawn

Failed subtests:

  • test_shared_memory_basics - test.test_multiprocessing_spawn.WithProcessesTestSharedMemory
  • test_shared_memory_basics - test.test_multiprocessing_forkserver.WithProcessesTestSharedMemory
  • test_shared_memory_basics - test.test_multiprocessing_fork.WithProcessesTestSharedMemory

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

410 tests OK.

10 slowest tests:

  • test_peg_generator: 3 min 45 sec
  • test_concurrent_futures: 2 min 32 sec
  • test_gdb: 1 min 43 sec
  • test_multiprocessing_spawn: 1 min 26 sec
  • test_multiprocessing_forkserver: 1 min 12 sec
  • test_nntplib: 1 min 5 sec
  • test_multiprocessing_fork: 1 min 4 sec
  • test_signal: 47.2 sec
  • test_asyncio: 45.1 sec
  • test_socket: 38.0 sec

3 tests failed:
test_multiprocessing_fork test_multiprocessing_forkserver
test_multiprocessing_spawn

14 tests skipped:
test_devpoll test_ioctl test_kqueue test_msilib test_nis
test_readline test_startfile test_tix test_tk test_ttk_guionly
test_winconsoleio test_winreg test_winsound test_zipfile64

3 re-run tests:
test_multiprocessing_fork test_multiprocessing_forkserver
test_multiprocessing_spawn

Total duration: 6 min 5 sec

Click to see traceback logs
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.lto/build/Lib/test/_test_multiprocessing.py", line 3777, in test_shared_memory_basics
    sms = shared_memory.SharedMemory('test01_tsmb', create=True, size=512)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.lto/build/Lib/multiprocessing/shared_memory.py", line 103, in __init__
    self._fd = _posixshmem.shm_open(
               ^^^^^^^^^^^^^^^^^^^^^
FileExistsError: [Errno 17] File exists: '/test01_tsmb'


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.lto/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
    ^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: '/psm_94aacdad'


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.lto/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
    ^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: '/psm_bb76f3e9'


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-fedora-z.lto/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
    ^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: '/psm_c472e72d'

serhiy-storchaka added a commit that referenced this pull request Sep 30, 2021
ambv added a commit to ambv/cpython that referenced this pull request Sep 30, 2021
@serhiy-storchaka serhiy-storchaka added the needs backport to 3.10 only security fixes label Sep 30, 2021
@miss-islington
Copy link
Contributor

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.10.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

ambv added a commit that referenced this pull request Sep 30, 2021
serhiy-storchaka added a commit that referenced this pull request Oct 1, 2021
serhiy-storchaka added a commit that referenced this pull request Oct 1, 2021
@serhiy-storchaka serhiy-storchaka added needs backport to 3.10 only security fixes and removed needs backport to 3.10 only security fixes labels Oct 1, 2021
@miss-islington
Copy link
Contributor

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.10.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

@bedevere-bot
Copy link

GH-28668 is a backport of this pull request to the 3.10 branch.

@bedevere-bot bedevere-bot removed the needs backport to 3.10 only security fixes label Oct 1, 2021
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Oct 1, 2021
(cherry picked from commit d441437)

Co-authored-by: Serhiy Storchaka <[email protected]>
miss-islington added a commit that referenced this pull request Oct 1, 2021
(cherry picked from commit d441437)

Co-authored-by: Serhiy Storchaka <[email protected]>
serhiy-storchaka added a commit that referenced this pull request Oct 1, 2021
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot x86-64 macOS 3.9 has failed when building commit e9d5cdd.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/108/builds/275) and take a look at the build logs.
  4. Check if the failure is related to this commit (e9d5cdd) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/108/builds/275

Failed tests:

  • test_importlib

Failed subtests:

  • test_multiprocessing_pool_circular_import - test.test_importlib.test_threaded_import.ThreadedImportTests

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

406 tests OK.

10 slowest tests:

  • test_concurrent_futures: 4 min 45 sec
  • test_multiprocessing_spawn: 3 min 55 sec
  • test_tokenize: 3 min 29 sec
  • test_unparse: 3 min 13 sec
  • test_multiprocessing_forkserver: 2 min 39 sec
  • test_lib2to3: 2 min 17 sec
  • test_unicodedata: 1 min 58 sec
  • test_capi: 1 min 48 sec
  • test_asyncio: 1 min 47 sec
  • test_io: 1 min 6 sec

1 test failed:
test_importlib

18 tests skipped:
test_devpoll test_epoll test_gdb test_ioctl test_msilib
test_multiprocessing_fork test_ossaudiodev test_smtpnet test_spwd
test_ssl test_startfile test_tix test_tk test_ttk_guionly
test_winconsoleio test_winreg test_winsound test_zipfile64

1 re-run test:
test_importlib

Total duration: 28 min 23 sec

Click to see traceback logs
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py", line 201, in main
    cache[rtype].remove(name)
KeyError: '/psm_730e30b3'


Traceback (most recent call last):
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/test/test_importlib/partial/pool_in_threads.py", line 9, in t
    with multiprocessing.Pool(1):
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/test/test_importlib/partial/pool_in_threads.py", line 9, in t
    with multiprocessing.Pool(1):
Traceback (most recent call last):
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/context.py", line 119, in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild,
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/test/test_importlib/partial/pool_in_threads.py", line 9, in t
    with multiprocessing.Pool(1):
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/context.py", line 119, in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild,
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/test/test_importlib/partial/pool_in_threads.py", line 9, in t
    with multiprocessing.Pool(1):
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/test/test_importlib/partial/pool_in_threads.py", line 9, in t
    with multiprocessing.Pool(1):
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/pool.py", line 212, in __init__
    self._repopulate_pool()
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/pool.py", line 212, in __init__
    self._repopulate_pool()
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/context.py", line 119, in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild,
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/pool.py", line 303, in _repopulate_pool
    return self._repopulate_pool_static(self._ctx, self.Process,
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/context.py", line 119, in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild,
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/context.py", line 119, in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild,
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/pool.py", line 326, in _repopulate_pool_static
    w.start()
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/pool.py", line 196, in __init__
    self._change_notifier = self._ctx.SimpleQueue()
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/pool.py", line 212, in __init__
    self._repopulate_pool()
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/context.py", line 113, in SimpleQueue
    return SimpleQueue(ctx=self.get_context())
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/pool.py", line 303, in _repopulate_pool
    return self._repopulate_pool_static(self._ctx, self.Process,
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/queues.py", line 347, in __init__
    self._wlock = ctx.Lock()
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/pool.py", line 326, in _repopulate_pool_static
    w.start()
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/context.py", line 68, in Lock
    return Lock(ctx=self.get_context())
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/synchronize.py", line 162, in __init__
    SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/popen_spawn_posix.py", line 58, in _launch
    self.pid = util.spawnv_passfds(spawn.get_executable(),
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/synchronize.py", line 57, in __init__
    sl = self._semlock = _multiprocessing.SemLock(
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/util.py", line 450, in spawnv_passfds
    errpipe_read, errpipe_write = os.pipe()
OSError: [Errno 24] Too many open files
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
OSError: [Errno 24] Too many open files
/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 40 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '
/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/mp-8cr2ntre': [Errno 2] No such file or directory
  warnings.warn('resource_tracker: %r: %s' % (name, e))
/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/mp-hx_it7fs': [Errno 2] No such file or directory
  warnings.warn('resource_tracker: %r: %s' % (name, e))
/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/mp-sqyu177q': [Errno 2] No such file or directory
  warnings.warn('resource_tracker: %r: %s' % (name, e))
/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/mp-hqvqd9hl': [Errno 2] No such file or directory
  warnings.warn('resource_tracker: %r: %s' % (name, e))
/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/mp-g70vtowj': [Errno 2] No such file or directory
  warnings.warn('resource_tracker: %r: %s' % (name, e))
/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/mp-fapxu8n7': [Errno 2] No such file or directory
  warnings.warn('resource_tracker: %r: %s' % (name, e))
/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/mp-73fwh8m1': [Errno 2] No such file or directory
  warnings.warn('resource_tracker: %r: %s' % (name, e))
---


Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/test/test_importlib/partial/pool_in_threads.py", line 9, in t
    with multiprocessing.Pool(1):
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/context.py", line 119, in Pool
    return Pool(processes, initializer, initargs, maxtasksperchild,
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/pool.py", line 212, in __init__
    self._repopulate_pool()
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/pool.py", line 303, in _repopulate_pool
    return self._repopulate_pool_static(self._ctx, self.Process,
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/pool.py", line 326, in _repopulate_pool_static
    w.start()
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/context.py", line 284, in _Popen
    return Popen(process_obj)
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/popen_spawn_posix.py", line 32, in __init__
    super().__init__(process_obj)
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/popen_spawn_posix.py", line 53, in _launch
    parent_r, child_w = os.pipe()
OSError: [Errno 24] Too many open files
/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 106 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '
/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/mp-x_sz47x4': [Errno 2] No such file or directory
  warnings.warn('resource_tracker: %r: %s' % (name, e))
/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/mp-3a2go6gn': [Errno 2] No such file or directory
  warnings.warn('resource_tracker: %r: %s' % (name, e))
/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/mp-iy4fjx_0': [Errno 2] No such file or directory
  warnings.warn('resource_tracker: %r: %s' % (name, e))
/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/mp-n4g7i_68': [Errno 2] No such file or directory
  warnings.warn('resource_tracker: %r: %s' % (name, e))
/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/mp-hmkz4tu9': [Errno 2] No such file or directory
  warnings.warn('resource_tracker: %r: %s' % (name, e))
/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/mp-q4pgway3': [Errno 2] No such file or directory
  warnings.warn('resource_tracker: %r: %s' % (name, e))
/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/mp-iczxkhjm': [Errno 2] No such file or directory
  warnings.warn('resource_tracker: %r: %s' % (name, e))
/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/mp-wzjb3y_c': [Errno 2] No such file or directory
  warnings.warn('resource_tracker: %r: %s' % (name, e))
/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/mp-9flvh6my': [Errno 2] No such file or directory
  warnings.warn('resource_tracker: %r: %s' % (name, e))
/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/mp-rsq0cyw1': [Errno 2] No such file or directory
  warnings.warn('resource_tracker: %r: %s' % (name, e))
/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/mp-j_r67s3e': [Errno 2] No such file or directory
  warnings.warn('resource_tracker: %r: %s' % (name, e))
/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/mp-mwnx3x26': [Errno 2] No such file or directory
  warnings.warn('resource_tracker: %r: %s' % (name, e))
/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/mp-2hng3evl': [Errno 2] No such file or directory
  warnings.warn('resource_tracker: %r: %s' % (name, e))
/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/multiprocessing/resource_tracker.py:229: UserWarning: resource_tracker: '/mp-_eaia84f': [Errno 2] No such file or directory
  warnings.warn('resource_tracker: %r: %s' % (name, e))
---


Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/test/test_importlib/test_threaded_import.py", line 258, in test_multiprocessing_pool_circular_import
    script_helper.assert_python_ok(fn)
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/test/support/script_helper.py", line 156, in assert_python_ok
    return _assert_python(True, *args, **env_vars)
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/test/support/script_helper.py", line 142, in _assert_python
    res.fail(cmd_line)
  File "/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/test/support/script_helper.py", line 70, in fail
    raise AssertionError("Process return code is %d\n"
AssertionError: Process return code is 1
command line: ['/Users/buildbot/buildarea/3.9.billenstein-macos/build/python.exe', '-X', 'faulthandler', '-I', '/Users/buildbot/buildarea/3.9.billenstein-macos/build/Lib/test/test_importlib/partial/pool_in_threads.py']

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
skip news tests Tests in the Lib/test dir
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants