bpo-40094: Add test.support.wait_process() #19254

vstinner · 2020-03-31T17:05:12Z

Moreover, the following tests now check the child process exit code:

test_os.PtyTests
test_mailbox.test_lock_conflict()
test_tempfile.test_process_awareness()
test_uuid.testIssue8621()
multiprocessing resource tracker tests

https://bugs.python.org/issue40094

Moreover, the following tests now check the child process exit code: * test_os.PtyTests * test_mailbox.test_lock_conflict() * test_tempfile.test_process_awareness() * test_uuid.testIssue8621() * multiprocessing resource tracker tests

vstinner · 2020-03-31T17:07:53Z

I have other pending changes for test_thread, test_threading, test_fork1, test_wait3 and test_wait4 that I will submit once this PR is merged. These tests require more work and so I prefer to submit a separated PR.

bedevere-bot · 2020-03-31T18:46:48Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Debian root 3.x has failed when building commit 278c1e1.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/225/builds/638) and take a look at the build logs.
Check if the failure is related to this commit (278c1e1) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/225/builds/638

Failed tests:

test_builtin

Failed subtests:

test_input_no_stdout_fileno - test.test_builtin.PtyTests

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

399 tests OK.

10 slowest tests:

test_concurrent_futures: 3 min 26 sec
test_multiprocessing_spawn: 2 min 40 sec
test_multiprocessing_forkserver: 1 min 45 sec
test_capi: 1 min 26 sec
test_tokenize: 1 min 25 sec
test_unparse: 1 min 19 sec
test_asyncio: 1 min 12 sec
test_multiprocessing_fork: 1 min 9 sec
test_signal: 1 min 2 sec
test_lib2to3: 1 min

1 test failed:
test_builtin

20 tests skipped:
test_devpoll test_gdb test_idle test_ioctl test_kqueue test_msilib
test_ossaudiodev test_smtpnet test_ssl test_startfile test_tcl
test_tix test_tk test_ttk_guionly test_ttk_textonly test_turtle
test_winconsoleio test_winreg test_winsound test_zipfile64

1 re-run test:
test_builtin

Total duration: 18 min 50 sec

Click to see traceback logs

Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_builtin.py", line 1945, in test_input_no_stdout_fileno
    lines = self.run_child(child, b"quux\r")
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_builtin.py", line 1894, in run_child
    support.wait_process(pid, exitcode=0)
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/support/__init__.py", line 3456, in wait_process
    raise AssertionError(f"process {pid} exited with code {exitcode2}, "
AssertionError: process 4269 exited with code -1, but exit code 0 is expected


Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_builtin.py", line 1945, in test_input_no_stdout_fileno
    lines = self.run_child(child, b"quux\r")
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/test_builtin.py", line 1894, in run_child
    support.wait_process(pid, exitcode=0)
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/support/__init__.py", line 3456, in wait_process
    raise AssertionError(f"process {pid} exited with code {exitcode2}, "
AssertionError: process 24194 exited with code -1, but exit code 0 is expected

bedevere-bot · 2020-03-31T19:17:56Z

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Ubuntu Shared 3.x has failed when building commit 278c1e1.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/101/builds/621) and take a look at the build logs.
Check if the failure is related to this commit (278c1e1) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/101/builds/621

Failed tests:

test_builtin

Failed subtests:

test_input_no_stdout_fileno - test.test_builtin.PtyTests

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

402 tests OK.

10 slowest tests:

test_multiprocessing_spawn: 4 min 26 sec
test_concurrent_futures: 4 min 16 sec
test_gdb: 3 min 27 sec
test_unparse: 3 min 13 sec
test_capi: 2 min 56 sec
test_tokenize: 2 min 50 sec
test_multiprocessing_forkserver: 2 min 18 sec
test_asyncio: 2 min 16 sec
test_lib2to3: 2 min 6 sec
test_multiprocessing_fork: 1 min 22 sec

1 test failed:
test_builtin

17 tests skipped:
test_devpoll test_idle test_ioctl test_kqueue test_msilib
test_ossaudiodev test_startfile test_tcl test_tix test_tk
test_ttk_guionly test_ttk_textonly test_turtle test_winconsoleio
test_winreg test_winsound test_zipfile64

1 re-run test:
test_builtin

Total duration: 30 min 4 sec

Click to see traceback logs

Traceback (most recent call last):
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_builtin.py", line 1945, in test_input_no_stdout_fileno
    lines = self.run_child(child, b"quux\r")
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_builtin.py", line 1894, in run_child
    support.wait_process(pid, exitcode=0)
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/support/__init__.py", line 3456, in wait_process
    raise AssertionError(f"process {pid} exited with code {exitcode2}, "
AssertionError: process 5810 exited with code -1, but exit code 0 is expected


Traceback (most recent call last):
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_builtin.py", line 1945, in test_input_no_stdout_fileno
    lines = self.run_child(child, b"quux\r")
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_builtin.py", line 1894, in run_child
    support.wait_process(pid, exitcode=0)
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/support/__init__.py", line 3456, in wait_process
    raise AssertionError(f"process {pid} exited with code {exitcode2}, "
AssertionError: process 3463 exited with code -1, but exit code 0 is expected

giampaolo · 2020-05-01T19:28:25Z

I wish to suggest a change. Sometimes I just want to wait for a PID, without checking its exit status (this is a common use case in psutil test suite). As such I would remove exitcode arg and move the assertion check in the unit-tests. It seems to me like an unnecessary complication of the function API.

What do you think?

Also, subprocess module on Windows uses WaitForSingleObject + GetExitCodeProcess (I do the same in psutil) while here you use os.waitpid(). I'm not sure about the differences in this specific context but... are you sure it serves the intended purpose (I don't)?

vstinner · 2020-05-01T21:14:43Z

I suggest to modify the function to return the exitcode, and accept exitcode=None parameter: in this case, ignore the exitcode.

giampaolo · 2020-05-01T21:47:46Z

Sounds good to me.

giampaolo · 2020-05-01T22:04:49Z

I can make a PR (or up to you, either way am fine).

vstinner · 2020-05-04T15:10:32Z

I can make a PR (or up to you, either way am fine).

I don't need wait_process(exitcode=None), so go ahead if you want to write it :-)

vstinner · 2020-05-04T15:14:24Z

Also, subprocess module on Windows uses WaitForSingleObject + GetExitCodeProcess (I do the same in psutil) while here you use os.waitpid(). I'm not sure about the differences in this specific context but... are you sure it serves the intended purpose (I don't)?

wait_process() code is a refactoring of existing code which used os.waitpid(): see commit 278c1e1. Currently, wait_process() uses os.waitpid() on Windows which means _cwait(). os.waitpid() shifts left the status by 8 bits, os.waitstatus_to_exitcode() shifts it right by 8 bits back.

It would be interesting to rewrite wait_process() with WaitForSingleObject() but I was just lazy to implement it. For example, timeout is ignored (not supported) on Windows.

Go ahead if you want to enhance the Windows implementation.

bpo-40094: Add test.support.wait_process()

5581d33

Moreover, the following tests now check the child process exit code: * test_os.PtyTests * test_mailbox.test_lock_conflict() * test_tempfile.test_process_awareness() * test_uuid.testIssue8621() * multiprocessing resource tracker tests

vstinner requested review from gpshead, rhettinger, tiran, vsajip and a team as code owners March 31, 2020 17:05

the-knights-who-say-ni added the CLA signed label Mar 31, 2020

bedevere-bot added the awaiting core review label Mar 31, 2020

vstinner mentioned this pull request Mar 31, 2020

[WIP] bpo-40068: test_reinit_tls_after_fork() detects crashes #19200

Closed

vstinner added 2 commits March 31, 2020 19:39

timeout is not available on Windows

36df3c7

Fix test_socketserver

233869c

vstinner merged commit 278c1e1 into python:master Mar 31, 2020

bedevere-bot removed the awaiting core review label Mar 31, 2020

vstinner deleted the support_wait_process branch March 31, 2020 18:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

bpo-40094: Add test.support.wait_process() #19254

bpo-40094: Add test.support.wait_process() #19254

Uh oh!

vstinner commented Mar 31, 2020 •

edited by bedevere-bot

Loading

Uh oh!

vstinner commented Mar 31, 2020

Uh oh!

bedevere-bot commented Mar 31, 2020

Uh oh!

bedevere-bot commented Mar 31, 2020

Uh oh!

giampaolo commented May 1, 2020 •

edited

Loading

Uh oh!

vstinner commented May 1, 2020

Uh oh!

giampaolo commented May 1, 2020

Uh oh!

giampaolo commented May 1, 2020

Uh oh!

vstinner commented May 4, 2020

Uh oh!

vstinner commented May 4, 2020

Uh oh!

Uh oh!

Uh oh!

bpo-40094: Add test.support.wait_process() #19254

bpo-40094: Add test.support.wait_process() #19254

Uh oh!

Conversation

vstinner commented Mar 31, 2020 • edited by bedevere-bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vstinner commented Mar 31, 2020

Uh oh!

bedevere-bot commented Mar 31, 2020

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

bedevere-bot commented Mar 31, 2020

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

giampaolo commented May 1, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vstinner commented May 1, 2020

Uh oh!

giampaolo commented May 1, 2020

Uh oh!

giampaolo commented May 1, 2020

Uh oh!

vstinner commented May 4, 2020

Uh oh!

vstinner commented May 4, 2020

Uh oh!

Uh oh!

vstinner commented Mar 31, 2020 •

edited by bedevere-bot

Loading

giampaolo commented May 1, 2020 •

edited

Loading