Skip to content

ProcessPoolExecutor executor manager thread crashes if task fails in some circumstances #119592

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
batterseapower opened this issue May 27, 2024 · 0 comments
Labels
type-bug An unexpected behavior, bug, or error

Comments

@batterseapower
Copy link

batterseapower commented May 27, 2024

Bug report

Bug description:

import time
from concurrent.futures import ProcessPoolExecutor

def fail():
    time.sleep(1)
    raise ValueError()

executor = ProcessPoolExecutor(max_workers=1, max_tasks_per_child=1)
try:
    print(executor.submit(fail).exception() is not None)
finally:
    executor.shutdown(wait=False)

If you run this, then the following is printed to the console:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "python3.11/threading.py", line 1045, in _bootstrap_inner
    self.run()
  File "python3.11/concurrent/futures/process.py", line 364, in run
    executor._adjust_process_count()
  File "python3.11/concurrent/futures/process.py", line 759, in _adjust_process_count
    process_count = len(self._processes)
                    ^^^^^^^^^^^^^^^^^^^^
TypeError: object of type 'NoneType' has no len()

If you change the code to using wait=True, or remove the max_tasks_per_child=1 option then the message is not printed.

Probably the fix is just to check if self.processes is None before accessing it at https://github.com/python/cpython/blob/main/Lib/concurrent/futures/process.py#L762

CPython versions tested on:

3.11

Operating systems tested on:

Linux

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

1 participant