Skip to content

Concurrent futures: max_task_per_child doesn't work #115831

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
tiedl25 opened this issue Feb 22, 2024 · 1 comment
Closed

Concurrent futures: max_task_per_child doesn't work #115831

tiedl25 opened this issue Feb 22, 2024 · 1 comment
Labels
topic-multiprocessing type-bug An unexpected behavior, bug, or error

Comments

@tiedl25
Copy link

tiedl25 commented Feb 22, 2024

Bug report

Bug description:

If I set a value for max_tasks_per_child in the ProcessPoolExecutor, there are no new processes created.
After a process has finished the tasks, it is killed, but instead of a new process is created nothing happens and the application is stuck.
I found a temporary solution: Add self._broken as condition in the following function in process.py

    def _adjust_process_count(self):
        # if there's an idle process, we don't need to spawn a new one.
        if self._idle_worker_semaphore.acquire(blocking=False) and self._broken:
            return

        process_count = len(self._processes)
        if process_count < self._max_workers:
            # Assertion disabled as this codepath is also used to replace a
            # worker that unexpectedly dies, even when using the 'fork' start
            # method. That means there is still a potential deadlock bug. If a
            # 'fork' mp_context worker dies, we'll be forking a new one when
            # we know a thread is running (self._executor_manager_thread).
            #assert self._safe_to_dynamically_spawn_children or not self._executor_manager_thread, 'https://github.com/python/cpython/issues/90622'
            self._spawn_process()

CPython versions tested on:

3.11

Operating systems tested on:

Linux

@tiedl25 tiedl25 added the type-bug An unexpected behavior, bug, or error label Feb 22, 2024
@gaogaotiantian
Copy link
Member

dup #115634 and will be fixed by #115642

@AlexWaygood AlexWaygood closed this as not planned Won't fix, can't repro, duplicate, stale Feb 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic-multiprocessing type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

3 participants