Skip to content

Commit 9492f33

Browse files
committed
backport ThreadedChildWatcher
1 parent de2b3d3 commit 9492f33

File tree

6 files changed

+403
-16
lines changed

6 files changed

+403
-16
lines changed

meta.yaml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,6 @@ requirements:
2626
- greenlet ==1.1.2
2727
- pyee ==8.1.0
2828
- websockets ==10.1
29-
- psutil ==5.9.0 # [py<38]
3029
- typing_extensions # [py<39]
3130
test:
3231
requires:

playwright/_impl/_connection.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -187,6 +187,7 @@ async def init() -> None:
187187
def stop_sync(self) -> None:
188188
self._transport.request_stop()
189189
self._dispatcher_fiber.switch()
190+
self._loop.run_until_complete(self._transport.wait_until_stopped())
190191
self.cleanup()
191192

192193
async def stop_async(self) -> None:
Lines changed: 377 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,377 @@
1+
# This is the LICENSE from the Python project:
2+
# A. HISTORY OF THE SOFTWARE
3+
# ==========================
4+
#
5+
# Python was created in the early 1990s by Guido van Rossum at Stichting
6+
# Mathematisch Centrum (CWI, see http://www.cwi.nl) in the Netherlands
7+
# as a successor of a language called ABC. Guido remains Python's
8+
# principal author, although it includes many contributions from others.
9+
#
10+
# In 1995, Guido continued his work on Python at the Corporation for
11+
# National Research Initiatives (CNRI, see http://www.cnri.reston.va.us)
12+
# in Reston, Virginia where he released several versions of the
13+
# software.
14+
#
15+
# In May 2000, Guido and the Python core development team moved to
16+
# BeOpen.com to form the BeOpen PythonLabs team. In October of the same
17+
# year, the PythonLabs team moved to Digital Creations, which became
18+
# Zope Corporation. In 2001, the Python Software Foundation (PSF, see
19+
# https://www.python.org/psf/) was formed, a non-profit organization
20+
# created specifically to own Python-related Intellectual Property.
21+
# Zope Corporation was a sponsoring member of the PSF.
22+
#
23+
# All Python releases are Open Source (see http://www.opensource.org for
24+
# the Open Source Definition). Historically, most, but not all, Python
25+
# releases have also been GPL-compatible; the table below summarizes
26+
# the various releases.
27+
#
28+
# Release Derived Year Owner GPL-
29+
# from compatible? (1)
30+
#
31+
# 0.9.0 thru 1.2 1991-1995 CWI yes
32+
# 1.3 thru 1.5.2 1.2 1995-1999 CNRI yes
33+
# 1.6 1.5.2 2000 CNRI no
34+
# 2.0 1.6 2000 BeOpen.com no
35+
# 1.6.1 1.6 2001 CNRI yes (2)
36+
# 2.1 2.0+1.6.1 2001 PSF no
37+
# 2.0.1 2.0+1.6.1 2001 PSF yes
38+
# 2.1.1 2.1+2.0.1 2001 PSF yes
39+
# 2.1.2 2.1.1 2002 PSF yes
40+
# 2.1.3 2.1.2 2002 PSF yes
41+
# 2.2 and above 2.1.1 2001-now PSF yes
42+
#
43+
# Footnotes:
44+
#
45+
# (1) GPL-compatible doesn't mean that we're distributing Python under
46+
# the GPL. All Python licenses, unlike the GPL, let you distribute
47+
# a modified version without making your changes open source. The
48+
# GPL-compatible licenses make it possible to combine Python with
49+
# other software that is released under the GPL; the others don't.
50+
#
51+
# (2) According to Richard Stallman, 1.6.1 is not GPL-compatible,
52+
# because its license has a choice of law clause. According to
53+
# CNRI, however, Stallman's lawyer has told CNRI's lawyer that 1.6.1
54+
# is "not incompatible" with the GPL.
55+
#
56+
# Thanks to the many outside volunteers who have worked under Guido's
57+
# direction to make these releases possible.
58+
#
59+
#
60+
# B. TERMS AND CONDITIONS FOR ACCESSING OR OTHERWISE USING PYTHON
61+
# ===============================================================
62+
#
63+
# PYTHON SOFTWARE FOUNDATION LICENSE VERSION 2
64+
# --------------------------------------------
65+
#
66+
# 1. This LICENSE AGREEMENT is between the Python Software Foundation
67+
# ("PSF"), and the Individual or Organization ("Licensee") accessing and
68+
# otherwise using this software ("Python") in source or binary form and
69+
# its associated documentation.
70+
#
71+
# 2. Subject to the terms and conditions of this License Agreement, PSF hereby
72+
# grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce,
73+
# analyze, test, perform and/or display publicly, prepare derivative works,
74+
# distribute, and otherwise use Python alone or in any derivative version,
75+
# provided, however, that PSF's License Agreement and PSF's notice of copyright,
76+
# i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010,
77+
# 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020 Python Software Foundation;
78+
# All Rights Reserved" are retained in Python alone or in any derivative version
79+
# prepared by Licensee.
80+
#
81+
# 3. In the event Licensee prepares a derivative work that is based on
82+
# or incorporates Python or any part thereof, and wants to make
83+
# the derivative work available to others as provided herein, then
84+
# Licensee hereby agrees to include in any such work a brief summary of
85+
# the changes made to Python.
86+
#
87+
# 4. PSF is making Python available to Licensee on an "AS IS"
88+
# basis. PSF MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
89+
# IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, PSF MAKES NO AND
90+
# DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
91+
# FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON WILL NOT
92+
# INFRINGE ANY THIRD PARTY RIGHTS.
93+
#
94+
# 5. PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON
95+
# FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS
96+
# A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON,
97+
# OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
98+
#
99+
# 6. This License Agreement will automatically terminate upon a material
100+
# breach of its terms and conditions.
101+
#
102+
# 7. Nothing in this License Agreement shall be deemed to create any
103+
# relationship of agency, partnership, or joint venture between PSF and
104+
# Licensee. This License Agreement does not grant permission to use PSF
105+
# trademarks or trade name in a trademark sense to endorse or promote
106+
# products or services of Licensee, or any third party.
107+
#
108+
# 8. By copying, installing or otherwise using Python, Licensee
109+
# agrees to be bound by the terms and conditions of this License
110+
# Agreement.
111+
#
112+
#
113+
# BEOPEN.COM LICENSE AGREEMENT FOR PYTHON 2.0
114+
# -------------------------------------------
115+
#
116+
# BEOPEN PYTHON OPEN SOURCE LICENSE AGREEMENT VERSION 1
117+
#
118+
# 1. This LICENSE AGREEMENT is between BeOpen.com ("BeOpen"), having an
119+
# office at 160 Saratoga Avenue, Santa Clara, CA 95051, and the
120+
# Individual or Organization ("Licensee") accessing and otherwise using
121+
# this software in source or binary form and its associated
122+
# documentation ("the Software").
123+
#
124+
# 2. Subject to the terms and conditions of this BeOpen Python License
125+
# Agreement, BeOpen hereby grants Licensee a non-exclusive,
126+
# royalty-free, world-wide license to reproduce, analyze, test, perform
127+
# and/or display publicly, prepare derivative works, distribute, and
128+
# otherwise use the Software alone or in any derivative version,
129+
# provided, however, that the BeOpen Python License is retained in the
130+
# Software, alone or in any derivative version prepared by Licensee.
131+
#
132+
# 3. BeOpen is making the Software available to Licensee on an "AS IS"
133+
# basis. BEOPEN MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
134+
# IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, BEOPEN MAKES NO AND
135+
# DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
136+
# FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF THE SOFTWARE WILL NOT
137+
# INFRINGE ANY THIRD PARTY RIGHTS.
138+
#
139+
# 4. BEOPEN SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF THE
140+
# SOFTWARE FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS
141+
# AS A RESULT OF USING, MODIFYING OR DISTRIBUTING THE SOFTWARE, OR ANY
142+
# DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
143+
#
144+
# 5. This License Agreement will automatically terminate upon a material
145+
# breach of its terms and conditions.
146+
#
147+
# 6. This License Agreement shall be governed by and interpreted in all
148+
# respects by the law of the State of California, excluding conflict of
149+
# law provisions. Nothing in this License Agreement shall be deemed to
150+
# create any relationship of agency, partnership, or joint venture
151+
# between BeOpen and Licensee. This License Agreement does not grant
152+
# permission to use BeOpen trademarks or trade names in a trademark
153+
# sense to endorse or promote products or services of Licensee, or any
154+
# third party. As an exception, the "BeOpen Python" logos available at
155+
# http://www.pythonlabs.com/logos.html may be used according to the
156+
# permissions granted on that web page.
157+
#
158+
# 7. By copying, installing or otherwise using the software, Licensee
159+
# agrees to be bound by the terms and conditions of this License
160+
# Agreement.
161+
#
162+
#
163+
# CNRI LICENSE AGREEMENT FOR PYTHON 1.6.1
164+
# ---------------------------------------
165+
#
166+
# 1. This LICENSE AGREEMENT is between the Corporation for National
167+
# Research Initiatives, having an office at 1895 Preston White Drive,
168+
# Reston, VA 20191 ("CNRI"), and the Individual or Organization
169+
# ("Licensee") accessing and otherwise using Python 1.6.1 software in
170+
# source or binary form and its associated documentation.
171+
#
172+
# 2. Subject to the terms and conditions of this License Agreement, CNRI
173+
# hereby grants Licensee a nonexclusive, royalty-free, world-wide
174+
# license to reproduce, analyze, test, perform and/or display publicly,
175+
# prepare derivative works, distribute, and otherwise use Python 1.6.1
176+
# alone or in any derivative version, provided, however, that CNRI's
177+
# License Agreement and CNRI's notice of copyright, i.e., "Copyright (c)
178+
# 1995-2001 Corporation for National Research Initiatives; All Rights
179+
# Reserved" are retained in Python 1.6.1 alone or in any derivative
180+
# version prepared by Licensee. Alternately, in lieu of CNRI's License
181+
# Agreement, Licensee may substitute the following text (omitting the
182+
# quotes): "Python 1.6.1 is made available subject to the terms and
183+
# conditions in CNRI's License Agreement. This Agreement together with
184+
# Python 1.6.1 may be located on the Internet using the following
185+
# unique, persistent identifier (known as a handle): 1895.22/1013. This
186+
# Agreement may also be obtained from a proxy server on the Internet
187+
# using the following URL: http://hdl.handle.net/1895.22/1013".
188+
#
189+
# 3. In the event Licensee prepares a derivative work that is based on
190+
# or incorporates Python 1.6.1 or any part thereof, and wants to make
191+
# the derivative work available to others as provided herein, then
192+
# Licensee hereby agrees to include in any such work a brief summary of
193+
# the changes made to Python 1.6.1.
194+
#
195+
# 4. CNRI is making Python 1.6.1 available to Licensee on an "AS IS"
196+
# basis. CNRI MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
197+
# IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, CNRI MAKES NO AND
198+
# DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
199+
# FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON 1.6.1 WILL NOT
200+
# INFRINGE ANY THIRD PARTY RIGHTS.
201+
#
202+
# 5. CNRI SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON
203+
# 1.6.1 FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS
204+
# A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON 1.6.1,
205+
# OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
206+
#
207+
# 6. This License Agreement will automatically terminate upon a material
208+
# breach of its terms and conditions.
209+
#
210+
# 7. This License Agreement shall be governed by the federal
211+
# intellectual property law of the United States, including without
212+
# limitation the federal copyright law, and, to the extent such
213+
# U.S. federal law does not apply, by the law of the Commonwealth of
214+
# Virginia, excluding Virginia's conflict of law provisions.
215+
# Notwithstanding the foregoing, with regard to derivative works based
216+
# on Python 1.6.1 that incorporate non-separable material that was
217+
# previously distributed under the GNU General Public License (GPL), the
218+
# law of the Commonwealth of Virginia shall govern this License
219+
# Agreement only as to issues arising under or with respect to
220+
# Paragraphs 4, 5, and 7 of this License Agreement. Nothing in this
221+
# License Agreement shall be deemed to create any relationship of
222+
# agency, partnership, or joint venture between CNRI and Licensee. This
223+
# License Agreement does not grant permission to use CNRI trademarks or
224+
# trade name in a trademark sense to endorse or promote products or
225+
# services of Licensee, or any third party.
226+
#
227+
# 8. By clicking on the "ACCEPT" button where indicated, or by copying,
228+
# installing or otherwise using Python 1.6.1, Licensee agrees to be
229+
# bound by the terms and conditions of this License Agreement.
230+
#
231+
# ACCEPT
232+
#
233+
#
234+
# CWI LICENSE AGREEMENT FOR PYTHON 0.9.0 THROUGH 1.2
235+
# --------------------------------------------------
236+
#
237+
# Copyright (c) 1991 - 1995, Stichting Mathematisch Centrum Amsterdam,
238+
# The Netherlands. All rights reserved.
239+
#
240+
# Permission to use, copy, modify, and distribute this software and its
241+
# documentation for any purpose and without fee is hereby granted,
242+
# provided that the above copyright notice appear in all copies and that
243+
# both that copyright notice and this permission notice appear in
244+
# supporting documentation, and that the name of Stichting Mathematisch
245+
# Centrum or CWI not be used in advertising or publicity pertaining to
246+
# distribution of the software without specific, written prior
247+
# permission.
248+
#
249+
# STICHTING MATHEMATISCH CENTRUM DISCLAIMS ALL WARRANTIES WITH REGARD TO
250+
# THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
251+
# FITNESS, IN NO EVENT SHALL STICHTING MATHEMATISCH CENTRUM BE LIABLE
252+
# FOR ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
253+
# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
254+
# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
255+
# OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
256+
257+
# type: ignore
258+
"""
259+
This is a backport of the ThreadedChildWatcher to Python 3.7.
260+
Source: https://github.com/python/cpython/blob/4649202ea75d48e1496e99911709824ca2d3170e/Lib/asyncio/unix_events.py#L1326
261+
"""
262+
263+
import itertools
264+
import os
265+
import threading
266+
import warnings
267+
from asyncio import AbstractChildWatcher, events
268+
from asyncio.log import logger
269+
270+
271+
class ThreadedChildWatcher(AbstractChildWatcher):
272+
"""Threaded child watcher implementation.
273+
The watcher uses a thread per process
274+
for waiting for the process finish.
275+
It doesn't require subscription on POSIX signal
276+
but a thread creation is not free.
277+
The watcher has O(1) complexity, its performance doesn't depend
278+
on amount of spawn processes.
279+
"""
280+
281+
def __init__(self):
282+
self._pid_counter = itertools.count(0)
283+
self._threads = {}
284+
285+
def is_active(self):
286+
return True
287+
288+
def close(self):
289+
self._join_threads()
290+
291+
def _join_threads(self):
292+
"""Internal: Join all non-daemon threads"""
293+
threads = [
294+
thread
295+
for thread in list(self._threads.values())
296+
if thread.is_alive() and not thread.daemon
297+
]
298+
for thread in threads:
299+
thread.join()
300+
301+
def __enter__(self):
302+
return self
303+
304+
def __exit__(self, exc_type, exc_val, exc_tb):
305+
pass
306+
307+
def __del__(self, _warn=warnings.warn):
308+
threads = [
309+
thread for thread in list(self._threads.values()) if thread.is_alive()
310+
]
311+
if threads:
312+
_warn(
313+
f"{self.__class__} has registered but not finished child processes",
314+
ResourceWarning,
315+
source=self,
316+
)
317+
318+
def add_child_handler(self, pid, callback, *args):
319+
loop = events.get_running_loop()
320+
thread = threading.Thread(
321+
target=self._do_waitpid,
322+
name=f"waitpid-{next(self._pid_counter)}",
323+
args=(loop, pid, callback, args),
324+
daemon=True,
325+
)
326+
self._threads[pid] = thread
327+
thread.start()
328+
329+
def remove_child_handler(self, pid):
330+
# asyncio never calls remove_child_handler() !!!
331+
# The method is no-op but is implemented because
332+
# abstract base classe requires it
333+
return True
334+
335+
def attach_loop(self, loop):
336+
pass
337+
338+
def _do_waitpid(self, loop, expected_pid, callback, args):
339+
assert expected_pid > 0
340+
341+
try:
342+
pid, status = os.waitpid(expected_pid, 0)
343+
except ChildProcessError:
344+
# The child process is already reaped
345+
# (may happen if waitpid() is called elsewhere).
346+
pid = expected_pid
347+
returncode = 255
348+
logger.warning(
349+
"Unknown child process pid %d, will report returncode 255", pid
350+
)
351+
else:
352+
returncode = _compute_returncode(status)
353+
if loop.get_debug():
354+
logger.debug(
355+
"process %s exited with returncode %s", expected_pid, returncode
356+
)
357+
358+
if loop.is_closed():
359+
logger.warning("Loop %r that handles pid %r is closed", loop, pid)
360+
else:
361+
loop.call_soon_threadsafe(callback, pid, returncode, *args)
362+
363+
self._threads.pop(expected_pid)
364+
365+
366+
def _compute_returncode(status):
367+
if os.WIFSIGNALED(status):
368+
# The child process died because of a signal.
369+
return -os.WTERMSIG(status)
370+
elif os.WIFEXITED(status):
371+
# The child process exited (e.g sys.exit()).
372+
return os.WEXITSTATUS(status)
373+
else:
374+
# The child exited, but we don't understand its status.
375+
# This shouldn't happen, but if it does, let's just
376+
# return that status; perhaps that helps debug it.
377+
return status

0 commit comments

Comments
 (0)