Async usage with new websockets and http async transports using asyncio #70

leszekhanusz · 2020-03-25T14:46:20Z

Necessitate Python 3.6 and asyncio

New Client

can be used as before synchronously

client.execute(query)

but can also be used asynchronously

async with Client(...) as session:
    await session.execute(query)

supports existing sync http transport (requests) (only synchronously)
supports also new async transports (websockets and aiohttp)
Supports subscriptions with the websockets transport
synchronously with an infinite python generator

for result in client.subscribe(query):
    ... result is a dict here

OR asynchronously with an async generator.

async with Client(...) as session:
    async for result in session.subscribe(query):
        ... result is a dict here

for async transports, fetching schema from transport is done when executing the first query

New Websockets async transport

Allows a connection to an Apollo GraphQL websocket endpoint using the apollo subscriptions-transport-ws protocol

Require the great websockets library

Supports:

queries, mutations AND subscriptions
SSL encryption
multiple queries using the same connection
multiple subscription in parallel on the same connection using asyncio tasks

New HTTP async transport

Supports thanks to the aiohttp library:

http headers
http cookies
Basic auth
SSL

gql-cli

Added gql-cli script to execute graphql queries from the command line to an http or websocket endpoint.

Necessitate Python 3.6 Allows a connection to an Apollo GraphQL websocket endpoint Supports fetching schema from transport using the introspection query Supports queries, mutations AND subscriptions Supports connection using a client ssl certificate Only one request per connection for now

coveralls · 2020-03-25T14:51:17Z

Coverage decreased (-98.8%) to 0.0% when pulling e8c46b1 on leszekhanusz:master into d49bd63 on graphql-python:master.

It is now possible to execute multiple queries in parallel using the same websocket connection

…om the command line

yiblet · 2020-04-01T21:53:58Z

thanks for doing this work, I was looking exactly for these features!

KingDarBoja · 2020-04-06T23:34:44Z

Been busy due to real-life stuff but will look into this PR as soon as possible.

setup.py

Adding fixtures to create a websocket server for the tests Allow ssl=False argument in transport

Adding exceptions: - TransportProtocolError - TransportQueryError - TransportServerError - TransportClosed Now 100% code coverage for transport/websockets.py Improvement of 'server' test fixture to simplify tests (list only answers)

KingDarBoja · 2020-04-13T21:56:26Z

This is looking great, of course requires several reviews from the graphql-python team but glad to see this effort 🚀

Also, I think its a good idea to link resources at the PR body, like apollo subscriptions-transport-ws as example.

Added _wait_closed event and wait_close coroutine to wait for the close task to finish Add exception 'TransportAlreadyConnected' used when trying to connect to an already connected transport Added new tests to test multiple connections in series or in parallel Refactor: - rename _close to _fail - rename _answer_loop to _receive_data_loop - rename listen_loop to receive_data_task

Testing validation using schema, type_def or introspection Fixing some flake8/black styling issues in tests_py36

This allows us to keep the execute method sync in the AsyncClient class Rename client to session in all tests It is now possible to execute GraphQL queries synchronously on asynchronous transports Returning ExecutionResult.data instead of ExecutionResult in AsyncClient (This corresponds to the previous usage of the library)

Now client.execute will use the existing asyncio loop if using an async transport.

Rename AsyncClient to Client in all tests Put back the LocalSchemaTransport in the AsyncClient class All the old tests are passing with the AsyncClient as the Client in python > 3.6 retries on the client is now removed

…n server url Adding yarl dependency (already in aiohttp...)

Fix AIOHTTPTransport exceptions Adding asserts to ensure client.execute and client.subscribe are not called with a running asyncio event loop

Removing pytest_aiohttp dependency (copy only the fixture we need)

Fixing import order with isort Modify isort config to be compatible with black Now isort will fail if invalid import order is found isort upgraded to version 4.2.8 to fix bug PyCQA/isort#537 Fix posargs in tox.ini to allow to run tox for specific tests

Add -s to pytest to have better logs in tox Add -vb (verbose) to isort to have better logs in tox

For some reason on pypy3 with the version on travis (3.6.1), it takes 10 seconds to leave the async generator... So we increase the default timeout for a sync execute

Add execute_timeout to AsyncClient Add connect_timeout, ack_timeout and close_timeout to websockets transport Fix asyncio bugs of pypy3 v3.6.1 by adding await generator.aclose() in async for

Now flake8 also verify the tests_py36 folder

Cito · 2020-05-15T11:56:56Z

Thanks again for this contribution. After creating a v2.x branch for the legacy support, and declaring the master to be the modern v3 branch, I merged this into master, and after fixing a small merge problem, tests are green. I will do some clean-up and modernization work in the master branch now.

Add missing 'pathlib' dependency

159a35c

This was referenced Mar 25, 2020

Are websockets supported? #42

Closed

Subscription - how consume subscription from GraphQL-Yoga API ? #37

Closed

Now using AsyncClient and AsyncTransport classes

d65c697

It is now possible to execute multiple queries in parallel using the same websocket connection

leszekhanusz mentioned this pull request Mar 26, 2020

Roadmap getting us from legacy to modern version #62

Closed

Adding gql-cli.py to allow to send queries to a websocket endpoint fr…

81b7950

…om the command line

leszekhanusz added 6 commits April 3, 2020 18:26

Better management of ConnectionClosed Exceptions

ae1ac36

Fix dependencies

ada2d6c

Using black to format code

2218850

Fix flake8 style enforcement

0a6863b

Adding typing hints

ff86744

Adding gql-cli as a python script + added to MANIFEST

2398866

KingDarBoja reviewed Apr 6, 2020

View reviewed changes

setup.py Outdated Show resolved Hide resolved

leszekhanusz added 9 commits April 7, 2020 10:52

setup.py adding websockets dependency only if python version is >= 3.6

3e314b8

Moving python3 code in separate files to try to fix tests for python2

00f2433

Fix typo in setup.py

4197748

Using relative imports to fix mypy

930ae7e

Using relative imports to fix mypy - try 2

55807c2

Better management of cleanup + adding some tests

58aa38a

Adding fixtures to create a websocket server for the tests Allow ssl=False argument in transport

Fix asyncio.wait_for for pypy3 ?

ece46e8

Fix asyncio.wait_for for pypy3 ? - try 2

6c90378

leszekhanusz added 3 commits April 19, 2020 16:13

TESTS_PY36 add tests for async client validation

a329324

Testing validation using schema, type_def or introspection Fixing some flake8/black styling issues in tests_py36

Merge branch 'master' into master

cad804f

leszekhanusz added 18 commits May 9, 2020 14:29

MAKEFILE add clean target

51df204

Adding subscribe generator (not async) to AsyncClient

789083f

Now client.execute will use the existing asyncio loop if using an async transport.

Using AsyncClient as the gql.Client if python>3.6

41645be

Rename AsyncClient to Client in all tests Put back the LocalSchemaTransport in the AsyncClient class All the old tests are passing with the AsyncClient as the Client in python > 3.6 retries on the client is now removed

Update gql-cli script to use http or websockets transport depending o…

8d277c5

…n server url Adding yarl dependency (already in aiohttp...)

Adding tests for AIOHTTPTransport

5ff88a7

Fix AIOHTTPTransport exceptions Adding asserts to ensure client.execute and client.subscribe are not called with a running asyncio event loop

Fix aiohttp tests

7faa6a2

Removing pytest_aiohttp dependency (copy only the fixture we need)

Fix isort/black compatibility

c3a5c18

Fixing import order with isort Modify isort config to be compatible with black Now isort will fail if invalid import order is found isort upgraded to version 4.2.8 to fix bug PyCQA/isort#537 Fix posargs in tox.ini to allow to run tox for specific tests

Fix test_websocket_exception on pypy3 with python 3.6.1

6250068

Add -s to pytest to have better logs in tox Add -vb (verbose) to isort to have better logs in tox

TRAVIS modify timeout factor to 100 for travis automated tests

a9684b2

Increase timeout for pypy3 version 3.6.1

f80243f

For some reason on pypy3 with the version on travis (3.6.1), it takes 10 seconds to leave the async generator... So we increase the default timeout for a sync execute

Better management of timeouts

77319a1

Add execute_timeout to AsyncClient Add connect_timeout, ack_timeout and close_timeout to websockets transport Fix asyncio bugs of pypy3 v3.6.1 by adding await generator.aclose() in async for

Set graphql-core dependency for now to 2.3.1 to fix tests

8cae234

tox.ini add --diff to isort to easily see where the problem is

6c29f39

setup.cfg adding ssl to known_standard_library for isort

514cce7

Merge branch 'master' of https://github.com/graphql-python/gql

33b6aa5

Merge branch 'master' of https://github.com/graphql-python/gql

cf7b5c6

README.md add sync client.subscribe usage + rename AsyncClient to Client

6298125

leszekhanusz changed the title ~~Implementation of a websockets transport using asyncio~~ Async usage with new websockets and http async transports using asyncio May 13, 2020

leszekhanusz added 2 commits May 13, 2020 16:13

Moving pytest fixtures to conftest.py

6d0a85d

Now flake8 also verify the tests_py36 folder

tox.ini trying to combine coverage of python2.7 and 3.8

9eef3fe

leszekhanusz mentioned this pull request May 13, 2020

[feature] is there any plan for a cli? #69

Closed

leszekhanusz added 2 commits May 14, 2020 12:18

Remove support for python versions < 3.6

d183573

.travis.yml: stop testing on python 2.7, 3.5 and pypy

564065b

leszekhanusz mentioned this pull request May 14, 2020

how to upload file #68

Closed

Cito added the enhancement label May 15, 2020

Cito added this to the Version 3 milestone May 15, 2020

Merge branch 'master' into master

e8c46b1

Cito merged commit 6d961b7 into graphql-python:master May 15, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Async usage with new websockets and http async transports using asyncio #70

Async usage with new websockets and http async transports using asyncio #70

Uh oh!

leszekhanusz commented Mar 25, 2020 •

edited

Loading

Uh oh!

coveralls commented Mar 25, 2020 •

edited

Loading

Uh oh!

yiblet commented Apr 1, 2020

Uh oh!

KingDarBoja commented Apr 6, 2020

Uh oh!

Uh oh!

KingDarBoja commented Apr 13, 2020 •

edited

Loading

Uh oh!

Cito commented May 15, 2020

Uh oh!

Uh oh!

Async usage with new websockets and http async transports using asyncio #70

Async usage with new websockets and http async transports using asyncio #70

Uh oh!

Conversation

leszekhanusz commented Mar 25, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

New Client

New Websockets async transport

New HTTP async transport

gql-cli

Uh oh!

coveralls commented Mar 25, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

yiblet commented Apr 1, 2020

Uh oh!

KingDarBoja commented Apr 6, 2020

Uh oh!

Uh oh!

KingDarBoja commented Apr 13, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Cito commented May 15, 2020

Uh oh!

Uh oh!

leszekhanusz commented Mar 25, 2020 •

edited

Loading

coveralls commented Mar 25, 2020 •

edited

Loading

KingDarBoja commented Apr 13, 2020 •

edited

Loading