[Python] Python 3.7 typing module change breaks util.py

[cassinaooo]

Description

Thanks for the nice tool =)

Despite the requirements showing Python 3.5.2+, running swagger generated flask+connexion code with python 3.7 will break util.py.

Here's a stack trace:

-------------------- >> begin captured logging << --------------------
flask.app: ERROR: Exception on /v1/orders/1000/invoices [PUT]
Traceback (most recent call last):
  File "/home/lightbringer/scudra/scudra-backend-api/.tox/py37/lib/python3.7/site-packages/flask/app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/lightbringer/scudra/scudra-backend-api/.tox/py37/lib/python3.7/site-packages/flask/app.py", line 1815, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/lightbringer/scudra/scudra-backend-api/.tox/py37/lib/python3.7/site-packages/flask/app.py", line 1718, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/lightbringer/scudra/scudra-backend-api/.tox/py37/lib/python3.7/site-packages/flask/_compat.py", line 35, in reraise
    raise value
  File "/home/lightbringer/scudra/scudra-backend-api/.tox/py37/lib/python3.7/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/lightbringer/scudra/scudra-backend-api/.tox/py37/lib/python3.7/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/lightbringer/scudra/scudra-backend-api/.tox/py37/lib/python3.7/site-packages/connexion/decorators/decorator.py", line 66, in wrapper
    response = function(request)
  File "/home/lightbringer/scudra/scudra-backend-api/.tox/py37/lib/python3.7/site-packages/connexion/decorators/validation.py", line 122, in wrapper
    response = function(request)
  File "/home/lightbringer/scudra/scudra-backend-api/.tox/py37/lib/python3.7/site-packages/connexion/decorators/validation.py", line 293, in wrapper
    return function(request)
  File "/home/lightbringer/scudra/scudra-backend-api/.tox/py37/lib/python3.7/site-packages/connexion/decorators/decorator.py", line 42, in wrapper
    response = function(request)
  File "/home/lightbringer/scudra/scudra-backend-api/.tox/py37/lib/python3.7/site-packages/connexion/decorators/parameter.py", line 218, in wrapper
    return function(**kwargs)
  File "/home/lightbringer/scudra/scudra-backend-api/api/controllers/orders_controller.py", line 64, in orders_order_id_invoices_put
    api_invoice = Invoice.from_dict(connexion.request.get_json())  # noqa: E501
  File "/home/lightbringer/scudra/scudra-backend-api/api/models/invoice.py", line 69, in from_dict
    return util.deserialize_model(dikt, cls)
  File "/home/lightbringer/scudra/scudra-backend-api/api/util.py", line 110, in deserialize_model
    setattr(instance, attr, _deserialize(value, attr_type))
  File "/home/lightbringer/scudra/scudra-backend-api/api/util.py", line 25, in _deserialize
    elif type(klass) == typing.GenericMeta:
AttributeError: module 'typing' has no attribute 'GenericMeta'
--------------------- >> end captured logging << ---------------------

This is because the typing module does not have a GenericMeta anymore. I got across this as I tried to serialize some typed generated models as part of trying to parallelize a specific task. This got me here:
python/typing#511, where I learned that I needed to upgrade to python 3.7 in order to pickle these objects. The pickle part started working just fine. However, the error above appeared.

After some google-fu I found this answer in stack overflow and worked around it for my project changing

elif type(klass) == typing.GenericMeta:
    if klass.__extra__ == list:
        return _deserialize_list(data, klass.__args__[0])
    if klass.__extra__ == dict:
        return _deserialize_dict(data, klass.__args__[1])

In util.py , to

elif hasattr(klass, '__origin__'):
    if klass.__origin__ == list:
        return _deserialize_list(data, klass.__args__[0])
    if klass.__origin__ == dict:
        return _deserialize_dict(data, klass.__args__[1])

Unfortunately, this fix is not backwards compatible with < 3.7.

The guys from sphinx used a more generic solution where they checked for python version before acessing GenericMeta or __origin__. See commit tk0miya/sphinx@e2389b4, fixing issue sphinx-doc/sphinx#4490.

I first sent this to the connexion guys given that someone had already opened a similar issue there spec-first/connexion#739, but I think this is the correct place.

I will gladly send a PR if someone guide me through it. As per gvanrossum (python/typing#136), we should not be using __extra__ anyway.

Edit: In a second read, it seems that we are not supposed to do runtime type checks with the typing module (????), and that we should not be using private APIs for this, even though there is no public APIs (????).

Swagger-codegen version

2.3.1

Steps to reproduce

Run any code (including sample) with python 3.7

[kcris]

Hi,
got the same issue, the generated util.py does not work with python 3.7.1

[2018-11-29 16:02:45,794] ERROR in app: Exception on /v1/event [POST]
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1982, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1614, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1517, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1612, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1598, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/lib/python3.7/site-packages/connexion/decorators/decorator.py", line 66, in wrapper
    response = function(request)
  File "/usr/local/lib/python3.7/site-packages/connexion/decorators/validation.py", line 122, in wrapper
    response = function(request)
  File "/usr/local/lib/python3.7/site-packages/connexion/decorators/validation.py", line 293, in wrapper
    return function(request)
  File "/usr/local/lib/python3.7/site-packages/connexion/decorators/decorator.py", line 42, in wrapper
    response = function(request)
  File "/usr/local/lib/python3.7/site-packages/connexion/decorators/parameter.py", line 218, in wrapper
    return function(**kwargs)
  File "/usr/src/app/swagger_server/controllers/openstack_event_controller.py", line 70, in add_event
    event = Event.from_dict(d)
  File "/usr/src/app/swagger_server/models/event.py", line 64, in from_dict
    return util.deserialize_model(dikt, cls)
  File "/usr/src/app/swagger_server/util.py", line 111, in deserialize_model
    setattr(instance, attr, _deserialize(value, attr_type))
  File "/usr/src/app/swagger_server/util.py", line 26, in _deserialize
    elif type(klass) == typing.GenericMeta:
AttributeError: module 'typing' has no attribute 'GenericMeta'

As far as I understand, GenericMeta does not exist anymore in python 3.7
hsolbrig/PyShEx#17 (comment)

All valid requests to the generated server will result in this error.

Thanks

[ElMoselYEE]

This is affecting our development as well--we love the codegen and use it throughout numerous apps, but because of this issue we're either forced to use python 3.6 or have to modify the generated code programmatically.

[ratijas]

Hi,

I've just came across this issue as well. Any progress on this? Should it be closed as fixed?

I'm just asking. I haven't tried running latest codegen — only using what Swagger hub has to offer.

[tarehart]

Same here, I had this issue when generating with swagger-codegen-cli-3.0.20.jar. I dealt with it by pasting in the fix from https://github.com/OpenAPITools/openapi-generator/pull/2884/files and adding util.py to my .swagger-codegen-ignore.

[add-n2x]

Hey there, is there any news on this?

I'm affected by the same problem using the Python-Flask generated stubs by Swaggerhub.

[becothas]

@HugoMario I just had this issue again.
I downloaded the generated python-flask code from Swagger Editor with the same issue.
And also the 'swagger-codegen'-repo still has the util file from before the fix.

[HugoMario]

It seems this was fixed with #11742 for codegen v2

On codegen v3, i tried to reproduce this issue but everything worked fine for me.

@becothas if you are familiar with codegen cli. Can you please check with this codegen v2 snapshot

or check using codegen v3 snapshot in case you work with an OpenAPI 3.0.X definition