"error: Too many arguments for" class with generated __init__ method by metaclass

[DevOpsCraftsman]

Simple metaclass that generates an __init__ method with _attrs class attributes:

# meta.py

from abc import ABCMeta

class GenInit(ABCMeta):

    def __new__(cls, name, bases, clsdict):

        attrs = clsdict.get('_attrs')

        if attrs:

            init_sig_template = """
def __init__(self, {attrs}):"""
            init_body_template = """
    self.{attr} = {attr}"""

            sig = init_sig_template.format(attrs=', '.join(attrs))
            body = ""

            for attr in attrs:
                body += init_body_template.format(attr=attr)

            init_code = sig + body

            exec(init_code, globals(), clsdict)

            return type(name, bases, clsdict)

class Point(metaclass=GenInit):
    _attrs = ['x', 'y']

p = Point(0, 0)

Mypy can't go through it:

$ mypy meta.py
meta.py:31: error: Too many arguments for "Point"

Don't know if it's possible to fix, since the init method is executed in the class namespace...
And I wasn't able to get its source code:

>>> from meta import Point
>>> from inspect import getsource
>>> getsource(Point.__init__)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.6/inspect.py", line 965, in getsource
    lines, lnum = getsourcelines(object)
  File "/usr/lib/python3.6/inspect.py", line 952, in getsourcelines
    lines, lnum = findsource(object)
  File "/usr/lib/python3.6/inspect.py", line 783, in findsource
    raise OSError('could not get source code')
OSError: could not get source code 

At list, how can I avoid the error without putting a #type: ignore at the end of every call to __init__ of every class using this trick?

[ilevkivskyi]

This is too dynamic for mypy (and realistically for any other static type checker). Also you might be interested in the dataclasses PEP. Note that dataclases (as the "blessed" stdlib solution) will have good mypy support.

[DevOpsCraftsman]

I was unaware about this solution!
But how will you support it? The __init__ is generated by an exec as well.
Source code here:

def _create_fn(name, args, body, globals=None, locals=None,
               return_type=_MISSING):
    # Note that we mutate locals when exec() is called. Caller beware!
    if locals is None:
        locals = {}
    return_annotation = ''
    if return_type is not _MISSING:
        locals['_return_type'] = return_type
        return_annotation = '->_return_type'
    args = ','.join(args)
    body = '\n'.join(f' {b}' for b in body)

    txt = f'def {name}({args}){return_annotation}:\n{body}'
    #print(txt)
    #print(locals)
    #print(globals)
    exec(txt, globals, locals)
return locals[name]

[ilevkivskyi]

But how will you support it?

This is a chef's secret :-) But seriously, we just know ourselves (not from code, but from design) what will be the __init__ signature for a given dataclass from fields declaration, so we can reconstruct it.

[DevOpsCraftsman]

Ok, this is what I was thinking!!! It's like I'm reading in you... ^_^

But if somebody want use the the dataclass wrapper inside a metaclasse?! Ah!

I say it because I'm developing a data validation library (that instaciates descriptors to enable typechecking at runtime according to the annotations) with a signature that looks like the dataclass wrapper... So I can pass the arguments to it. But it's gonna been use inside a metaclass.

[JukkaL]

In the future we may support extending mypy to understand some custom metaclass magic through user-defined mypy plugins. The plugin interface is still very much in progress, however.

[DevOpsCraftsman]

Ok, interesting... Thanks.

[DevOpsCraftsman]

Other idea: How about inspecting the signature of the function if the body's not found? Is it to heavy for mypy?

For exemple, for the moment whith dataclasses :

@dataclass
class Point:
    x: int
    y: int

p = Point(0, 0)

>>> vars(Point)['__init__']
<function __init__>

>>> import inspect

>>> inspect.signature(Point.__init__)
<Signature (self, x:int, y:int) -> None>

$ mypy test.py
test.py:x: error: Too many arguments for "Point"

Would be simpler by getting the signature, no?

[JukkaL]

Mypy can't inspect the signature of the function since mypy doesn't actually import any type checked code. It only looks at the source code -- and this is not going to change.

[JukkaL]

Closing this issue as out of scope for mypy (besides the potential workarounds discussed above).

[scorphus]

New Python 3.7.0a3 brings dataclass builtin. I wish I could see this issue reopened 🙏

In the meantime, how can I ignore the warnings related to this?

Thanks!

[ilevkivskyi]

@scorphus There will be a dedicated support for data classes. However, other libraries with such functionality (like attrs) will be only supported via mypy's plugin system.