Incorrect type inferred for **dict(iterable)

[gvanrossum]

This was reported by @ahaven on our internal Slack; @ddfisher came up with the following minimal repro.

from typing import *

def f1(iterable: Iterable[Tuple[str, int]] = None) -> None:
    f2(**dict(iterable))  # <------- Error here

def f2(iterable: Iterable[Tuple[str, int]], **kw: Any) -> None:
    pass

We get the following mysterious error:

star.py: note: In function "f1":
star.py:4: error: Argument 1 to "dict" has incompatible type Iterable[Tuple[str, int]]; expected Iterable[...]

Some debugging shows that Iterable[...] in the error message stands for Iterable[Tuple[Tuple[str, int], <something>], IOW it somehow takes Dict[KT, VT] and substitutes the Tuple[str, int] for just KT, even though the variant it chose from the overloaded dict constructor has iterable: Iterable[Tuple[KT, VT]] as its lone arg.

I think I've followed the trail into infer_type_arguments() at the very end of mypy/infer.py; it gets called with:

type_var_ids: [-1, -2]
template: builtins.dict[_KT`-1, _VT`-2]
actual: typing.Iterable[Tuple[builtins.str, builtins.int]]

After calling infer_constraints(template, actual, SUBTYPE_OF), constraints is set to

[-1 <: Tuple[builtins.str, builtins.int], -1 :> Tuple[builtins.str, builtins.int]]

which seems to tell me that type var -1 (i.e. _KT) is both a subtype and a supertype of Tuple[str, int] while leaving type var -2 (i.e. _VT) unconstrained. The reported error follows from this I think.

I can also understand why these constraints are inferred -- it tries to match Dict[KT, VT] with Iterable[Tuple[...]] and of course Dict[KT, VT] inherits from Iterable[KT] so the tuple type ends up associated with KT -- but it appears confused because we should actually take the argument rather than the return type.

That's all I have for now.

[gvanrossum]

Perhaps also interesting: if I comment out the call

                callee = self.infer_function_type_arguments_using_context(
                    callee, context)

from check_call() in mypy/checkexpr.py, I get a different error message:

star.py: note: In function "f1":
star.py:4: error: Argument 1 to "f2" has incompatible type Dict[str, int]; expected Iterable[Tuple[str, int]]

What's so interesting about this is that I get exactly the same error (still with that infer_ call commented out) if I remove the ** from the call to f2(), i.e. if I change line 4 to

    f2(dict(iterable))

That suggests to me that the original problem is really caused by incorrectly taking the type of the first argument to f2(), which also happens to be Iterable[Tuple[str, int]], for the context of the expression dict(iterable), despite the ** in front of the latter.

[gvanrossum]

Hm... Looks like either infer_arg_types_in_context2() is in need of skipping args of kind ARG_STAR2? Or map_actuals_to_formals() is doing a poor job if kind == ARG_STAR2 (in the block asserting that).

[gvanrossum]

I think map_actuals_to_formals() is correct -- it determines that in a call like f2(**a), both of f2's arguments (iterable and kw) come from a, and for this it returns [[0], [0]]. But something that does something with the mapping is not properly taking the ** in the call site into account.

[gvanrossum]

Here's a simpler example showing that at least part of the problem seems to be the message formatting:

def f(a: int) -> None: pass
a = {'': 0}  # OK (!)
f(a)  # Error
f(**a)
b = {'': ''}
f(b)  # Error
f(**b)  # Error

All three lines with errors give the same error message, even though the reasons are actually different:

x.py:3: error: Argument 1 to "f" has incompatible type Dict[str, int]; expected "int"
x.py:6: error: Argument 1 to "f" has incompatible type Dict[str, str]; expected "int"
x.py:7: error: Argument 1 to "f" has incompatible type Dict[str, str]; expected "int"

But even if we were to fix this, we'd still have the problem that in a call like f2(**f3(args)) the context for checking f3(args) is incorrectly taken to be the type of the first arg of f2() -- it should use Dict[str, X] where X is the type of the first arg of f2(). So there's still more than one bug here.

[gvanrossum]

I strongly believe the correct fix for the incorrect inference context is to add ARG_STAR2 to the kinds excepted in infer_arg_types_in_context2(). I'm going to merge the fix for #1357 so I can focus on this and the confusing error messages.