abstractmethod + generic subclass resulting in failed inference

[beezee]

This appears to be a bug.

Minimal repro:

from abc import ABC, abstractmethod
from typing import Generic, TypeVar

A = TypeVar('A')
B = TypeVar('B')
C = TypeVar('C')

class TwoTypes(Generic[A, B]): pass

class ReproBase(ABC, Generic[C]):

  @abstractmethod
  def repro(self, a: A) -> TwoTypes[C, A]: pass

class ReproSub(ReproBase[C]):

  @abstractmethod
  def repro_defer(self, a: A) -> TwoTypes[C, A]: pass

  def repro(self, a: A) -> TwoTypes[C, A]:
    return self.repro_defer(a)

class ReproSub2(ReproBase[C], Generic[C]):

  @abstractmethod
  def repro_defer(self, a: A) -> TwoTypes[C, A]: pass

  def repro(self, a: A) -> TwoTypes[C, A]:
    return self.repro_defer(a)

Type-checking results in error: Incompatible return value type (got "TwoTypes[A, A]", expected "TwoTypes[C, A]")

mypy doesn't like ReproSub or ReproSub2 for the same reason, but interestingly only complains about the first it encounters, depending on which order you put the subclasses in.

I'd expect C to be captured correctly and the return type inferred on any concrete definition of the repro method to be TwoTypes[C, A], as is specified by the delegated defer_repro method in both cases.

Using mypy 0.770, Python 3.7.6

Issue appears to be fixed in master

Not using any flags

Considering it's fixed in master, if someone knows specifically what the fix is, I'd love a link to the commit(s) just to understand what I'm running into on current release, and any info about when the fix might become generally available.

[JukkaL]

This is bizarre. I was able to reduce the repro to this:

from typing import Generic, TypeVar

A = TypeVar('A')
B = TypeVar('B')
C = TypeVar('C')

class TwoTypes(Generic[A, B]): pass

class X(Generic[C]):
    def f(self, a: A) -> TwoTypes[C, A]: pass

class Y(X[C]):
    def g(self, a: A) -> TwoTypes[C, A]: pass

    def f(self, a: A) -> TwoTypes[C, A]:
        return self.g(a)  # Error here

class Z(X[C]):
    def g(self, a: A) -> TwoTypes[C, A]: pass

    def f(self, a: A) -> TwoTypes[C, A]:
        return self.g(a)  # But not here!

If I switch classes Y with Z, the error moves to the other class. My hypothesis is that somehow the textual order of the classes plays a role.

[beezee]

Yeah it's strange. I'd note that it's an error even with only one subclass of X - it seems that only the first encountered is reported.

[beezee]

New information @JukkaL - this is not fixed on master, and it appears related to the cache. Are there different logical pathways for use of cache vs not?

On both master and the current release, I observe the following:

This code typechecks initially after mypy installation (seems to be a hard cache clear here):

from abc import ABC, abstractmethod
from typing import Callable, Generic, TypeVar

A = TypeVar('A')
B = TypeVar('B')
C = TypeVar('C')

class TwoTypes(Generic[A, B]): pass

class ReproBase(ABC, Generic[C]):

  @abstractmethod
  def repro(self, a: A) -> TwoTypes[C, A]: pass

class ReproSub(ReproBase[C]):

  @abstractmethod
  def repro_defer(self, a: A) -> TwoTypes[C, A]: pass

  def repro(self, a: A) -> TwoTypes[C, A]:
    return self.repro_defer(a)

If I purposely break ReproSub().repro by returning None, I get the error I'd expect error: Incompatible return value type (got "None", expected "TwoTypes[C, A]")

Now if I fix ReproSub().repro by returning to it's previous implementation, the error doesn't clear, rather it changes to the unexpected and incorrect error highlighting a failed inference error: Incompatible return value type (got "TwoTypes[A, A]", expected "TwoTypes[C, A]")

Now if I run with mypy --cache-dir=/dev/null repro.py, the error clears, but only with cache bypassed. Something bad is getting into the cache and not coming out.

It's worth noting this inference problem looks identical to one I fixed around 6 months ago here #7933 - I had thought this was a regression, now I'm wondering if there are more divergent/duplicated pathways between cached vs uncached operation.

Appreciate any insight or pointers, would be open to taking a crack at a fix if it's not a super heavy lift.

[erictraut]

It looks like this bug has been fixed at some point. I'm not able to repro any of the errors in the code samples above. I also can't repro the apparent cache issue. Following the repro steps above, mypy correctly emits an error when it should and then clears it when the error is eliminated.