Weird behavior with Optional[List[<smth>]] as classmethod return type and value produced from Any as part of that list

[actionless]

• Are you reporting a bug, or opening a feature request?
  Bug

• Please insert below the code you are checking with mypy,
  or a mock-up repro if the source is private. We would appreciate
  if you try to simplify your case to a minimal repro.

from typing import List, Optional, Any


class MyClass1:

    _result: Optional[List[str]] = None
    _foo: Any = None

    @classmethod
    def get(cls, some_arg: Optional[str]) -> List[str]:
        if not cls._result:
            splitted_foo: List[str] = (
                cls._foo.split(',') if cls._foo else []
            )
            cls._result = [some_arg or 'a'] + splitted_foo
        return cls._result


class MyClass2:

    _result: Optional[List[str]] = None
    _foo: Any = None

    @classmethod
    def get(cls) -> List[str]:
        if not cls._result:
            cls._result = (
                cls._foo.split(',') if cls._foo else []
            )
        return cls._result


class MyClass3:

    _result: Optional[List[str]] = None
    _foo: Any = None

    @classmethod
    def get(cls, some_arg: Optional[str]) -> List[str]:
        if not cls._result:
            cls._result = [some_arg or 'a'] + (
                cls._foo.split(',') if cls._foo else []
            )
        return cls._result
#       ^ [mypy] : Incompatible return value type (got "Optional[List[str]]", expected "List[str]")


class MyClass4:

    _result: Optional[List[str]] = None
    _foo: Optional[str] = None

    @classmethod
    def get(cls, some_arg: Optional[str]) -> List[str]:
        if not cls._result:
            cls._result = [some_arg or 'a'] + (
                cls._foo.split(',') if cls._foo else []
            )
        return cls._result

• What is the actual behavior/output?

(env) $ mypy --version
mypy 0.720
(env) $ mypy mypy_func.py
mypy_func.py:44: error: Incompatible return value type (got "Optional[List[str]]", expected "List[str]")

(env) $ pip install git+https://github.com/python/mypy/
.....
      Successfully uninstalled mypy-0.720
Successfully installed mypy-0.740+dev.6c4599dbdfe3bcc721dd480bc82e055e37102184
(env) $ mypy mypy_func.py
mypy_func.py:44: error: Incompatible return value type (got "Optional[List[str]]", expected "List[str]")
Found 1 error in 1 file (checked 1 source file)

(env) $ deactivate
$ mypy --version
mypy 0.730
$ mypy mypy_func.py
mypy_func.py:44: error: Incompatible return value type (got "Optional[List[str]]", expected "List[str]")
Found 1 error in 1 file (checked 1 source file)

• What is the behavior/output you expect?
  without that error, i think the right error should be what 'Any' type doesn't have '.split()' method

But important to note what Class1 and Class2 not causing any mypy warnings, while Class3 does: it seems to me what all 3 of them have the same problem so should be handled with the same result.

• What are the versions of mypy and Python you are using?
  Do you see the same issue after installing mypy from Git master?

$ python --version
Python 3.7.4

• What are the mypy flags you are using? (For example --strict-optional)
  none of them

[msullivan]

This is a weird one and comes from a bunch of weird interactions and tradeoffs in the design space for Any types and type binding. I believe that this is not a bug, though the interactions here are weird.

• In most cases, when assigning an Any to a variable, we reset the variable to its declared type instead of binding its type to Any. This is to avoid letting use of any override useful type declarations. This is why case 3 errors.
• Case 1 and case 4 both work because there aren't any Anys involved and the types are what is expected.
• Case 2 works because the type gets inferred as Union[List[str], Any], and that we are willing to bind as the type.

To complicated matters even further, case 3 will work if you write the test as cls._result is not None as a result of #5629, which was the compromise solution we settled on after a lot of wrangling about behavior in cases like this.
(Of course, that test has different semantics...)

[JukkaL]

Closing as this works as designed. Changing the mypy behavior is not totally out of the question, but this would at least need more justification and analysis.

[actionless]

@JukkaL

from the further discussion it seems what the real root cause of the problem is different behavior between

        if not cls._cmd:

        if cls._cmd is None:

is it considered to be a bug and if so, is it reported already?

[JukkaL]

That seems new, but I missed it when reading the discussion. Could you create an issue about that problem in particular?

[actionless]

created: #7696