Unexpected type 'unicode' for os.path.join() with (Any, str) args

[JukkaL]

The revealed type for the os.path.join call in the program below is unicode, even though I'd expect either Any or str, as the arguments have types Any and str:

import os
def f(): pass
reveal_type(os.path.join(f(), 'x'))  # builtins.unicode

Here's a self-contained example:

# a.pyi
from typing import overload
@overload
def join(__p1: bytes, *p: bytes) -> bytes: ...
@overload
def join(__p1: unicode, *p: unicode) -> unicode: ...

# program.py
from a import join
def f(): pass
reveal_type(join(f(), 'x'))  # builtins.unicode

Run as mypy -2 program.py.

@Michael0x2a Do you have time to have a look at this? This is causing issues in internal Dropbox repos. If you are busy, I can try to fix this. This was apparently introduced by f61c2ba.

[ilevkivskyi]

I am looking at this now.

[Michael0x2a]

I think the issue is that due to the way we handle promotions in mypy, bytes is actually technically considered to be a subtype of unicode -- so the union math logic concludes that it's valid to return unicode instead of Any since the former is more precise.

Not sure what the best way to handle this is -- I'd personally prefer just removing that promotion altogether, but that's a very invasive and difficult-to-implement change. Alternatively, we could special-case this in the union math somehow, but that would make it inconsistent with the rest of mypy. E.g. this currently typechecks, for example:

def expects_unicode(x):
    # type: (unicode) -> None
    pass

b = b'foo'  # type: bytes

expects_unicode(b)

[Michael0x2a]

I guess other options include:

1. Modifying the "is the last return type a supertype of the previous ones" checks so promotions are disabled altogether.
2. Removing that check altogether so we always return Any.

[ilevkivskyi]

My original motivation for requesting it was that this example should work

@overload
def f(x: List[int]) -> List[int]: ...
@overload
def f(x: List[Any]) -> List[Any]: ...

arg: Any
f(arg)  # should be List[Any], _not_ just Any

Also now I think that we should really check both ways, because the logic is that replacing a precise type with Any should ideally not cause new errors. For example:

def f() -> int: ...

@overload
def g(x: int) -> int: ...
@overload
def g(x: float) -> float: ...

x: int = g(f())  # this passes, but will fail if I remove the annotation from 'f'

I will now prepare a PR to return last type only if it is compatible both ways with all other return types (like in the first example with List[Any]).