Forwarding references don't work inside aliases

[jgarvin]

I was trying to define the following alias:

from typing import Callable, Dict, Set, Any, DefaultDict
from pathlib import Path

Recipe = Callable[["Tracker", Dict[str, Any], Path], Set[Path]]

but I get this error:

error: Name 'Tracker' is not defined

But of course the whole point of forwarding references is that it shouldn't have to be defined.

[gvanrossum]

Thanks; we've already got an issue for this: #2241. Closing as a duplicate.

[jgarvin]

@gvanrossum That's actually not the same issue. Here an alias uses a reference, there they try to use an alias as a reference.

[gvanrossum]

Oh, sorry. In that case, your example appears incomplete -- where's the definition of Tracker? It must exist in the same module. If I follow your example with class Tracker: pass I get no error.

[jgarvin]

My mistake then, I assumed that you could use forward references to break import cycles like you would normally use forward declarations to break include cycles in C (my module doesn't actually need to use the definition of Tracker, I only need it for the Recipe type), in which case you wouldn't expect the definition to be required to appear later in the same module. I just found the documentation on TYPE_CHECKING -- it's a little confusing though because you use it to introduce what is still a circular import, but I guess only when type checking that's okay?

[gvanrossum]

PEP 484 forward references exist to deal with the constraint (due to executing all type annotations as the module body is being executed) that all expression elements must be defined. They have nothing in common with C forward references.

Import cycles during type checking are dealt with differently (it's an evolving topic actually, see this query: https://github.com/python/mypy/issues?q=is%3Aopen+is%3Aissue+label%3A%22Import+Cycle%22+sort%3Aupdated-desc).

I'll label this as a question then -- do you think there's something we could have said in the docs that would have prevented you from going down this dead end? (It's the first time this particular confusion has come to my attention.)

[jgarvin]

I think it would suffice to change the documentation at (http://mypy.readthedocs.io/en/latest/kinds_of_types.html#class-name-forward-references) to explicitly state that a definition must appear later in the same module, and to say that you should look at the docs for TYPE_CHECKING instead to avoid circular references. Even better though would be if the same mechanism was used to solve both problems like in C, then the confusion can just never happen. They are the same issue in the sense that you want to use a type before it's fully defined.

[gvanrossum]

Hope this update is to your liking. Regarding your suggestion of using forward references to avoid import cycles, in C you can get away with that because there's only a single global namespace; but Python (or mypy) wouldn't have a way to map the reference to a definition without an import, so I don't think it will fly. Even names used in type comments must be imported so that mypy can understand what they refer to.

[JukkaL]

Yeah, it's quite possible to have multiple classes with the same name in different modules, and this becomes more likely if you use 3rd party libraries (or just have a large codebase), so it's not just a theoretical consideration. Imports make it explicit where the class is defined.

[jgarvin]

Doc change looks great :D

C++ has namespaces and still uses forward declarations, but that's because names are defined and/or forward declared in explicit namespace foo { ... } blocks. Some projects will even have "forwarding headers" that just contain forward declarations. So it's possible, but you have to have a syntax for it. The stub syntax (where the function body is ellipses) looks pretty close to a forward function declaration in C or C++, maybe something could be built on that? You'd still need a syntax to put specify the function is inside a namespace/module though.