Allow a module to be imported with no __init__.py(i) file in its path

[jo-sm]

This change allows a user to import a module without the explicit need for an __init__.py(i) in the paths. For example, it a user wants to do the following:

# /project/main.py
from app.utils import foobar

# /project/app/utils.py
def foobar() -> None: pass

they now don't need to add an __init__.py(i) file into /project/app.

[refi64]

Isn't this incompatible with CPython itself?

[jo-sm]

I removed the error case from semanal-errors.test and added a similar but expanded test case in check-modules.test. I also wasn't sure the best way to handle this because the current architecture relies on files explicitly.

[jo-sm]

@kirbyfan64 In Python 2, it's a requirement to have __init__.py but not in Python 3, but you bring up a good point that the code paths I change should only be allowed to run in Python 3 code.

Mypy runs in Python 3 mode by default, right, and I can see the flag (-py2) if a user explicitly asks for it, right?

[gvanrossum]

It's PEP 420. It's Python 3.3+ only, so the PR should at least contain a new version check somewhere. Also, PEP 420 changes the way the search path is used for namespace packages, so this PR still doesn't implement it properly.

Even though this is indeed accepted in Python 3 I'm still pretty concerned about adding support for this to mypy. For example, this would seem to break the functionality of crawl_up() in mypy/main.py, which determines the proper directory to add the the search path by crawling up from a source file until it finds a directory that doesn't contain an __init__.py[i] file -- it would stop too low and insert the module to the search path, breaking import resolution.

I'd much rather see a compromise where mypy continues not to support PEP 420 namespace packages but you can work around it by dropping an empty __init__.pyi file in there.

[jo-sm]

Hmm, so is the recommended approach to add __init__.pyi files into all the folders of my project? I kind of dislike this because I'm in Python 3 only and I don't have __init__.py files unless I need them for some reason, so it feels like a concession to support mypy. Would another possible compromise to be to support this behind a flag, like --namespace-packages?

[gvanrossum]

I suppose a flag is a possibility. I do think you're in a small minority for not using `__init__.py` files.

[jo-sm]

Huh -- I would assume more people would stop using __init__.py files because they are (mostly) extraneous now.

Anyway, if this is a relatively minor thing (as in most people don't wouldn't have a use for it) and the accepted solution is to use __init__.pyi files instead, I can work around that.

[gvanrossum]

Closing for now. If you want to do the command line flag thing feel free to submit a new PR (you can even reopen this one).