Include Python's sys.path into build.default_lib_path?

[lrem]

Mypy refuses to cooperate with any code importing things not in standard library. I see that it doesn't search in sys.path. Any particular reason for that? A single line change like lrem@6e363a0 lets mypy pick up all modules from current environment (not that it parses all of them, looking into that separately from this issue).

[JukkaL]

The main reason why we don't do it right now it because mypy still often chokes on valid Python code. Once the parser and semantic analyzer are robust enough to handle much/most arbitrary Python code then this would make sense, I think, and could even be a game-changer for mypy!

These things would be nice to have in order to enable searching sys.path by default:

1. Don't type check modules that don't import typing (Do not type check modules that don't import typing #186). We'd only perform semantic analysis, and it's much less strict than type checking.
2. Implement parsing of all/most remaining Python syntax (I'm not sure what exactly is missing -- it would be easy to just run mypy.parse on a corpus of Python code and file any parse errors as features/bugs).
3. Make the semantic analyzer more robust and only complain about very obvious errors for modules that don't import typing, or maybe just don't complain about anything for these modules.
4. Make sure that when we use some unmodified Python libraries from statically typed code mypy behaves reasonably. We can probably report some errors for code that uses these modules such as accessing undefined module attributes, invalid function argument counts or invalid keyword argument names even if that target module has no type annotations.

@lrem Are you interested in moving this forward? At least steps 1 and 2 should not be very tricky and we could start really working on steps 3+4.

Step 3 is probably the trickiest to implement. We can start with a kind-of working implementation (E.g., if we could use, say, 50% of the Python standard library, this would already be a useful feature to have. If we can use only 10% of the Python standard library I'd probably be against enabling this feature by default.) Step 4 is open-ended but we can start with something not too fancy.

You can always use MYPYPATH to point to whatever directories you want as a workaround until we get the above things fixed/tested, but it's clearly awkward.

[JukkaL]

Added issue #487 for investigating parser coverage.

[refi64]

I'd be very happy for a command-line flag on whether or not to run semantic checks on modules that include typing.

[spkersten]

I'm in favor of having at least the (command line) option of doing some semantic analysis or even some type checking on modules that don't include typing. Would be a shame not to catch things that are clearly errors. Until mypy is robust enough, the default value of that option could of course be not to any of those.

[lrem]

#492 has the flag for looking into sys.path.

So will a standard pip install now install a mypy version that tracks sys.path? Thanks!

[lrem]

Not yet. The parser still chokes on some perfectly valid Python. I hope we
can fix that in a reasonable time frame and release everything together.

On Sat, 29 Nov 2014 00:42 Dedoig notifications@github.com wrote:

So will a standard pip install now install a mypy version that tracks
sys.path? Thanks!

—
Reply to this email directly or view it on GitHub
#486 (comment).

[JukkaL]

I think we are pretty close to being able to parse almost arbitrary Python code. The remaining task is to tweak the semantic analyzer and the type checker to be less picky and to ignore things that aren't actual problems in code that doesn't import typing. This is the bigger unknown, but I don't expect huge challenges. At least testing is easy -- we can just point mypy to arbitrary Python modules and see what it reports, and file issues :-)

[JukkaL]

Issue #638 is basically targeting the same problem. Closing this as redundant.