mypy 0.780 source file found twice under different module names error

[cristianmatache]

This bazel plugin works well with mypy 0.770 but after upgrading to 0.780 it complains with
"source file found twice under different module names error". mypy 0.780 works fine outside of bazel.

There is a related issue on mypy: python/mypy#8944. In my case, directories on mypy_path do not contain __init__.py files.

• I am setting the mypy_path within mypy.ini. Even if I disable that, the problem persists.
• Bazel tests pass if I disable the bazel mypy checks (so I think the bazel setup is - mostly - correct).
• The template adds $(pwd) to the front of MYPYPATH. I tried playing around with that but still hitting the same issue.

So, I am not very sure where the problem comes from.

EDIT: usually I am setting PYTHONPATH in .bazelrc and mypy_path in mypy.ini but even if I add imports = ["<path e.g. ..>"] in all BUILD files the above steps still apply.

[thundergolfer]

I just ran the aspect with 0.780 and didn't get that error (things worked fine). Can you provide a minimal reproduction, or pointers towards likely sources of the problem?

This is interesting to me:

usually I am setting PYTHONPATH in .bazelrc

How exactly are you doing that, and why? (--action-env ?)

[cristianmatache]

Yes, I am using --action-env to set PYTHONPATH simply because I noticed it works and it reduces the recurring imports= ... so it is easier.

My repo looks like:

monorepo/
     WORKSPACE
     mypy.ini
     python_package_1/
          BUILD
          setup.py
          my_first_package/
               BUILD
               __init__.py
              subpackage/
                  ....
     python_package_2/
          BUILD
          setup.py
          my_second_package/
               <similar to my_first_package>
      ...

In mypy.ini I set mypy_path = my_first_package:my_second_package.
mypy complains:
error: Source file found twice under different module names: 'python_package_1.my_first_package....x.py' and 'my_first_package....x.py'

If I go to mypy.sh.tpl and set --package-root to python_package_1 the error goes away, tests get to run and pass. However, there is no such thing like package roots - note the plural (even though Guido had it in plan initially).

[cristianmatache]

Code to reproduce the error:

File structure:

WORKSPACE
mypy.ini
mypy_version.txt
requirements.txt (empty)
BUILD.bazel (empty)
mylib/
    BUILD.bazel (empty)
    welltyped/
         BUILD.bazel
         __init__.py
         bazelized/
            __init__.py
            x.py

Contents:

• .bazelrc (not sure if it matters but will put it here)

startup --output_user_root=<some dir>
build --disk_cache=<some other dir>
build --aspects @mypy_integration//:mypy.bzl%mypy_aspect
build --output_groups=+mypy
build --action_env=PYTHONPATH=mylib

• mypy.ini

[mypy]
ignore_missing_imports = True
mypy_path = mylib

• mypy_version.txt
  mypy==0.780 !! Change to 0.770 and the error goes away

• welltyped/BUILD.bazel

load("@rules_python//python:defs.bzl", "py_library")
py_library(
    name = "bazelized",
    srcs = glob(["bazelized/*.py"]),
    deps = [],
)

• welltyped/bazelized/x.py

def f(x: int) -> int:
    return x

Now the strange part:

• welltyped/bazelized/__init__.py

# Expose f as "from welltyped.bazelized import f"
from welltyped.bazelize.x import f  # !!! Comment this line and the error no longer appears

ERROR on bazel build mylib/welltyped:bazelized:
mylib/welltyped/bazelized/x.py: error: Source file found twice under different module names: 'mylib.welltyped.bazelized.x' and 'welltyped.bazelized.x'

• mypy alone (outside bazel) works just fine
• mypy 0.770 works fine
• mypy 0.780 breaks only when doing that import in __init__.py
• setting --package-root to mylib instead of . in mypy.sh.tpl fixes the error

output=$({MYPY_EXE} {VERBOSE_OPT} --bazel --package-root mylib --config-file {MYPY_INI_PATH} --cache-map {CACHE_MAP_TRIPLES} -- {SRCS} 2>&1)

The problem with --package-root is that it only allows one root AFAIK.

[thundergolfer]

Thanks for reproduction. I'll take a look.

[dhalperi]

Hmm. I'm trying to figure out the solution here. Collect up the imports of each python dependency, and use that for mypy_path? Otherwise, it seems like one is basically out of luck if they don't use WORKSPACE root on the PYTHONPATH.

[dshahbaz]

#29 addresses this by using -m arguments for modules rather than source files. That way, source files and module paths that they're imported with are matched up, which is what mypy wants. More here: https://github.com/dshahbaz/bazel-mypy-integration/wiki#improvement-3-imports-path-aware-mypy-invocations