[release 0.780] Source file found twice under different module names error.

[ayushr2]

Maintainer edit from 2023

Most likely, you want to run mypy via mypy -p package instead of mypy package/.

To investigate your situation further, see https://mypy.readthedocs.io/en/stable/running_mypy.html#mapping-file-paths-to-modules

I don't recommend reading this thread, there's a lot of out of date information in here.

---

After yesterday's release 0.780, our CI starting failing with the message Source file found twice under different module names: 'grader.testlib.framework' and 'testlib.framework' but we only have one file named framework.py in grader/testlib/framework.py. This error does not show up in version 0.770.

[emmatyping]

How are you invoking mypy, and what does your directory structure look like? Also, please share your config file/command line flags if possible.

[ayushr2]

Directory Structure:

/grader
              /testlib
                         /components
                         framework.py
              /testers
              .mypy.ini
              ...

Invocation:
mypy grader/ --config-file grader/.mypy.ini

Config .mypy.ini

# Global options:

[mypy]
python_version = 3.6
mypy_path = grader

[mypy-pytest]
ignore_missing_imports = True

[roitk]

We're running into the same problem. I thought it might have been caused by mypy-protobuf, but it doesn't look like the OP was using that. I posted some details in the above issue, but here are some more:

Error message:

buildgrid/_protos/build/bazel/semver/semver_pb2.pyi: error: Source file found twice under different module names: 'buildgrid._protos.build.bazel.semver.semver_pb2' and 'build.bazel.semver.semver_pb2'
Found 1 error in 1 file (checked 144 source files)

Directory structure (all files outside of buildgrid/_protos omitted)

.
├── buildgrid
│   ├── _app
│   │   ├── bots
│   │   ├── commands
│   │   └── settings
│   ├── _protos
│   │   ├── __init__.py
│   │   ├── build
│   │   │   ├── __init__.py
│   │   │   └── bazel
│   │   │       ├── __init__.py
│   │   │       ├── remote
│   │   │       │   ├── __init__.py
│   │   │       │   └── execution
│   │   │       │       ├── __init__.py
│   │   │       │       └── v2
│   │   │       │           ├── __init__.py
│   │   │       │           ├── remote_execution.proto
│   │   │       │           ├── remote_execution_pb2.py
│   │   │       │           ├── remote_execution_pb2.pyi
│   │   │       │           └── remote_execution_pb2_grpc.py
│   │   │       └── semver
│   │   │           ├── __init__.py
│   │   │           ├── semver.proto
│   │   │           ├── semver_pb2.py
│   │   │           ├── semver_pb2.pyi
│   │   │           └── semver_pb2_grpc.py
│   │   ├── buildgrid
│   │   │   ├── __init__.py
│   │   │   └── v2
│   │   │       ├── __init__.py
│   │   │       ├── monitoring.proto
│   │   │       ├── monitoring_pb2.py
│   │   │       ├── monitoring_pb2.pyi
│   │   │       └── monitoring_pb2_grpc.py
│   │   ├── buildstream
│   │   │   ├── __init__.py
│   │   │   └── v2
│   │   │       ├── __init__.py
│   │   │       ├── buildstream.proto
│   │   │       ├── buildstream_pb2.py
│   │   │       ├── buildstream_pb2.pyi
│   │   │       └── buildstream_pb2_grpc.py
│   │   └── google
│   │       ├── __init__.py
│   │       ├── api
│   │       │   ├── __init__.py
│   │       │   ├── annotations.proto
│   │       │   ├── annotations_pb2.py
│   │       │   ├── annotations_pb2.pyi
│   │       │   ├── annotations_pb2_grpc.py
│   │       │   ├── http.proto
│   │       │   ├── http_pb2.py
│   │       │   ├── http_pb2.pyi
│   │       │   └── http_pb2_grpc.py
│   │       ├── bytestream
│   │       │   ├── __init__.py
│   │       │   ├── bytestream.proto
│   │       │   ├── bytestream_pb2.py
│   │       │   ├── bytestream_pb2.pyi
│   │       │   └── bytestream_pb2_grpc.py
│   │       ├── devtools
│   │       │   ├── __init__.py
│   │       │   └── remoteworkers
│   │       │       ├── __init__.py
│   │       │       └── v1test2
│   │       │           ├── __init__.py
│   │       │           ├── artman_remoteworkers.yaml
│   │       │           ├── bots.proto
│   │       │           ├── bots_pb2.py
│   │       │           ├── bots_pb2.pyi
│   │       │           ├── bots_pb2_grpc.py
│   │       │           ├── command.proto
│   │       │           ├── command_pb2.py
│   │       │           ├── command_pb2.pyi
│   │       │           ├── command_pb2_grpc.py
│   │       │           ├── remote_workers.yaml
│   │       │           ├── remoteworkers_gapic.yaml
│   │       │           ├── tasks.proto
│   │       │           ├── tasks_pb2.py
│   │       │           ├── tasks_pb2.pyi
│   │       │           ├── tasks_pb2_grpc.py
│   │       │           ├── worker.proto
│   │       │           ├── worker_pb2.py
│   │       │           ├── worker_pb2.pyi
│   │       │           └── worker_pb2_grpc.py
│   │       ├── longrunning
│   │       │   ├── __init__.py
│   │       │   ├── operations.proto
│   │       │   ├── operations_pb2.py
│   │       │   ├── operations_pb2.pyi
│   │       │   └── operations_pb2_grpc.py
│   │       └── rpc
│   │           ├── __init__.py
│   │           ├── code.proto
│   │           ├── code_pb2.py
│   │           ├── code_pb2.pyi
│   │           ├── code_pb2_grpc.py
│   │           ├── status.proto
│   │           ├── status_pb2.py
│   │           ├── status_pb2.pyi
│   │           └── status_pb2_grpc.py
│   ├── bot
│   │   └── hardware
│   ├── cleanup
│   ├── client
│   └── server
│       ├── actioncache
│       ├── bots
│       ├── capabilities
│       ├── cas
│       │   └── storage
│       │       └── index
│       │           └── sql_dialect_delegates
│       ├── execution
│       ├── operations
│       ├── persistence
│       │   ├── mem
│       │   └── sql
│       │       └── alembic
│       │           └── versions
│       └── referencestorage
├── data
│   ├── bin
│   └── config
│       └── grafana
│           ├── dashboards
│           │   └── buildgrid
│           └── datasources
├── dockerfile-scripts
├── docs
│   └── source
│       ├── data
│       ├── developer
│       └── user
└── tests
    ├── auth
    │   └── data
    ├── cas
    │   ├── data
    │   │   └── hello
    │   │       ├── docs
    │   │       │   └── reference
    │   │       └── utils
    │   └── index
    ├── integration
    ├── persistence
    ├── stress-testing-dockerfiles
    │   └── mnt
    │       ├── configs
    │       └── scripts
    └── utils

mypy.ini

[mypy]
plugins = sqlmypy
mypy_path = buildgrid/_protos

[mypy-botocore.*]
ignore_missing_imports = True

[mypy-alembic.*]
ignore_missing_imports = True

[mypy-grpc_reflection.*]
ignore_missing_imports = True

[mypy-janus.*]
ignore_missing_imports = True

mypy invocation (run from the root of the repository):

mypy buildgrid/

[emmatyping]

@ayushr2 @roitk it seems that mypy is now flagging those files as duplicates because they are both searched by being in the MYPYPATH and the sources list. This seems correct to me. This may have to do with a change in how mypy searches namespace packages but I am not sure.

[JelleZijlstra]

I ran into this too, even though I'm not setting $MYPYPATH. It's dependent on exactly how I invoke mypy:

jelle@devshared28:~/ans/yo$ /home/jelle/ans/yo/.tox/py36/bin/python3.6 -m mypy --config-file /home/jelle/ans/yo/src/mypy.ini /home/jelle/ans/yo/src
src/__init__.py: error: Source file found twice under different module names: 'yo.src' and 'src'
Found 1 error in 1 file (checked 37 source files)
jelle@devshared28:~/ans/yo$ /home/jelle/ans/yo/.tox/py36/bin/python3.6 -m mypy --config-file /home/jelle/ans/yo/src/mypy.ini src
Success: no issues found in 37 source files

The config file doesn't have options that should be relevant, and the same error appears if I don't pass any config file.

[emmatyping]

Hm, then perhaps this is a regression after all? I think this definitely merits further investigation.

[JukkaL]

MYPYPATH (or mypy_path) should not point to a directory with an __init__.py file, as this can result in a package with an ambiguous name. Previously mypy sometimes accepted this. This could cause crashes so mypy now complains about it.

@ayushr2 If /grader/__init__.py exists, you've hit the above issue. You should perhaps point mypy_path to the parent directory of grader.

@roitk You have mypy_path that points to a directory with a __init__.py file, so mypy gets confused.

@JelleZijlstra Your issue might be different. Can you give a full example, including all relevant source files?

[hauntsaninja]

I suspect Jelle's issue is also a spurious __init__.py and that it manifests differently with relative paths because of how we crawl up to find the base directory:

mypy/mypy/find_sources.py

Line 101 in 3adb2e9

def crawl_up(self, arg: str) -> Tuple[str, str]:

[JelleZijlstra]

@hauntsaninja is correct. The directory structure in my example is as follows:

├── __init__.py
├── src
    ├── __init__.py
    ├── mypy.ini
    └── yo_bot.py

(plus numerous other files not relevant here). If I remove the outer __init__.py, the error goes away.

[wadetregaskis-linkedin]

I'm hitting this. It happens because I have to specify all the subfolders explicitly in order to get mypy to process them, since it doesn't process folders recursively (contrary to what its documentation says). e.g.

$ echo $MYPYPATH

$ mypy src test test/jobs
test/jobs/job_utils.py: error: Source file found twice under different module names: 'job_utils' and 'jobs.job_utils'  [misc]

If I turn off namespace_packages it seems to work, but it's unclear to me what the ramifications of that will be (the documentation is unclear to its practical purpose, but seems to imply this will cause some modules I import to not be recognised, and thus not have their use by my code typechecked?).

I've noticed for some time that it was double-reporting everything, implying it's been processing things twice, but it didn't originally do that - I'm not certain, but I think this was a mypy change, not something I did. Alas I don't know which version of mypy introduced that duplicate processing.