Transformers crashes mypy

[davidhewitt]

Ubuntu 21.04, tested on both Python 3.8 and Python 3.9 with mypy 0.910

I get a crash caused by transformers 4.10.0 when typechecking a private codebase which imports transformers. This is the error message:

src/transformers/trainer.py:1435: error: INTERNAL ERROR -- Please try using mypy master on Github:
https://mypy.readthedocs.io/en/stable/common_issues.html#using-a-development-mypy-build
Please report a bug at https://github.com/python/mypy/issues
version: 0.910

(I also get the crash trying mypy master. I'm going to submit a corresponding bug report directly to the mypy issue tracker.)

This is the corresponding line:

transformers/src/transformers/trainer.py

Line 1435 in b91e65a

logs["learning_rate"] = self._get_learning_rate()

That method is imported into the class, which presumably upsets mypy somehow.

Repro

I can reproduce this locally in the following way:

1. Clone the transformers repo
2. Create a blank virtualenv and install mypy 0.910
3. Add the following mypy.ini file:

   [mypy]
   check_untyped_defs=True
   

4. Run mypy src/transformers/trainer.py

Confusingly, this local repro method also crashes mypy when using the 4.9.1 tag, but I don't experience the crash in my private codebase when using that transformers version 🤷

cc @sgugger

[sgugger]

I am unsure what you want us to do on the Transformers side? This seems to be a bug in mypi.

[davidhewitt]

I agree the bug is in mypy.

On the transformers side, I would think it's possible to rearrange the code very slightly to avoid exercising the buggy mypy code path - it's easy to reproduce using the steps I provided above.

I wanted to open this issue mostly so you're aware that there are users who can't upgrade to 4.10 because of the crash. The codebase I was upgrading doesn't need any features of 4.10 so I'm afraid I'm not rushing to try to fix this myself.

If you're not worried, feel free to close this issue.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[aphedges]

Although I hope mypy fixes the bug on their side, I did some digging into possible workarounds:

• The transformers commit that causes mypy to crash is d8fb278, which makes sense. Adding a py.typed enabled the type checking, which exposed the bug even though no code was changed in the commit itself. Any commit before it works without issues.
• If you want to use mypy but are okay with not having transformers type checked, add the following code to mypy.ini:

[mypy-transformers.*]
follow_imports = skip

Hope this helps!

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[phiresky]

Just want to mention that the workaround for the newer pyproject.toml-based config files is

[[tool.mypy.overrides]]
follow_imports = "skip"
module = [
  "transformers.*",
]