behavior change in astroid 3.3.10: pyc files cause syntax-error in pylint

[vulli321]

issue caused by this pr: #2728

in astroid 3.3.9 pyc files would trigger a NoSourceFile exception in get_source_file, in 3.3.10 they don't
in my codebase this causes syntax-error reports from pylint

i assume this change was not intentional?

[Pierre-Sassoulas]

Thank you for opening the issue. I think you're right, it's probably due to #2722.

What do you think @Chasarr, should we revert or can we fix in a reasonable time ?

[Chasarr]

Looking into it

[Chasarr]

Can you elaborate on what command you use that triggers it to analyze .pyc files?

Because pylint * would absolutely trigger the issue here.

[vulli321]

Can you elaborate on what command you use that triggers it to analyze .pyc files?

Because pylint * would absolutely trigger the issue here.

doesn't matter in my case if pylint is called with a path, package or single file, the issue occurs for me when pylint analyzes a file that imports a third party package which only exists as pyc files in my venv

[Chasarr]

I'm unable to reproduce, could you please give me a reproducible example?

[Chasarr]

I tried this setup myself:

1. Create lib.py:

"""Documentation for library."""


def give_compliment() -> None:
    """Gives the user a compliment"""
    print("You are very good at Python coding!")

2. Create main.py:

"""Pylint has no complaints here either."""

import lib

def main() -> None:
    """Main function."""
    lib.give_compliment()


if __name__ == "__main__":
    main()

3. Run pylint to confirm everything works:
   pylint main.py lib.py (Score: 10/10)
4. Compile it and it still works:
   python -m compileall main.py. (still works with pylint)
5. Change something in __pycache__ that makes Python unable to run the code:

Traceback (most recent call last):
  File "/Users/chasar/code/python/test/main.py", line 3, in <module>
    import lib
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 879, in exec_module
  File "<frozen importlib._bootstrap_external>", line 1012, in get_code
  File "<frozen importlib._bootstrap_external>", line 672, in _compile_bytecode
ValueError: bad marshal data (unknown type code)

6. Run pylint:
   pylint main.py lib.py

Result: 10/10 no problems, so I conclude that Pylint does not care about the __pycache__ folder, unless you specify it explicitly like this:
python -m pylint __pycache__/lib.cpython-310.pyc

[vulli321]

maybe there is a misunderstanding. i am not forcing pylint to analyze the pyc explicitly, it happens implicitly due to a third party library that is distributed as bytecode only

main.py:

import lib

lib.do_something()

lib.py:

def do_something():
    pass

to reproduce the "obfuscated" library i am forced to use now run main once, delete lib.py, move the pyc from pycache to root and rename to lib.pyc, then run pylint on main.py

main.py:1:0: E0001: Cannot import 'lib' due to 'invalid or missing encoding declaration for '<path to pyc>' (syntax-error)

obviously distributing a library like this is stupid, that does not change the fact that there is a bug though

[Chasarr]

Astroid 3.3.9 also gives an error in this case, but a different one. That tells me that this release changed something at least(Not able to recreate anymore)., but I would love to see an example of a .pyc only library as you are describing. Is there a specific library on pypi that makes Pylint fail for you? That would help with me debugging the problem.

[vulli321]

ah damn you are right, i totally missed that since in my project the library in question is windows only and there already was a pylint disable for the import-error since the dependency is declared windows only in requirements and thus the whole package is missing on linux anyway.

the library is not publically hosted on pypi, i don't know of any other library using such a distribution type

so actually i misinterpreted the change, there is an error in any case, it's just a different one with 3.3.10
i'm fine with adding another pylint disable for the import statement in that case 👍

[Chasarr]

Good to see it working for you!

[Chasarr]

@Pierre-Sassoulas I don't think there is any bug here? Not completely sure, let's keep an eye out for similar problems?

[Pierre-Sassoulas]

Indeed. Thanks a lot for following up and trying to reproduce @Chasarr !