astroid infinite loop when encountering comparison operator

[smontanaro]

Steps to reproduce

1. mv regopcodes.txt regopcodes.py
2. pylint regopcodes.py

Current behavior

Infinite loop when it encounters the expression op >= opcode.HAVE_ARGUMENT

Expected behavior

No infinite loop

Output of pylint --version:

pylint 2.4.4
astroid 2.3.3
Python 3.8.1 (default, Jan 8 2020, 22:29:32)
[GCC 7.3.0]

regopcodes.txt

[PCManticore]

Thanks for the report @smontanaro !

I'm afraid I can't reproduce it, does the reproduction requires some additional steps? Here's my output using the same version as you:

pylint 2.4.4
astroid 2.3.3
Python 3.8.1 (default, Feb 13 2020, 10:17:07)
[Clang 8.1.0 (clang-802.0.42)]

************* Module regopcodes
regopcodes.py:8:0: C0115: Missing class docstring (missing-class-docstring)
regopcodes.py:11:12: C0103: Variable name "op" doesn't conform to snake_case naming style (invalid-name)
regopcodes.py:19:4: C0103: Argument name "op" doesn't conform to snake_case naming style (invalid-name)
regopcodes.py:53:4: C0103: Argument name "op" doesn't conform to snake_case naming style (invalid-name)
regopcodes.py:53:4: C0116: Missing function or method docstring (missing-function-docstring)
regopcodes.py:60:4: C0103: Argument name "op" doesn't conform to snake_case naming style (invalid-name)
regopcodes.py:60:4: C0116: Missing function or method docstring (missing-function-docstring)
regopcodes.py:68:4: C0103: Argument name "op" doesn't conform to snake_case naming style (invalid-name)
regopcodes.py:68:4: C0116: Missing function or method docstring (missing-function-docstring)
regopcodes.py:68:4: R0201: Method could be a function (no-self-use)
regopcodes.py:73:0: C0103: Constant name "def_op" doesn't conform to UPPER_CASE naming style (invalid-name)
regopcodes.py:75:29: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:76:29: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:77:31: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:78:29: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:79:33: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:80:30: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:81:25: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:82:36: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:83:36: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:84:31: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:85:34: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:86:44: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:87:45: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:88:34: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:89:37: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:90:35: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:91:32: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:92:37: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:93:35: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:94:41: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:95:40: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:96:42: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:97:41: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:98:29: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:99:39: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:100:31: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:101:31: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:102:39: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:103:35: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:104:33: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:105:38: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:106:38: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:107:36: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:108:34: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:109:35: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:110:35: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:111:35: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:112:32: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:113:32: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:114:31: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:115:35: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:116:30: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:117:41: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:118:32: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:119:38: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:120:32: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:121:35: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:122:42: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:123:36: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:124:36: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:125:33: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:126:33: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:127:32: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:128:35: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:129:34: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:130:33: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:131:39: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:132:33: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:133:31: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:134:32: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:138:32: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:139:33: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:140:37: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:141:30: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:142:31: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:143:32: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:144:33: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:145:34: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:146:35: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:148:32: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:149:31: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:150:33: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:151:32: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:152:31: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:153:31: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:154:31: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:156:32: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:157:33: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:158:33: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:159:34: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:160:42: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:161:41: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:162:35: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:163:39: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:164:38: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:165:33: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:166:27: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:167:33: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:168:43: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:169:35: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:171:31: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:173:32: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:175:33: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:177:35: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:178:35: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:179:35: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:180:33: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:182:34: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:184:32: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:186:33: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:188:34: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:189:38: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:190:38: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:191:32: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:192:33: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:193:29: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:194:29: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:196:37: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:198:34: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:199:38: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:200:34: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:201:41: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:202:34: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:203:33: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:204:33: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:205:33: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:206:32: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:207:32: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:208:33: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:213:38: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:218:40: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:219:38: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:220:39: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:221:37: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:222:38: C0321: More than one statement on a single line (multiple-statements)
regopcodes.py:230:7: E0602: Undefined variable 'op' (undefined-variable)

---

Your code has been rated at 5.17/10 (previous run: 5.17/10, +0.00)

Could you maybe post the traceback as well?

[smontanaro]

Thanks for the quick response @PCManticore. I thought it very strange that it would barf on such a basic thing, but couldn't make it go away. I've attached the traceback. Let me know if I can provide more data.

traceback.txt

[smontanaro]

@PCManticore Is there anything I can do to help debug this problem?

[PCManticore]

Hey @smontanaro I'm afraid I still can't reproduce it with

pylint 2.4.4
astroid 2.3.3
Python 3.8.1 (default, Feb 13 2020, 10:17:07)
[Clang 8.1.0 (clang-802.0.42)]

From the traceback it seems the recursion starts when checking for comparison-with-callable which triggers the inference process.

Can you also post the pylintrc configuration file if you are using one?
Also do you have a opcode.py file locally that pylint might use instead of the stdlib one?
I see from the traceback it uses miniconda but I doubt it has any relevance here. One thing you can try to do is to print the relevant node in _check_callable_comparison to see where exactly it barfs. You can use the .as_string() method to see the Python representation.

Sorry for not having more ideas, it would be great if I'd manage to reproduce it.

[smontanaro]

I've now tried with three different versions of Python

1. /usr/bin/python3 - 3.7.5
2. Miniconda Python - 3.8.1
3. 3.9.0a5+ - running from my git checkou

Note that the attached opcode.py (from my git repo) contains a few extra instructions. I'm working on a translator from Python's stack virtual machine into a register-based virtual machine. :-) Still, I have tried now with three different versions of Python, so while my (uninstalled) Lib/opcode.py file is used when running with my sandbox Python, the other two won't use it. Also, at this stage, no register instructions are written to .pyc files, so pylint shouldn't encounter anything weird and annoying.

Since the /usr/bin/python3 use is new, here's the version info from it:

% /usr/bin/pylint --version
pylint 2.2.2
astroid 2.1.0
Python 3.7.5 (default, Nov 20 2019, 09:21:52)
[GCC 9.2.1 20191008]

I'll work on the other debug recommendations and comment separately. Might not be for a day or two.

pylintrc.txt
opcode.py.txt

[smontanaro]

This was simpler than I anticipated. I tweaked

/home/skip/miniconda3/envs/python38/lib/python3.8/site-packages/pylint/checkers/base.py

to print the node to stderr. The infinite recursion seems to be triggered in the assert statement at the very end of lib/rattlesnake/opcodes.py. Here's the printed node:

Compare(left=<Name.OP l.211 at 0x7f1e4d484d30>,
ops=[('<=', <Const.int l.211 at 0x7f1e4d484dc0>)])

If I comment out the assert statement, the pylint command runs to completion.

So, here's the command I'm running right now (from the top of my git checkout):

pylint Lib/rattlesnake

My CPython fork is here:

https://github.com/python/cpython

and I'm running off the register branch. As of this writing my checkout is up-to-date with what you see. (At the moment, I'm mostly fussing with C code trying to stamp out refleaks, so the Python code might not change much in the next couple days.)

I messed around briefly with lower levels of the traceback, but didn't really understand what I was doing, so gave up pretty quickly.

[PCManticore]

Hey @smontanaro I was able to reproduce it using your instructions, thanks.
I don't have a solution at the moment, other than increasing the recursion limit with --init-hook="import sys; sys.setrecursionlimit(2000). The reason this crashes right now is that we're inferring all the augmented assignments in that file and that increases the stack size tremendously.
We might be able to add a stop gap in AugAssign inference by using a path wrapper, which prevents reinference of the same object (e.g. with @decorators.path_wrapper), but that seems to affect some tests. Once I have some time, I'll investigate more for a possible solution

https://github.com/PyCQA/astroid/blob/17a5ee681bcf4aacffcc4ec5afbc3436cfdc4537/astroid/inference.py#L786

[smontanaro]

Good that you can reproduce it and great that you were able to offer a workaround.

Edit: One last followup from me on this. I eventually eliminated the opcodes module from my project, but .../Lib/opcode.py is similarly modified to use "+=". I needed to boost the recursion limit a bit higher (2750 failed, but 2875 worked). Since my first report, I've implemented a fair number of new instructions, so it's understandable that the recursion limit needed to be increased.