You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Referring to a local variable in a list comprehension with query's @ does not work, but it does work using f-strings. I tried on Pandas 1.5.3 and 2.0.1.
Expected Behavior
test1 should work as test2.
Installed Versions
INSTALLED VERSIONS
commit : 37ea63d
python : 3.10.11.final.0
python-bits : 64
OS : Linux
OS-release : 5.15.0-71-generic
Version : #78-Ubuntu SMP Tue Apr 18 09:00:29 UTC 2023
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8
df.query only accepts local variables inside a query string, see here.
Inside a list comprehension the result from calling locals() is the local scope inside the list comprehension only, not the function. This is a limitation of python itself, so I don't think we can do so much about it in pandas. So you've hit a limitation of python here, AFAIK.
Whether this can be mitigated somehow I don't know, but that would be an enhancement to pandas, not a bug fix.
So I think this will be closed as wontfix, unfortunately. You can check if the docs can be made clearer in this regard.
A small cleanup of the above explanation: the dict comprehension tries first the locals(), then globals(). So the below will work and the g variable is 0:
Thank you for your answers. Why doesn't it apply for f-strings as well (test2 in my example)? Shouldn't it be the same as a result of the list comprehension scoping?
Notice that g isn't stored in the list comprehension, meaning it won't be available for use later...
AFAIK test2 works because list comprehensions are weird: scope when calling locals() and when accessing variables directly are different in list comprehension. I don't have a better explanation, sorry. There's this SO discussion about it
Uh oh!
There was an error while loading. Please reload this page.
Pandas version checks
I have checked that this issue has not already been reported.
I have confirmed this bug exists on the latest version of pandas.
I have confirmed this bug exists on the main branch of pandas.
Reproducible Example
Issue Description
Referring to a local variable in a list comprehension with query's @ does not work, but it does work using f-strings. I tried on Pandas 1.5.3 and 2.0.1.
Expected Behavior
test1
should work astest2
.Installed Versions
INSTALLED VERSIONS
commit : 37ea63d
python : 3.10.11.final.0
python-bits : 64
OS : Linux
OS-release : 5.15.0-71-generic
Version : #78-Ubuntu SMP Tue Apr 18 09:00:29 UTC 2023
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : None
LANG : en_US.UTF-8
LOCALE : en_US.UTF-8
pandas : 2.0.1
numpy : 1.24.3
pytz : 2022.7
dateutil : 2.8.2
setuptools : 66.0.0
pip : 23.0.1
Cython : None
pytest : None
hypothesis : None
sphinx : 5.0.2
blosc : None
feather : None
xlsxwriter : None
lxml.etree : 4.9.2
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 3.1.2
IPython : 8.12.0
pandas_datareader: None
bs4 : 4.12.2
bottleneck : 1.3.5
brotli :
fastparquet : None
fsspec : 2023.3.0
gcsfs : None
matplotlib : 3.7.1
numba : None
numexpr : 2.8.4
odfpy : None
openpyxl : 3.0.10
pandas_gbq : None
pyarrow : None
pyreadstat : None
pyxlsb : None
s3fs : None
scipy : 1.10.1
snappy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
zstandard : None
tzdata : 2023.3
qtpy : 2.2.0
pyqt5 : None
/home/yuval/anaconda3/envs/cltorch/lib/python3.10/site-packages/_distutils_hack/init.py:33: UserWarning: Setuptools is replacing distutils.
warnings.warn("Setuptools is replacing distutils.")
The text was updated successfully, but these errors were encountered: