Skip to content

BUG: Cannot sample on DataFrameGroupBy with weights when index is specified #39927

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
2 of 3 tasks
wjsi opened this issue Feb 20, 2021 · 0 comments · Fixed by #40015
Closed
2 of 3 tasks

BUG: Cannot sample on DataFrameGroupBy with weights when index is specified #39927

wjsi opened this issue Feb 20, 2021 · 0 comments · Fixed by #40015
Milestone

Comments

@wjsi
Copy link
Contributor

wjsi commented Feb 20, 2021

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • (optional) I have confirmed this bug exists on the master branch of pandas.


Code Sample, a copy-pastable example

import pandas as pd
df1 = pd.DataFrame({'a': [3, 5, 2, 7, 1, 2, 4, 6, 2, 4],
                    'b': [8, 3, 4, 1, 8, 2, 2, 2, 2, 3],
                    'c': [1, 8, 8, 5, 3, 5, 0, 0, 5, 4],
                    'd': [9, 7, 6, 3, 6, 3, 2, 1, 5, 8]}, index=list(range(10, 20)))
df1.groupby('c').sample(1, weights=df1['d'].to_numpy())

Problem description

Sampling ought to work but exception is reported with error stack below:

Traceback (most recent call last):
  File "/Users/wenjun/miniconda3/lib/python3.8/site-packages/IPython/core/interactiveshell.py", line 3417, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-4-1fc31a504740>", line 1, in <module>
    df1.groupby('c').sample(1, weights=df1['d'].to_numpy())
  File "/Users/wenjun/miniconda3/lib/python3.8/site-packages/pandas/core/groupby/groupby.py", line 3024, in sample
    ws = [weights[idx] for idx in self.indices.values()]
  File "/Users/wenjun/miniconda3/lib/python3.8/site-packages/pandas/core/groupby/groupby.py", line 3024, in <listcomp>
    ws = [weights[idx] for idx in self.indices.values()]
  File "/Users/wenjun/miniconda3/lib/python3.8/site-packages/pandas/core/series.py", line 875, in __getitem__
    return self._get_with(key)
  File "/Users/wenjun/miniconda3/lib/python3.8/site-packages/pandas/core/series.py", line 910, in _get_with
    return self.loc[key]
  File "/Users/wenjun/miniconda3/lib/python3.8/site-packages/pandas/core/indexing.py", line 895, in __getitem__
    return self._getitem_axis(maybe_callable, axis=axis)
  File "/Users/wenjun/miniconda3/lib/python3.8/site-packages/pandas/core/indexing.py", line 1113, in _getitem_axis
    return self._getitem_iterable(key, axis=axis)
  File "/Users/wenjun/miniconda3/lib/python3.8/site-packages/pandas/core/indexing.py", line 1053, in _getitem_iterable
    keyarr, indexer = self._get_listlike_indexer(key, axis, raise_missing=False)
  File "/Users/wenjun/miniconda3/lib/python3.8/site-packages/pandas/core/indexing.py", line 1266, in _get_listlike_indexer
    self._validate_read_indexer(keyarr, indexer, axis, raise_missing=raise_missing)
  File "/Users/wenjun/miniconda3/lib/python3.8/site-packages/pandas/core/indexing.py", line 1308, in _validate_read_indexer
    raise KeyError(f"None of [{key}] are in the [{axis_name}]")
KeyError: "None of [Int64Index([6, 7], dtype='int64')] are in the [index]"

Output of pd.show_versions()

INSTALLED VERSIONS

commit : 7d32926
python : 3.8.5.final.0
python-bits : 64
OS : Darwin
OS-release : 19.6.0
Version : Darwin Kernel Version 19.6.0: Tue Jan 12 22:13:05 PST 2021; root:xnu-6153.141.16~1/RELEASE_X86_64
machine : x86_64
processor : i386
byteorder : little
LC_ALL : None
LANG : None
LOCALE : zh_CN.UTF-8
pandas : 1.2.2
numpy : 1.19.1
pytz : 2021.1
dateutil : 2.8.1
pip : 20.2.2
setuptools : 49.6.0.post20200814
Cython : 0.29.21
pytest : 6.0.1
hypothesis : None
sphinx : 3.2.1
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : 2.11.2
IPython : 7.17.0
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : None
fastparquet : 0.4.2
gcsfs : None
matplotlib : 3.2.2
numexpr : 2.7.1
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : 1.0.1
pyxlsb : None
s3fs : None
scipy : 1.6.0
sqlalchemy : 1.3.18
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
numba : 0.51.2

@wjsi wjsi added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Feb 20, 2021
@rhshadrach rhshadrach added Groupby and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Feb 24, 2021
@rhshadrach rhshadrach added this to the 1.3 milestone Feb 24, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants