Skip to content

Boolean indexing inconsistency #61090

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions doc/source/whatsnew/v3.0.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -697,6 +697,7 @@ Indexing
- Bug in :meth:`DataFrame.from_records` throwing a ``ValueError`` when passed an empty list in ``index`` (:issue:`58594`)
- Bug in :meth:`DataFrame.loc` with inconsistent behavior of loc-set with 2 given indexes to Series (:issue:`59933`)
- Bug in :meth:`MultiIndex.insert` when a new value inserted to a datetime-like level gets cast to ``NaT`` and fails indexing (:issue:`60388`)
- Bug in :meth:`Series.iloc` with inconsistent behavior between ``__getitem__`` and ``__setitem__`` for Series boolean indexers (:issue:`60994`)
- Bug in printing :attr:`Index.names` and :attr:`MultiIndex.levels` would not escape single quotes (:issue:`60190`)

Missing
Expand Down
16 changes: 16 additions & 0 deletions pandas/core/indexing.py
Original file line number Diff line number Diff line change
Expand Up @@ -1637,6 +1637,22 @@ def _has_valid_setitem_indexer(self, indexer) -> bool:
"Consider using .loc with a DataFrame indexer for automatic alignment.",
)

# Check for Series boolean indexer
if (
com.is_bool_indexer(indexer)
and hasattr(indexer, "index")
and isinstance(indexer.index, Index)
):
if indexer.index.inferred_type == "integer":
raise NotImplementedError(
"iLocation based boolean "
"indexing on an integer type "
"is not available"
)
raise ValueError(
"iLocation based boolean indexing cannot use an indexable as a mask"
)

if not isinstance(indexer, tuple):
indexer = _tuplify(self.ndim, indexer)

Expand Down
5 changes: 5 additions & 0 deletions pandas/tests/extension/base/reduce.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,11 @@ def test_reduce_series_numeric(self, data, all_numeric_reductions, skipna):
msg = (
"[Cc]annot perform|Categorical is not ordered for operation|"
"does not support operation|"
"unsupported operand type|"
"NotImplemented|"
"not supported between instances|"
"NotImplementedType|"
"iLocation based boolean"
)

with pytest.raises(TypeError, match=msg):
Expand Down
15 changes: 15 additions & 0 deletions pandas/tests/frame/indexing/test_getitem.py
Original file line number Diff line number Diff line change
Expand Up @@ -407,6 +407,21 @@ def test_getitem_frozenset_unique_in_column(self):
expected = Series([1], name=frozenset(["KEY"]))
tm.assert_series_equal(result, expected)

def test_series_boolean_indexer_iloc_consistency(self):
# GH#60994 - Test consistency between __getitem__ and __setitem__ for Series
# boolean indexers
ser = Series([0, 1, 2])
mask = Series([True, False, False])

# __getitem__ should raise NotImplementedError
msg = "iLocation based boolean indexing on an integer type is not available"
with pytest.raises(NotImplementedError, match=msg):
ser.iloc[mask]

# __setitem__ should also raise NotImplementedError for consistency
with pytest.raises(NotImplementedError, match=msg):
ser.iloc[mask] = 10


class TestGetitemSlice:
def test_getitem_slice_float64(self, frame_or_series):
Expand Down
Loading