-
-
Notifications
You must be signed in to change notification settings - Fork 18.7k
ENH: support any/all for pyarrow numeric and duration dtypes #50717
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
pandas/core/arrays/arrow/array.py
Outdated
zero = Timedelta(0) | ||
else: | ||
zero = 0 | ||
result = (self != zero)._reduce(name, skipna=skipna, **kwargs) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we use data = pc.not_equal(self._data, zero)
and continue with the logic below?
pandas/core/arrays/arrow/array.py
Outdated
else: | ||
zero = 0 | ||
|
||
not_eq = pc.not_equal(self._data, zero) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah sorry, I had meant
if name in ...:
...
data = pc.not_equal(self._data, zero)
else:
data = self._data
....
try:
result = pyarrow_method(data...)
updated per requests + green |
Thanks @jbrockmendel |
…dev#50717) * ENH: support any/all for pyarrow numeric and duration dtypes * mypy fixup * use pc.not_equal * use suggested pattern * fix not_eq
doc/source/whatsnew/vX.X.X.rst
file if fixing a bug or adding a new feature.Cuts the runtime for
pytest pandas/tests/extension/test_arrow.py::TestBaseBooleanReduce::test_reduce_series
from 39.8s to 2.3s.