Skip to content

Commit fbb3f06

Browse files
Merge remote-tracking branch 'upstream/master' into groupby_agg_methods
2 parents d47c819 + d33b002 commit fbb3f06

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

50 files changed

+669
-502
lines changed

doc/source/whatsnew/v1.0.2.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,7 @@ Bug fixes
7474
**I/O**
7575

7676
- Using ``pd.NA`` with :meth:`DataFrame.to_json` now correctly outputs a null value instead of an empty object (:issue:`31615`)
77+
- Fixed pickling of ``pandas.NA``. Previously a new object was returned, which broke computations relying on ``NA`` being a singleton (:issue:`31847`)
7778
- Fixed bug in parquet roundtrip with nullable unsigned integer dtypes (:issue:`31896`).
7879

7980
**Experimental dtypes**

doc/source/whatsnew/v1.1.0.rst

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,7 @@ Other enhancements
6666
- :class:`Styler` may now render CSS more efficiently where multiple cells have the same styling (:issue:`30876`)
6767
- When writing directly to a sqlite connection :func:`to_sql` now supports the ``multi`` method (:issue:`29921`)
6868
- `OptionError` is now exposed in `pandas.errors` (:issue:`27553`)
69+
- :func:`timedelta_range` will now infer a frequency when passed ``start``, ``stop``, and ``periods`` (:issue:`32377`)
6970
-
7071

7172
.. ---------------------------------------------------------------------------
@@ -198,6 +199,7 @@ Categorical
198199

199200
- Bug where :func:`merge` was unable to join on non-unique categorical indices (:issue:`28189`)
200201
- Bug when passing categorical data to :class:`Index` constructor along with ``dtype=object`` incorrectly returning a :class:`CategoricalIndex` instead of object-dtype :class:`Index` (:issue:`32167`)
202+
- Bug where :class:`Categorical` comparison operator ``__ne__`` would incorrectly evaluate to ``False`` when either element was missing (:issue:`32276`)
201203
-
202204

203205
Datetimelike
@@ -213,6 +215,7 @@ Timedelta
213215
^^^^^^^^^
214216

215217
- Bug in constructing a :class:`Timedelta` with a high precision integer that would round the :class:`Timedelta` components (:issue:`31354`)
218+
- Bug in dividing ``np.nan`` or ``None`` by :class:`Timedelta`` incorrectly returning ``NaT`` (:issue:`31869`)
216219
-
217220

218221
Timezones
@@ -295,8 +298,10 @@ I/O
295298
- Bug in :meth:`DataFrame.to_parquet` overwriting pyarrow's default for
296299
``coerce_timestamps``; following pyarrow's default allows writing nanosecond
297300
timestamps with ``version="2.0"`` (:issue:`31652`).
301+
- Bug in :meth:`read_csv` was raising `TypeError` when `sep=None` was used in combination with `comment` keyword (:issue:`31396`)
298302
- Bug in :class:`HDFStore` that caused it to set to ``int64`` the dtype of a ``datetime64`` column when reading a DataFrame in Python 3 from fixed format written in Python 2 (:issue:`31750`)
299303

304+
300305
Plotting
301306
^^^^^^^^
302307

@@ -322,6 +327,7 @@ Reshaping
322327
- Bug in :func:`crosstab` when inputs are two Series and have tuple names, the output will keep dummy MultiIndex as columns. (:issue:`18321`)
323328
- :meth:`DataFrame.pivot` can now take lists for ``index`` and ``columns`` arguments (:issue:`21425`)
324329
- Bug in :func:`concat` where the resulting indices are not copied when ``copy=True`` (:issue:`29879`)
330+
- :meth:`DataFrame.replace` and :meth:`Series.replace` will raise a ``TypeError`` if ``to_replace`` is not an expected type. Previously the ``replace`` would fail silently (:issue:`18634`)
325331

326332

327333
Sparse

pandas/_libs/lib.pyx

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -100,11 +100,9 @@ def values_from_object(obj: object):
100100
"""
101101
func: object
102102

103-
if getattr(obj, '_typ', '') == 'dataframe':
104-
return obj.values
105-
106103
func = getattr(obj, '_internal_get_values', None)
107104
if func is not None:
105+
# Includes DataFrame, for which we get frame.values
108106
obj = func()
109107

110108
return obj

pandas/_libs/missing.pyx

Lines changed: 15 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -10,10 +10,13 @@ cnp.import_array()
1010

1111
cimport pandas._libs.util as util
1212

13-
from pandas._libs.tslibs.np_datetime cimport (
14-
get_timedelta64_value, get_datetime64_value)
13+
14+
from pandas._libs.tslibs.np_datetime cimport get_datetime64_value, get_timedelta64_value
1515
from pandas._libs.tslibs.nattype cimport (
16-
checknull_with_nat, c_NaT as NaT, is_null_datetimelike)
16+
c_NaT as NaT,
17+
checknull_with_nat,
18+
is_null_datetimelike,
19+
)
1720
from pandas._libs.ops_dispatch import maybe_dispatch_ufunc_to_dunder_op
1821

1922
from pandas.compat import is_platform_32bit
@@ -44,7 +47,7 @@ cpdef bint checknull(object val):
4447
4548
Returns
4649
-------
47-
result : bool
50+
bool
4851
4952
Notes
5053
-----
@@ -223,7 +226,7 @@ def isnaobj2d_old(arr: ndarray) -> ndarray:
223226

224227
Returns
225228
-------
226-
result : ndarray (dtype=np.bool_)
229+
ndarray (dtype=np.bool_)
227230

228231
Notes
229232
-----
@@ -248,17 +251,11 @@ def isnaobj2d_old(arr: ndarray) -> ndarray:
248251

249252

250253
def isposinf_scalar(val: object) -> bool:
251-
if util.is_float_object(val) and val == INF:
252-
return True
253-
else:
254-
return False
254+
return util.is_float_object(val) and val == INF
255255

256256

257257
def isneginf_scalar(val: object) -> bool:
258-
if util.is_float_object(val) and val == NEGINF:
259-
return True
260-
else:
261-
return False
258+
return util.is_float_object(val) and val == NEGINF
262259

263260

264261
cdef inline bint is_null_datetime64(v):
@@ -364,6 +361,9 @@ class NAType(C_NAType):
364361
exponent = 31 if is_32bit else 61
365362
return 2 ** exponent - 1
366363

364+
def __reduce__(self):
365+
return "NA"
366+
367367
# Binary arithmetic and comparison ops -> propagate
368368

369369
__add__ = _create_binary_propagating_op("__add__")
@@ -423,7 +423,6 @@ class NAType(C_NAType):
423423
return NA
424424
elif isinstance(other, np.ndarray):
425425
return np.where(other == 1, other, NA)
426-
427426
return NotImplemented
428427

429428
# Logical ops using Kleene logic
@@ -433,8 +432,7 @@ class NAType(C_NAType):
433432
return False
434433
elif other is True or other is C_NA:
435434
return NA
436-
else:
437-
return NotImplemented
435+
return NotImplemented
438436

439437
__rand__ = __and__
440438

@@ -443,8 +441,7 @@ class NAType(C_NAType):
443441
return True
444442
elif other is False or other is C_NA:
445443
return NA
446-
else:
447-
return NotImplemented
444+
return NotImplemented
448445

449446
__ror__ = __or__
450447

pandas/_libs/tslibs/timedeltas.pyx

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1407,15 +1407,23 @@ class Timedelta(_Timedelta):
14071407
# convert to Timedelta below
14081408
pass
14091409

1410+
elif util.is_nan(other):
1411+
# i.e. np.nan or np.float64("NaN")
1412+
raise TypeError("Cannot divide float by Timedelta")
1413+
14101414
elif hasattr(other, 'dtype'):
1415+
if other.dtype.kind == "O":
1416+
# GH#31869
1417+
return np.array([x / self for x in other])
14111418
return other / self.to_timedelta64()
14121419

14131420
elif not _validate_ops_compat(other):
14141421
return NotImplemented
14151422

14161423
other = Timedelta(other)
14171424
if other is NaT:
1418-
return NaT
1425+
# In this context we treat NaT as timedelta-like
1426+
return np.nan
14191427
return float(other.value) / self.value
14201428

14211429
def __floordiv__(self, other):

pandas/_testing.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -742,9 +742,9 @@ def repr_class(x):
742742
raise_assert_detail(obj, msg, repr_class(left), repr_class(right))
743743

744744

745-
def assert_attr_equal(attr, left, right, obj="Attributes"):
745+
def assert_attr_equal(attr: str, left, right, obj: str = "Attributes"):
746746
"""
747-
checks attributes are equal. Both objects must have attribute.
747+
Check attributes are equal. Both objects must have attribute.
748748
749749
Parameters
750750
----------

pandas/_typing.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -64,7 +64,7 @@
6464
Label = Optional[Hashable]
6565
Level = Union[Label, int]
6666
Ordered = Optional[bool]
67-
JSONSerializable = Union[PythonScalar, List, Dict]
67+
JSONSerializable = Optional[Union[PythonScalar, List, Dict]]
6868
Axes = Collection
6969

7070
# For functions like rename that convert one label to another

pandas/conftest.py

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1047,6 +1047,16 @@ def series_with_simple_index(indices):
10471047
for dtype in _narrow_dtypes
10481048
}
10491049

1050+
1051+
@pytest.fixture(params=_narrow_series.keys())
1052+
def narrow_series(request):
1053+
"""
1054+
Fixture for Series with low precision data types
1055+
"""
1056+
# copy to avoid mutation, e.g. setting .name
1057+
return _narrow_series[request.param].copy()
1058+
1059+
10501060
_index_or_series_objs = {**indices_dict, **_series, **_narrow_series}
10511061

10521062

pandas/core/algorithms.py

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -313,8 +313,8 @@ def unique(values):
313313
314314
See Also
315315
--------
316-
Index.unique
317-
Series.unique
316+
Index.unique : Return unique values from an Index.
317+
Series.unique : Return unique values of Series object.
318318
319319
Examples
320320
--------
@@ -1515,7 +1515,7 @@ def take(arr, indices, axis: int = 0, allow_fill: bool = False, fill_value=None)
15151515
15161516
See Also
15171517
--------
1518-
numpy.take
1518+
numpy.take : Take elements from an array along an axis.
15191519
15201520
Examples
15211521
--------

pandas/core/arrays/categorical.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -103,7 +103,10 @@ def func(self, other):
103103
mask = (self._codes == -1) | (other_codes == -1)
104104
if mask.any():
105105
# In other series, the leads to False, so do that here too
106-
ret[mask] = False
106+
if opname == "__ne__":
107+
ret[(self._codes == -1) & (other_codes == -1)] = True
108+
else:
109+
ret[mask] = False
107110
return ret
108111

109112
if is_scalar(other):

0 commit comments

Comments
 (0)