Merge branch 'main' into 9586-inconsistent-labeling-sub-daily-super-daily-frequencies

Author: natmokval

doc/source/user_guide/sparse.rst

@@ -153,70 +153,14 @@ the correct dense result.
    np.abs(arr)
    np.abs(arr).to_dense()
 
-.. _sparse.migration:
-
-Migrating
----------
-
-.. note::
-
-   ``SparseSeries`` and ``SparseDataFrame`` were removed in pandas 1.0.0. This migration
-   guide is present to aid in migrating from previous versions.
-
-In older versions of pandas, the ``SparseSeries`` and ``SparseDataFrame`` classes (documented below)
-were the preferred way to work with sparse data. With the advent of extension arrays, these subclasses
-are no longer needed. Their purpose is better served by using a regular Series or DataFrame with
-sparse values instead.
-
-.. note::
-
-  There's no performance or memory penalty to using a Series or DataFrame with sparse values,
-  rather than a SparseSeries or SparseDataFrame.
-
-This section provides some guidance on migrating your code to the new style. As a reminder,
-you can use the Python warnings module to control warnings. But we recommend modifying
-your code, rather than ignoring the warning.
-
-**Construction**
-
-From an array-like, use the regular :class:`Series` or
-:class:`DataFrame` constructors with :class:`arrays.SparseArray` values.
-
-.. code-block:: python
-
-   # Previous way
-   >>> pd.SparseDataFrame({"A": [0, 1]})
-
-.. ipython:: python
-
-   # New way
-   pd.DataFrame({"A": pd.arrays.SparseArray([0, 1])})
-
-From a SciPy sparse matrix, use :meth:`DataFrame.sparse.from_spmatrix`,
-
-.. code-block:: python
-
-   # Previous way
-   >>> from scipy import sparse
-   >>> mat = sparse.eye(3)
-   >>> df = pd.SparseDataFrame(mat, columns=['A', 'B', 'C'])
-
-.. ipython:: python
-
-   # New way
-   from scipy import sparse
-   mat = sparse.eye(3)
-   df = pd.DataFrame.sparse.from_spmatrix(mat, columns=['A', 'B', 'C'])
-   df.dtypes
 
 **Conversion**
 
-From sparse to dense, use the ``.sparse`` accessors
+To convert data from sparse to dense, use the ``.sparse`` accessors
 
 .. ipython:: python
 
-   df.sparse.to_dense()
-   df.sparse.to_coo()
+   sdf.sparse.to_dense()
 
 From dense to sparse, use :meth:`DataFrame.astype` with a :class:`SparseDtype`.
 
@@ -226,40 +170,6 @@ From dense to sparse, use :meth:`DataFrame.astype` with a :class:`SparseDtype`.
    dtype = pd.SparseDtype(int, fill_value=0)
    dense.astype(dtype)
 
-**Sparse Properties**
-
-Sparse-specific properties, like ``density``, are available on the ``.sparse`` accessor.
-
-.. ipython:: python
-
-   df.sparse.density
-
-**General differences**
-
-In a ``SparseDataFrame``, *all* columns were sparse. A :class:`DataFrame` can have a mixture of
-sparse and dense columns. As a consequence, assigning new columns to a :class:`DataFrame` with sparse
-values will not automatically convert the input to be sparse.
-
-.. code-block:: python
-
-   # Previous Way
-   >>> df = pd.SparseDataFrame({"A": [0, 1]})
-   >>> df['B'] = [0, 0]  # implicitly becomes Sparse
-   >>> df['B'].dtype
-   Sparse[int64, nan]
-
-Instead, you'll need to ensure that the values being assigned are sparse
-
-.. ipython:: python
-
-   df = pd.DataFrame({"A": pd.arrays.SparseArray([0, 1])})
-   df['B'] = [0, 0]  # remains dense
-   df['B'].dtype
-   df['B'] = pd.arrays.SparseArray([0, 0])
-   df['B'].dtype
-
-The ``SparseDataFrame.default_kind`` and ``SparseDataFrame.default_fill_value`` attributes
-have no replacement.
 
 .. _sparse.scipysparse:
 

doc/source/whatsnew/v0.25.0.rst

@@ -921,7 +921,7 @@ by a ``Series`` or ``DataFrame`` with sparse values.
    df = pd.DataFrame({"A": pd.arrays.SparseArray([0, 0, 1, 2])})
    df.dtypes
 
-The memory usage of the two approaches is identical. See :ref:`sparse.migration` for more (:issue:`19239`).
+The memory usage of the two approaches is identical (:issue:`19239`).
 
 msgpack format
 ^^^^^^^^^^^^^^

doc/source/whatsnew/v1.0.0.rst

@@ -872,8 +872,7 @@ Removal of prior version deprecations/changes
 
 ``SparseSeries``, ``SparseDataFrame`` and the ``DataFrame.to_sparse`` method
 have been removed (:issue:`28425`). We recommend using a ``Series`` or
-``DataFrame`` with sparse values instead. See :ref:`sparse.migration` for help
-with migrating existing code.
+``DataFrame`` with sparse values instead.
 
 .. _whatsnew_100.matplotlib_units:
 

doc/source/whatsnew/v2.1.0.rst

@@ -101,9 +101,12 @@ Deprecations
 - Deprecated :meth:`DataFrame._data` and :meth:`Series._data`, use public APIs instead (:issue:`33333`)
 - Deprecating pinning ``group.name`` to each group in :meth:`SeriesGroupBy.aggregate` aggregations; if your operation requires utilizing the groupby keys, iterate over the groupby object instead (:issue:`41090`)
 - Deprecated the default of ``observed=False`` in :meth:`DataFrame.groupby` and :meth:`Series.groupby`; this will default to ``True`` in a future version (:issue:`43999`)
+- Deprecated :meth:`DataFrameGroupBy.dtypes`, check ``dtypes`` on the underlying object instead (:issue:`51045`)
 - Deprecated ``axis=1`` in :meth:`DataFrame.groupby` and in :class:`Grouper` constructor, do ``frame.T.groupby(...)`` instead (:issue:`51203`)
 - Deprecated passing a :class:`DataFrame` to :meth:`DataFrame.from_records`, use :meth:`DataFrame.set_index` or :meth:`DataFrame.drop` instead (:issue:`51353`)
 - Deprecated accepting slices in :meth:`DataFrame.take`, call ``obj[slicer]`` or pass a sequence of integers instead (:issue:`51539`)
+- Deprecated ``axis=1`` in :meth:`DataFrame.ewm`, :meth:`DataFrame.rolling`, :meth:`DataFrame.expanding`, transpose before calling the method instead (:issue:`51778`)
+- Deprecated the ``axis`` keyword in :meth:`DataFrame.ewm`, :meth:`Series.ewm`, :meth:`DataFrame.rolling`, :meth:`Series.rolling`, :meth:`DataFrame.expanding`, :meth:`Series.expanding` (:issue:`51778`)
 - Deprecated 'method', 'limit', and 'fill_axis' keywords in :meth:`DataFrame.align` and :meth:`Series.align`, explicitly call ``fillna`` on the alignment results instead (:issue:`51856`)
 - Deprecated the 'axis' keyword in :meth:`.GroupBy.idxmax`, :meth:`.GroupBy.idxmin`, :meth:`.GroupBy.fillna`, :meth:`.GroupBy.take`, :meth:`.GroupBy.skew`, :meth:`.GroupBy.rank`, :meth:`.GroupBy.cumprod`, :meth:`.GroupBy.cumsum`, :meth:`.GroupBy.cummax`, :meth:`.GroupBy.cummin`, :meth:`.GroupBy.pct_change`, :meth:`GroupBy.diff`, :meth:`.GroupBy.shift`, and :meth:`DataFrameGroupBy.corrwith`; for ``axis=1`` operate on the underlying :class:`DataFrame` instead (:issue:`50405`, :issue:`51046`)
 -

pandas/core/apply.py

@@ -656,10 +656,6 @@ def columns(self) -> Index:
     def values(self):
         return self.obj.values
 
-    @cache_readonly
-    def dtypes(self) -> Series:
-        return self.obj.dtypes
-
     def apply(self) -> DataFrame | Series:
         """compute the results"""
         # dispatch to agg

pandas/core/generic.py

@@ -11956,12 +11956,32 @@ def rolling(
         center: bool_t = False,
         win_type: str | None = None,
         on: str | None = None,
-        axis: Axis = 0,
+        axis: Axis | lib.NoDefault = lib.no_default,
         closed: str | None = None,
         step: int | None = None,
         method: str = "single",
     ) -> Window | Rolling:
-        axis = self._get_axis_number(axis)
+        if axis is not lib.no_default:
+            axis = self._get_axis_number(axis)
+            name = "rolling"
+            if axis == 1:
+                warnings.warn(
+                    f"Support for axis=1 in {type(self).__name__}.{name} is "
+                    "deprecated and will be removed in a future version. "
+                    f"Use obj.T.{name}(...) instead",
+                    FutureWarning,
+                    stacklevel=find_stack_level(),
+                )
+            else:
+                warnings.warn(
+                    f"The 'axis' keyword in {type(self).__name__}.{name} is "
+                    "deprecated and will be removed in a future version. "
+                    "Call the method without the axis keyword instead.",
+                    FutureWarning,
+                    stacklevel=find_stack_level(),
+                )
+        else:
+            axis = 0
 
         if win_type is not None:
             return Window(
@@ -11995,10 +12015,30 @@ def rolling(
     def expanding(
         self,
         min_periods: int = 1,
-        axis: Axis = 0,
+        axis: Axis | lib.NoDefault = lib.no_default,
         method: str = "single",
     ) -> Expanding:
-        axis = self._get_axis_number(axis)
+        if axis is not lib.no_default:
+            axis = self._get_axis_number(axis)
+            name = "expanding"
+            if axis == 1:
+                warnings.warn(
+                    f"Support for axis=1 in {type(self).__name__}.{name} is "
+                    "deprecated and will be removed in a future version. "
+                    f"Use obj.T.{name}(...) instead",
+                    FutureWarning,
+                    stacklevel=find_stack_level(),
+                )
+            else:
+                warnings.warn(
+                    f"The 'axis' keyword in {type(self).__name__}.{name} is "
+                    "deprecated and will be removed in a future version. "
+                    "Call the method without the axis keyword instead.",
+                    FutureWarning,
+                    stacklevel=find_stack_level(),
+                )
+        else:
+            axis = 0
         return Expanding(self, min_periods=min_periods, axis=axis, method=method)
 
     @final
@@ -12012,11 +12052,32 @@ def ewm(
         min_periods: int | None = 0,
         adjust: bool_t = True,
         ignore_na: bool_t = False,
-        axis: Axis = 0,
+        axis: Axis | lib.NoDefault = lib.no_default,
         times: np.ndarray | DataFrame | Series | None = None,
         method: str = "single",
     ) -> ExponentialMovingWindow:
-        axis = self._get_axis_number(axis)
+        if axis is not lib.no_default:
+            axis = self._get_axis_number(axis)
+            name = "ewm"
+            if axis == 1:
+                warnings.warn(
+                    f"Support for axis=1 in {type(self).__name__}.{name} is "
+                    "deprecated and will be removed in a future version. "
+                    f"Use obj.T.{name}(...) instead",
+                    FutureWarning,
+                    stacklevel=find_stack_level(),
+                )
+            else:
+                warnings.warn(
+                    f"The 'axis' keyword in {type(self).__name__}.{name} is "
+                    "deprecated and will be removed in a future version. "
+                    "Call the method without the axis keyword instead.",
+                    FutureWarning,
+                    stacklevel=find_stack_level(),
+                )
+        else:
+            axis = 0
+
         return ExponentialMovingWindow(
             self,
             com=com,

pandas/core/groupby/generic.py

@@ -2637,6 +2637,14 @@ def hist(
     @property
     @doc(DataFrame.dtypes.__doc__)
     def dtypes(self) -> Series:
+        # GH#51045
+        warnings.warn(
+            f"{type(self).__name__}.dtypes is deprecated and will be removed in "
+            "a future version. Check the dtypes on the base object instead",
+            FutureWarning,
+            stacklevel=find_stack_level(),
+        )
+
         # error: Incompatible return value type (got "DataFrame", expected "Series")
         return self.apply(lambda df: df.dtypes)  # type: ignore[return-value]
 

pandas/tests/groupby/test_allowlist.py

@@ -274,7 +274,9 @@ def test_groupby_selection_other_methods(df):
 
     # methods which aren't just .foo()
     tm.assert_frame_equal(g.fillna(0), g_exp.fillna(0))
-    tm.assert_frame_equal(g.dtypes, g_exp.dtypes)
+    msg = "DataFrameGroupBy.dtypes is deprecated"
+    with tm.assert_produces_warning(FutureWarning, match=msg):
+        tm.assert_frame_equal(g.dtypes, g_exp.dtypes)
     tm.assert_frame_equal(g.apply(lambda x: x.sum()), g_exp.apply(lambda x: x.sum()))
 
     tm.assert_frame_equal(g.resample("D").mean(), g_exp.resample("D").mean())

pandas/tests/window/test_api.py

@@ -129,7 +129,9 @@ def test_agg(step):
 def test_multi_axis_1_raises(func):
     # GH#46904
     df = DataFrame({"a": [1, 1, 2], "b": [3, 4, 5], "c": [6, 7, 8]})
-    r = df.rolling(window=3, axis=1)
+    msg = "Support for axis=1 in DataFrame.rolling is deprecated"
+    with tm.assert_produces_warning(FutureWarning, match=msg):
+        r = df.rolling(window=3, axis=1)
     with pytest.raises(NotImplementedError, match="axis other than 0 is not supported"):
         r.agg(func)
 
@@ -344,29 +346,28 @@ def test_dont_modify_attributes_after_methods(
 
 def test_centered_axis_validation(step):
     # ok
-    Series(np.ones(10)).rolling(window=3, center=True, axis=0, step=step).mean()
+    msg = "The 'axis' keyword in Series.rolling is deprecated"
+    with tm.assert_produces_warning(FutureWarning, match=msg):
+        Series(np.ones(10)).rolling(window=3, center=True, axis=0, step=step).mean()
 
     # bad axis
     msg = "No axis named 1 for object type Series"
     with pytest.raises(ValueError, match=msg):
         Series(np.ones(10)).rolling(window=3, center=True, axis=1, step=step).mean()
 
     # ok ok
-    DataFrame(np.ones((10, 10))).rolling(
-        window=3, center=True, axis=0, step=step
-    ).mean()
-    DataFrame(np.ones((10, 10))).rolling(
-        window=3, center=True, axis=1, step=step
-    ).mean()
+    df = DataFrame(np.ones((10, 10)))
+    msg = "The 'axis' keyword in DataFrame.rolling is deprecated"
+    with tm.assert_produces_warning(FutureWarning, match=msg):
+        df.rolling(window=3, center=True, axis=0, step=step).mean()
+    msg = "Support for axis=1 in DataFrame.rolling is deprecated"
+    with tm.assert_produces_warning(FutureWarning, match=msg):
+        df.rolling(window=3, center=True, axis=1, step=step).mean()
 
     # bad axis
     msg = "No axis named 2 for object type DataFrame"
     with pytest.raises(ValueError, match=msg):
-        (
-            DataFrame(np.ones((10, 10)))
-            .rolling(window=3, center=True, axis=2, step=step)
-            .mean()
-        )
+        (df.rolling(window=3, center=True, axis=2, step=step).mean())
 
 
 def test_rolling_min_min_periods(step):

pandas/tests/window/test_apply.py

@@ -321,6 +321,8 @@ def test_center_reindex_frame(raw, frame):
 def test_axis1(raw):
     # GH 45912
     df = DataFrame([1, 2])
-    result = df.rolling(window=1, axis=1).apply(np.sum, raw=raw)
+    msg = "Support for axis=1 in DataFrame.rolling is deprecated"
+    with tm.assert_produces_warning(FutureWarning, match=msg):
+        result = df.rolling(window=1, axis=1).apply(np.sum, raw=raw)
     expected = DataFrame([1.0, 2.0])
     tm.assert_frame_equal(result, expected)

pandas/tests/window/test_ewm.py

@@ -198,7 +198,9 @@ def test_float_dtype_ewma(func, expected, float_numpy_dtype):
     df = DataFrame(
         {0: range(5), 1: range(6, 11), 2: range(10, 20, 2)}, dtype=float_numpy_dtype
     )
-    e = df.ewm(alpha=0.5, axis=1)
+    msg = "Support for axis=1 in DataFrame.ewm is deprecated"
+    with tm.assert_produces_warning(FutureWarning, match=msg):
+        e = df.ewm(alpha=0.5, axis=1)
     result = getattr(e, func)()
 
     tm.assert_frame_equal(result, expected)