DEPR: Series/DataFrame.append (#35407) (#44539)

Author: neinkeinkaffee

doc/source/getting_started/comparison/comparison_with_spreadsheets.rst

@@ -435,13 +435,14 @@ The equivalent in pandas:
 Adding a row
 ~~~~~~~~~~~~
 
-Assuming we are using a :class:`~pandas.RangeIndex` (numbered ``0``, ``1``, etc.), we can use :meth:`DataFrame.append` to add a row to the bottom of a ``DataFrame``.
+Assuming we are using a :class:`~pandas.RangeIndex` (numbered ``0``, ``1``, etc.), we can use :func:`concat` to add a row to the bottom of a ``DataFrame``.
 
 .. ipython:: python
 
     df
-    new_row = {"class": "E", "student_count": 51, "all_pass": True}
-    df.append(new_row, ignore_index=True)
+    new_row = pd.DataFrame([["E", 51, True]],
+                           columns=["class", "student_count", "all_pass"])
+    pd.concat([df, new_row])
 
 
 Find and Replace

doc/source/user_guide/10min.rst

@@ -478,7 +478,6 @@ Concatenating pandas objects together with :func:`concat`:
    a row requires a copy, and may be expensive. We recommend passing a
    pre-built list of records to the :class:`DataFrame` constructor instead
    of building a :class:`DataFrame` by iteratively appending records to it.
-   See :ref:`Appending to dataframe <merging.concatenation>` for more.
 
 Join
 ~~~~

doc/source/user_guide/cookbook.rst

@@ -929,9 +929,9 @@ Valid frequency arguments to Grouper :ref:`Timeseries <timeseries.offset_aliases
 Merge
 -----
 
-The :ref:`Concat <merging.concatenation>` docs. The :ref:`Join <merging.join>` docs.
+The :ref:`Join <merging.join>` docs.
 
-`Append two dataframes with overlapping index (emulate R rbind)
+`Concatenate two dataframes with overlapping index (emulate R rbind)
 <https://stackoverflow.com/questions/14988480/pandas-version-of-rbind>`__
 
 .. ipython:: python
@@ -944,7 +944,7 @@ Depending on df construction, ``ignore_index`` may be needed
 
 .. ipython:: python
 
-   df = df1.append(df2, ignore_index=True)
+   df = pd.concat([df1, df2], ignore_index=True)
    df
 
 `Self Join of a DataFrame

doc/source/user_guide/merging.rst

@@ -237,59 +237,6 @@ Similarly, we could index before the concatenation:
    p.plot([df1, df4], result, labels=["df1", "df4"], vertical=False);
    plt.close("all");
 
-.. _merging.concatenation:
-
-Concatenating using ``append``
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-A useful shortcut to :func:`~pandas.concat` are the :meth:`~DataFrame.append`
-instance methods on ``Series`` and ``DataFrame``. These methods actually predated
-``concat``. They concatenate along ``axis=0``, namely the index:
-
-.. ipython:: python
-
-   result = df1.append(df2)
-
-.. ipython:: python
-   :suppress:
-
-   @savefig merging_append1.png
-   p.plot([df1, df2], result, labels=["df1", "df2"], vertical=True);
-   plt.close("all");
-
-In the case of ``DataFrame``, the indexes must be disjoint but the columns do not
-need to be:
-
-.. ipython:: python
-
-   result = df1.append(df4, sort=False)
-
-.. ipython:: python
-   :suppress:
-
-   @savefig merging_append2.png
-   p.plot([df1, df4], result, labels=["df1", "df4"], vertical=True);
-   plt.close("all");
-
-``append`` may take multiple objects to concatenate:
-
-.. ipython:: python
-
-   result = df1.append([df2, df3])
-
-.. ipython:: python
-   :suppress:
-
-   @savefig merging_append3.png
-   p.plot([df1, df2, df3], result, labels=["df1", "df2", "df3"], vertical=True);
-   plt.close("all");
-
-.. note::
-
-   Unlike the :py:meth:`~list.append` method, which appends to the original list
-   and returns ``None``, :meth:`~DataFrame.append`  here **does not** modify
-   ``df1`` and returns its copy with ``df2`` appended.
-
 .. _merging.ignore_index:
 
 Ignoring indexes on the concatenation axis
@@ -309,19 +256,6 @@ do this, use the ``ignore_index`` argument:
    p.plot([df1, df4], result, labels=["df1", "df4"], vertical=True);
    plt.close("all");
 
-This is also a valid argument to :meth:`DataFrame.append`:
-
-.. ipython:: python
-
-   result = df1.append(df4, ignore_index=True, sort=False)
-
-.. ipython:: python
-   :suppress:
-
-   @savefig merging_append_ignore_index.png
-   p.plot([df1, df4], result, labels=["df1", "df4"], vertical=True);
-   plt.close("all");
-
 .. _merging.mixed_ndims:
 
 Concatenating with mixed ndims
@@ -473,14 +407,13 @@ like GroupBy where the order of a categorical variable is meaningful.
 Appending rows to a DataFrame
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-While not especially efficient (since a new object must be created), you can
-append a single row to a ``DataFrame`` by passing a ``Series`` or dict to
-``append``, which returns a new ``DataFrame`` as above.
+If you have a series that you want to append as a single row to a ``DataFrame``, you can convert the row into a
+``DataFrame`` and use ``concat``
 
 .. ipython:: python
 
    s2 = pd.Series(["X0", "X1", "X2", "X3"], index=["A", "B", "C", "D"])
-   result = df1.append(s2, ignore_index=True)
+   result = pd.concat([df1, s2.to_frame().T], ignore_index=True)
 
 .. ipython:: python
    :suppress:
@@ -493,20 +426,6 @@ You should use ``ignore_index`` with this method to instruct DataFrame to
 discard its index. If you wish to preserve the index, you should construct an
 appropriately-indexed DataFrame and append or concatenate those objects.
 
-You can also pass a list of dicts or Series:
-
-.. ipython:: python
-
-   dicts = [{"A": 1, "B": 2, "C": 3, "X": 4}, {"A": 5, "B": 6, "C": 7, "Y": 8}]
-   result = df1.append(dicts, ignore_index=True, sort=False)
-
-.. ipython:: python
-   :suppress:
-
-   @savefig merging_append_dits.png
-   p.plot([df1, pd.DataFrame(dicts)], result, labels=["df1", "dicts"], vertical=True);
-   plt.close("all");
-
 .. _merging.join:
 
 Database-style DataFrame or named Series joining/merging

doc/source/whatsnew/v0.6.1.rst

@@ -6,7 +6,7 @@ Version 0.6.1 (December 13, 2011)
 
 New features
 ~~~~~~~~~~~~
-- Can :ref:`append single rows <merging.append.row>` (as Series) to a DataFrame
+- Can append single rows (as Series) to a DataFrame
 - Add Spearman and Kendall rank :ref:`correlation <computation.correlation>`
   options to Series.corr and DataFrame.corr (:issue:`428`)
 - :ref:`Added <indexing.basics.get_value>` ``get_value`` and ``set_value`` methods to

doc/source/whatsnew/v0.7.0.rst

@@ -19,7 +19,7 @@ New features
   intersection of the other axes. Improves performance of ``Series.append`` and
   ``DataFrame.append`` (:issue:`468`, :issue:`479`, :issue:`273`)
 
-- :ref:`Can <merging.concatenation>` pass multiple DataFrames to
+- Can pass multiple DataFrames to
   ``DataFrame.append`` to concatenate (stack) and multiple Series to
   ``Series.append`` too
 

doc/source/whatsnew/v1.4.0.rst

@@ -275,7 +275,7 @@ ignored when finding the concatenated dtype.  These are now consistently *not* i
 
     df1 = pd.DataFrame({"bar": [pd.Timestamp("2013-01-01")]}, index=range(1))
     df2 = pd.DataFrame({"bar": np.nan}, index=range(1, 2))
-    res = df1.append(df2)
+    res = pd.concat([df1, df2])
 
 Previously, the float-dtype in ``df2`` would be ignored so the result dtype would be ``datetime64[ns]``. As a result, the ``np.nan`` would be cast to ``NaT``.
 
@@ -510,6 +510,49 @@ when given numeric data, but in the future, a :class:`NumericIndex` will be retu
     Out [4]: NumericIndex([1, 2, 3], dtype='uint64')
 
 
+.. _whatsnew_140.deprecations.frame_series_append:
+
+Deprecated Frame.append and Series.append
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+:meth:`DataFrame.append` and :meth:`Series.append` have been deprecated and will be removed in Pandas 2.0.
+Use :func:`pandas.concat` instead (:issue:`35407`).
+
+*Deprecated syntax*
+
+.. code-block:: ipython
+
+    In [1]: pd.Series([1, 2]).append(pd.Series([3, 4])
+    Out [1]:
+    <stdin>:1: FutureWarning: The series.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
+    0    1
+    1    2
+    0    3
+    1    4
+    dtype: int64
+
+    In [2]: df1 = pd.DataFrame([[1, 2], [3, 4]], columns=list('AB'))
+    In [3]: df2 = pd.DataFrame([[5, 6], [7, 8]], columns=list('AB'))
+    In [4]: df1.append(df2)
+    Out [4]:
+    <stdin>:1: FutureWarning: The series.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.
+       A  B
+    0  1  2
+    1  3  4
+    0  5  6
+    1  7  8
+
+*Recommended syntax*
+
+.. ipython:: python
+
+    pd.concat([pd.Series([1, 2]), pd.Series([3, 4])])
+
+    df1 = pd.DataFrame([[1, 2], [3, 4]], columns=list('AB'))
+    df2 = pd.DataFrame([[5, 6], [7, 8]], columns=list('AB'))
+    pd.concat([df1, df2])
+
+
 .. _whatsnew_140.deprecations.other:
 
 Other Deprecations

pandas/core/frame.py

@@ -3261,9 +3261,10 @@ def memory_usage(self, index: bool = True, deep: bool = False) -> Series:
             index=self.columns,
         )
         if index:
-            result = self._constructor_sliced(
+            index_memory_usage = self._constructor_sliced(
                 self.index.memory_usage(deep=deep), index=["Index"]
-            ).append(result)
+            )
+            result = index_memory_usage._append(result)
         return result
 
     def transpose(self, *args, copy: bool = False) -> DataFrame:
@@ -9003,6 +9004,23 @@ def append(
         3  3
         4  4
         """
+        warnings.warn(
+            "The frame.append method is deprecated "
+            "and will be removed from pandas in a future version. "
+            "Use pandas.concat instead.",
+            FutureWarning,
+            stacklevel=find_stack_level(),
+        )
+
+        return self._append(other, ignore_index, verify_integrity, sort)
+
+    def _append(
+        self,
+        other,
+        ignore_index: bool = False,
+        verify_integrity: bool = False,
+        sort: bool = False,
+    ) -> DataFrame:
         combined_columns = None
         if isinstance(other, (Series, dict)):
             if isinstance(other, dict):
@@ -9728,7 +9746,9 @@ def c(x):
             idx_diff = result_index.difference(correl.index)
 
             if len(idx_diff) > 0:
-                correl = correl.append(Series([np.nan] * len(idx_diff), index=idx_diff))
+                correl = correl._append(
+                    Series([np.nan] * len(idx_diff), index=idx_diff)
+                )
 
         return correl
 

pandas/core/indexing.py

@@ -1993,7 +1993,7 @@ def _setitem_with_indexer_missing(self, indexer, value):
                     df = df.infer_objects()
                 self.obj._mgr = df._mgr
             else:
-                self.obj._mgr = self.obj.append(value)._mgr
+                self.obj._mgr = self.obj._append(value)._mgr
             self.obj._maybe_update_cacher(clear=True)
 
     def _ensure_iterable_column_indexer(self, column_indexer):

pandas/core/reshape/pivot.py

@@ -287,7 +287,7 @@ def _add_margins(
     if not values and isinstance(table, ABCSeries):
         # If there are no values and the table is a series, then there is only
         # one column in the data. Compute grand margin and return it.
-        return table.append(Series({key: grand_margin[margins_name]}))
+        return table._append(Series({key: grand_margin[margins_name]}))
 
     elif values:
         marginal_result_set = _generate_marginal_results(
@@ -325,7 +325,7 @@ def _add_margins(
         margin_dummy[cols] = margin_dummy[cols].apply(
             maybe_downcast_to_dtype, args=(dtype,)
         )
-    result = result.append(margin_dummy)
+    result = result._append(margin_dummy)
     result.index.names = row_names
 
     return result
@@ -738,7 +738,7 @@ def _normalize(table, normalize, margins: bool, margins_name="All"):
 
         elif normalize == "index":
             index_margin = index_margin / index_margin.sum()
-            table = table.append(index_margin)
+            table = table._append(index_margin)
             table = table.fillna(0)
             table.index = table_index
 
@@ -747,7 +747,7 @@ def _normalize(table, normalize, margins: bool, margins_name="All"):
             index_margin = index_margin / index_margin.sum()
             index_margin.loc[margins_name] = 1
             table = concat([table, column_margin], axis=1)
-            table = table.append(index_margin)
+            table = table._append(index_margin)
 
             table = table.fillna(0)
             table.index = table_index

pandas/core/series.py

@@ -2893,6 +2893,19 @@ def append(
         ...
         ValueError: Indexes have overlapping values: [0, 1, 2]
         """
+        warnings.warn(
+            "The series.append method is deprecated "
+            "and will be removed from pandas in a future version. "
+            "Use pandas.concat instead.",
+            FutureWarning,
+            stacklevel=find_stack_level(),
+        )
+
+        return self._append(to_append, ignore_index, verify_integrity)
+
+    def _append(
+        self, to_append, ignore_index: bool = False, verify_integrity: bool = False
+    ):
         from pandas.core.reshape.concat import concat
 
         if isinstance(to_append, (list, tuple)):