BUG: to_csv line endings with compression

doc/source/user_guide/io.rst

@@ -1689,7 +1689,7 @@ The ``Series`` and ``DataFrame`` objects have an instance method ``to_csv`` whic
 allows storing the contents of the object as a comma-separated-values file. The
 function takes a number of arguments. Only the first is required.
 
-* ``path_or_buf``: A string path to the file to write or a StringIO
+* ``path_or_buf``: A string path to the file to write or a file object.  If a file object it must be opened with `newline=''`
 * ``sep`` : Field delimiter for the output file (default ",")
 * ``na_rep``: A string representation of a missing value (default '')
 * ``float_format``: Format string for floating point numbers
@@ -1702,7 +1702,7 @@ function takes a number of arguments. Only the first is required.
 * ``mode`` : Python write mode, default 'w'
 * ``encoding``: a string representing the encoding to use if the contents are
   non-ASCII, for Python versions prior to 3
-* ``line_terminator``: Character sequence denoting line end (default '\\n')
+* ``line_terminator``: Character sequence denoting line end (default `os.linesep`)
 * ``quoting``: Set quoting rules as in csv module (default csv.QUOTE_MINIMAL). Note that if you have set a `float_format` then floats are converted to strings and csv.QUOTE_NONNUMERIC will treat them as non-numeric
 * ``quotechar``: Character used to quote fields (default '"')
 * ``doublequote``: Control quoting of ``quotechar`` in fields (default True)

doc/source/whatsnew/v0.24.2.rst

@@ -32,6 +32,7 @@ Fixed Regressions
 - Fixed regression in creating a period-dtype array from a read-only NumPy array of period objects. (:issue:`25403`)
 - Fixed regression in :class:`Categorical`, where constructing it from a categorical ``Series`` and an explicit ``categories=`` that differed from that in the ``Series`` created an invalid object which could trigger segfaults. (:issue:`25318`)
 - Fixed pip installing from source into an environment without NumPy (:issue:`25193`)
+- Fixed regression in :meth:`DataFrame.to_csv` writing duplicate line endings with gzip compress (:issue:`25311`)
 
 .. _whatsnew_0242.enhancements:
 

pandas/core/generic.py

@@ -2920,7 +2920,8 @@ def to_csv(self, path_or_buf=None, sep=",", na_rep='', float_format=None,
         ----------
         path_or_buf : str or file handle, default None
             File path or object, if None is provided the result is returned as
-            a string.
+            a string.  If a file object is passed it should be opened with
+            `newline=''`, disabling universal newlines.
 
             .. versionchanged:: 0.24.0
 

pandas/io/common.py

@@ -434,7 +434,7 @@ def _get_handle(path_or_buf, mode, encoding=None, compression=None,
     if (compat.PY3 and is_text and
             (compression or isinstance(f, need_text_wrapping))):
         from io import TextIOWrapper
-        f = TextIOWrapper(f, encoding=encoding)
+        f = TextIOWrapper(f, encoding=encoding, newline='')
         handles.append(f)
 
     if memory_map and hasattr(f, 'fileno'):

pandas/tests/frame/test_to_csv.py

@@ -4,6 +4,7 @@
 
 import csv
 import os
+import gzip
 
 import numpy as np
 import pytest
@@ -1221,3 +1222,14 @@ def test_multi_index_header(self):
                          '1,5,6,7,8']
         expected = tm.convert_rows_list_to_csv_str(expected_rows)
         assert result == expected
+
+    def test_gz_lineend(self):
+        # GH 25311
+        df = pd.DataFrame({'a': [1, 2]})
+        expected_rows = ['a', '1', '2']
+        expected = tm.convert_rows_list_to_csv_str(expected_rows)
+        with ensure_clean('__test_gz_lineend.csv.gz') as path:
+            df.to_csv(path, index=False)
+            result = gzip.open(path, mode='rt', newline='').read()
+
+        assert result == expected