df.to_csv ignores compression when provided with a file handle

[guinny65]

Code at issue

with open(outputfile, "w") as file_handle:
    df2.to_csv(file_handle, compression='gzip')

Problem description

The written file is not gzip compressed.

Expected Output

The output should be a gzip compressed csv file. Similar to what is obtained when using:

df2.to_csv('/path/to/file.csv.gz',compression='gzip')

[WillAyd]

Maybe related to #21144

[minggli]

>>> import os
>>> import pandas as pd
>>> from pandas import *
>>>
>>> pd.__version__
'0.20.3'
>>>
>>> df = DataFrame(100 * [[123, 234, 435]])
>>>
>>> with open('test_compressed', 'w') as fh:
...     df.to_csv(fh, compression='gzip')
...
>>> fh_size = os.path.getsize('test_compressed')
>>> df.to_csv('test_compressed', compression='gzip')
>>> f_size = os.path.getsize('test_compressed')
>>>
>>> os.remove('test_compressed')
>>> assert fh_size == f_size
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AssertionError

looks like an existing behaviour dating back to version 0.20 or earlier.

Actually, the documentation of compression= says:

compression : string, optional

A string representing the compression to use in the output file. Allowed values are ‘gzip’, ‘bz2’, ‘zip’, ‘xz’. This input is only used when the first argument is a filename.

so it is not supported but may be a new use case perhaps?