Skip to content

to_csv and read_csv return inconsistent result when value contains \r #27737

Closed
@KaitoHH

Description

@KaitoHH

Code Sample, a copy-pastable example if possible

dff = pd.DataFrame(data={'a':['a1\r','a2'],'b':['b1','b2']})
print("===BEFORE SAVE===")
print(dff)
dff.to_csv('test.csv', index=False)
print("===AFTER SAVE===")
print(pd.read_csv('test.csv'))

The result will be

===BEFORE SAVE===
      a   b
0  a1\r  b1
1    a2  b2
===AFTER SAVE===
     a    b
0   a1  NaN
1  NaN   b1
2   a2   b2

Problem description

The problem can be resolved by adding quoting=csv.QUOTE_NONNUMERIC option, but it seems like a bug under default parameters of to_csv.

Expected Output

===BEFORE SAVE===
      a   b
0  a1\r  b1
1    a2  b2
===AFTER SAVE===
      a   b
0  a1\r  b1
1    a2  b2

Output of pd.show_versions()

INSTALLED VERSIONS
------------------
commit: None
python: 3.6.8.final.0
python-bits: 64
OS: Linux
OS-release: 4.4.0-151-generic
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8

pandas: 0.24.2
pytest: 4.6.3
pip: 19.1.1
setuptools: 41.0.1
Cython: 0.29.13
numpy: 1.16.4
scipy: 1.3.0
pyarrow: None
xarray: None
IPython: 7.5.0
sphinx: 2.1.2
patsy: None
dateutil: 2.8.0
pytz: 2019.1
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: 3.1.0
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml.etree: None
bs4: 4.8.0
html5lib: None
sqlalchemy: 1.3.5
pymysql: None
psycopg2: None
jinja2: 2.10.1
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
gcsfs: None

Metadata

Metadata

Assignees

No one assigned

    Labels

    IO CSVread_csv, to_csv

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions