Skip to content

BUG: Fix crash using to_csv with QUOTE_NONE #4752

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 5, 2013

Conversation

jtratner
Copy link
Contributor

@jtratner jtratner commented Sep 5, 2013

Fixes #4328.

Now passes quotechar to csv's writer. Default is same as csv ('"'), but sets to None with quoting=csv.QUOTE_NONE.

@ghost ghost assigned jtratner Sep 5, 2013
@jtratner
Copy link
Contributor Author

jtratner commented Sep 5, 2013

@jreback can you glance at this before I merge?

@jreback
Copy link
Contributor

jreback commented Sep 5, 2013

looks good

weird csv bug

@jtratner
Copy link
Contributor Author

jtratner commented Sep 5, 2013

yeah, the issue is that the default quotechar is '"' for csv itself - I wonder whether I can trigger this with normal csv writing. Maybe worth filing a bug report...

@jtratner
Copy link
Contributor Author

jtratner commented Sep 5, 2013

yep, easy to trigger. Maybe I'll file an issue...

Now passes `quotechar` to csv's writer. Default is same as csv (`'"'`),
but sets to `None` with `quoting=csv.QUOTE_NONE`.
jtratner added a commit that referenced this pull request Sep 5, 2013
BUG: Fix crash using to_csv with QUOTE_NONE
@jtratner jtratner merged commit f373864 into pandas-dev:master Sep 5, 2013
@jtratner jtratner deleted the fix-to_csv_QUOTE_NONE-GH4328 branch September 5, 2013 02:37
@jtratner
Copy link
Contributor Author

jtratner commented Sep 6, 2013

I think this was the wrong solution actually...there's a specific parameter escapechar defined that is used only when quoting is set to QUOTE_NONE, so we probably should have just added an extra kwarg escapechar and used that instead (and pushed the user to make a decision). it's such a minor case, not sure if it's worth revisiting...

@pronojitsaha
Copy link

Hi,
I am having this issue still now. My data frame contains unicode strings. Now when I use 'to_csv' without any quoting parameter, I get a csv file where certain fields (2 out of 11 to be exact) are in double quotes. For example:
28,"May 8, 2013 10:22 AM","76th St, Yates Blvd",41.75717,-87.56633,3,-,1,4,-
Now when I use csv.QUOTE_NONE i get the following error:
Error: need to escape, but no escapechar set
So i do the following (quoting=csv.QUOTE_NONE, escapechar='"'), and I get this:
28,May 8", 2013 10:22 AM,76th St", Yates Blvd,41.75717,-87.56633,3,-,1,4,-
So I get a new quote (") in those 2 fields. How to get around this problem? Thanks.

@jreback
Copy link
Contributor

jreback commented Aug 10, 2015

you need to provide a copy-pastable example which reproduces

@pronojitsaha
Copy link

Please find below:

id = [u'1']
date = [u'Jan 12, 2013 08:30 AM']
location = [u'Ashland Avenue, Polk Street']
vehicles = [u'4']
drunken_persons = [u'-']
fatalities = [u'3']
persons = [u'4']
pedestrians = [u'-']

data = pd.DataFrame({'#':id, 'Date':date, 'Vehicles':vehicles, 'Drunken Persons':drunken_persons, 'Fatalities':fatalities, 'Persons':persons,  'Pedestrians':pedestrians, 'Location':location})

data[['#','Date','Location','Vehicles','Drunken Persons','Fatalities','Persons', 'Pedestrians']].astype(str).to_csv('Test.csv',  sep=',', index = False, quoting=csv.QUOTE_NONE, escapechar='"')

@jreback
Copy link
Contributor

jreback commented Aug 10, 2015

pls only an example that is not subject to these types of external dependencies.

@pronojitsaha
Copy link

OK. I have updated the original post.

@jreback
Copy link
Contributor

jreback commented Aug 10, 2015

and this is a merged issues, so pls open a new one

@pronojitsaha
Copy link

Ok, done. Here - #10783

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: Crash using to_csv with QUOTE_NONE
3 participants