Skip to content

BUG: hour string automatically converted to datetime in groupby apply #5789

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jorisvandenbossche opened this issue Dec 28, 2013 · 0 comments · Fixed by #5790
Closed

BUG: hour string automatically converted to datetime in groupby apply #5789

jorisvandenbossche opened this issue Dec 28, 2013 · 0 comments · Fixed by #5790
Labels
Bug Dtype Conversions Unexpected or buggy dtype conversions Groupby
Milestone

Comments

@jorisvandenbossche
Copy link
Member

(Encountered with same SO question as #5786 and #5788. With following dataframe:

from StringIO import StringIO
s="""2011.05.16,00:00,1.40893
2011.05.16,01:00,1.40760
2011.05.16,02:00,1.40750
2011.05.16,03:00,1.40649
2011.05.17,02:00,1.40893
2011.05.17,03:00,1.40760
2011.05.17,04:00,1.40750
2011.05.17,05:00,1.40649
2011.05.18,02:00,1.40893
2011.05.18,03:00,1.40760
2011.05.18,04:00,1.40750
2011.05.18,05:00,1.40649"""
df = pd.read_csv(StringIO(s), header=None, names=['date', 'time', 'value'])

where all dates or hours are strings. When doing a groupby on the date and then accessing the value in column time where value is maximal, the hour string is converted to a datetime:

In [21]: df.groupby('date').apply(lambda x: x['time'][x['value'].idxmax()])
Out[21]: 
date
2011.05.16   2013-12-29 00:00:00
2011.05.17   2013-12-29 02:00:00
2011.05.18   2013-12-29 02:00:00
dtype: datetime64[ns]

while in 0.12 this remains a string:

In [63]: df.groupby('date').apply(lambda x: x['time'][x['value'].idxmax()])
Out[63]: 
date
2011.05.16    00:00
2011.05.17    02:00
2011.05.18    02:00
dtype: object

I think in this case this conversion can be highly inappropriate (and you have a chance that just because a string is looking like a date/time component it is converted, while this will not always be your intention).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Dtype Conversions Unexpected or buggy dtype conversions Groupby
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant