Skip to content

BUG: pandas.to_datetime raises exception when more than 50 values needs coercion to NaT (#43732) #43791

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from

Conversation

thomasqueirozb
Copy link

@thomasqueirozb thomasqueirozb commented Sep 28, 2021

@jbrockmendel
Copy link
Member

can you write a test

@jreback jreback added Bug Datetime Datetime data dtype labels Sep 28, 2021
@thomasqueirozb
Copy link
Author

Constructing an Index from a value that will resolve to NaT causes an error, but I'm not sure if this is by design.

import datetime
import pytz
import pandas as pd

# t is a date after 2262 that should be coerced into NaT
t = datetime.datetime(3000, 1, 1, 0, 0, 0, 0, pytz.UTC)
print(pd.Index([t], tz="utc")) # pandas._libs.tslibs.np_datetime.OutOfBoundsDatetime: Out of bounds nanosecond timestamp: 3000-01-01 00:00:00

@jreback
Copy link
Contributor

jreback commented Oct 3, 2021

Constructing an Index from a value that will resolve to NaT causes an error, but I'm not sure if this is by design.

import datetime
import pytz
import pandas as pd

# t is a date after 2262 that should be coerced into NaT
t = datetime.datetime(3000, 1, 1, 0, 0, 0, 0, pytz.UTC)
print(pd.Index([t], tz="utc")) # pandas._libs.tslibs.np_datetime.OutOfBoundsDatetime: Out of bounds nanosecond timestamp: 3000-01-01 00:00:00

this is expected, it is an out of bounds date.

@HenryRocha
Copy link

@jbrockmendel we wrote some tests, is there anything else we need to do?

@thomasqueirozb thomasqueirozb force-pushed the issue_43732 branch 2 times, most recently from 1bab61c to b257ccf Compare October 19, 2021 23:40
Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code & tests look ok

but CI seems to be failing on lots of things

@jreback jreback added this to the 1.4 milestone Nov 12, 2021
thomasqueirozb and others added 2 commits November 12, 2021 17:01
…ds coercion to NaT (pandas-dev#43732)

Raise proper error from objects_to_datetime64ns
Check for OutOfBoundsDatetime in _maybe_cache

Co-authored-by: André Elimelek de Weber (andrekwr) <[email protected]>
Co-authored-by: Henry Rocha (HenryRocha) <[email protected]>
Co-authored-by: André Elimelek de Weber (andrekwr) <[email protected]>
Co-authored-by: Henry Rocha (HenryRocha) <[email protected]>
@jreback
Copy link
Contributor

jreback commented Nov 14, 2021

pls merge master, but tests seem to be failing

1 similar comment
@jreback
Copy link
Contributor

jreback commented Nov 28, 2021

pls merge master, but tests seem to be failing

@jreback jreback removed this from the 1.4 milestone Dec 23, 2021
@jreback
Copy link
Contributor

jreback commented Dec 23, 2021

is worthwhile but needs to be passing (and pls merge master)

@jreback
Copy link
Contributor

jreback commented Jan 16, 2022

would welcome this change but its stale. pls ping if you want to continue.

@jreback jreback closed this Jan 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Datetime Datetime data dtype
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: pandas.to_datetime raises exception when more than 50 values needs coercion to NaT
4 participants