-
-
Notifications
You must be signed in to change notification settings - Fork 18.5k
PERF: improve resample perf #7673
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
else: | ||
max_stamp = masked.asi8.max() | ||
try: | ||
max_stamp = self[self.asi8 != tslib.iNaT].asi8.max() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
since nats are normally not present
you should check mask.any()
if this fails that you don't need to do self[mask] u can just use self
Thanks. Modified. Based on the cProfile,
|
it's the re sampling this shouldn't be 2x slower |
Period resample issue has been solved.
|
ok, looks a lot better. ping me when green (and see if the perf diff in timeseries resample is fixable). |
@sinhrks merged, but still havev a look at resample perf degredation |
Related to #7633. It gets better than the result attached #7633, but still slower more than 1.2 times compared to 1.4.0
Modified:
import
inIndex.max/min
max
call fromresample/_get_time_bins
and_get_range_edges
.lib/generate_bins_dt64
andtslib/period_asfreq_arr
.Remaining bottlenecks are
NaT
masking performed inlib/generate_bins_dt64
andtslib/period_asfreq_arr
. Is there any better way to do that?