Skip to content

[BUG] TimedeltaIndex intersects with decreasing TimedeltaIndex #25661

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 5 commits into from

Conversation

makbigc
Copy link
Contributor

@makbigc makbigc commented Mar 11, 2019

Beside the bug of decreasing TimedeltaIndex, this PR also solve the following

In [2]: idx1 = pd.to_timedelta(range(1,20), unit='s')                           

In [3]: idx2 = pd.to_timedelta((1,2,3,4,5,6,7,8,9,10,20), unit='s')             

In [4]: result = idx1.intersection(idx2)                                        

In [5]: result                                                                  
Out[5]: 
TimedeltaIndex(['00:00:01', '00:00:02', '00:00:03', '00:00:04', '00:00:05',
                '00:00:06', '00:00:07', '00:00:08', '00:00:09', '00:00:10',
                '00:00:11', '00:00:12', '00:00:13', '00:00:14', '00:00:15',
                '00:00:16', '00:00:17', '00:00:18', '00:00:19'],
               dtype='timedelta64[ns]', freq=None)

The above is also added into the tests.

@codecov
Copy link

codecov bot commented Mar 11, 2019

Codecov Report

Merging #25661 into master will increase coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #25661      +/-   ##
==========================================
+ Coverage   91.29%   91.29%   +<.01%     
==========================================
  Files         173      173              
  Lines       52961    52966       +5     
==========================================
+ Hits        48350    48357       +7     
+ Misses       4611     4609       -2
Flag Coverage Δ
#multiple 89.87% <100%> (ø) ⬆️
#single 41.73% <0%> (-0.01%) ⬇️
Impacted Files Coverage Δ
pandas/core/indexes/timedeltas.py 90.95% <100%> (+0.68%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c4fa3c9...a7e6853. Read the comment docs.

@codecov
Copy link

codecov bot commented Mar 11, 2019

Codecov Report

Merging #25661 into master will decrease coverage by 0.52%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #25661      +/-   ##
==========================================
- Coverage   91.82%   91.29%   -0.53%     
==========================================
  Files         175      173       -2     
  Lines       52551    52966     +415     
==========================================
+ Hits        48256    48357     +101     
- Misses       4295     4609     +314
Flag Coverage Δ
#multiple 89.87% <100%> (-0.51%) ⬇️
#single 41.73% <0%> (+0.87%) ⬆️
Impacted Files Coverage Δ
pandas/core/indexes/timedeltas.py 90.95% <100%> (+0.68%) ⬆️
pandas/io/gbq.py 25% <0%> (-62.5%) ⬇️
pandas/io/common.py 72.86% <0%> (-19.01%) ⬇️
pandas/compat/__init__.py 58.03% <0%> (-17.8%) ⬇️
pandas/io/excel/_util.py 78.82% <0%> (-8.68%) ⬇️
pandas/compat/chainmap.py 61.9% <0%> (-4.77%) ⬇️
pandas/core/groupby/categorical.py 95.45% <0%> (-4.55%) ⬇️
pandas/core/computation/common.py 85.71% <0%> (-3.76%) ⬇️
pandas/core/dtypes/cast.py 88.16% <0%> (-3.2%) ⬇️
pandas/io/s3.py 86.36% <0%> (-3.12%) ⬇️
... and 118 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a1fee91...90d32f2. Read the comment docs.

@@ -141,7 +141,7 @@ Datetimelike
Timedelta
^^^^^^^^^

-
- bug when TimedeltaIndex intersects with another decreasing TimedeltaIndex (:issue:`17391`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Capitalize bug. Reference the method :meth:`TimedeltaIndex.intersection`.

Remove the trailing whitespace.

@@ -474,6 +474,10 @@ def intersection(self, other):
return self
if len(other) == 0:
return other

self = self.sort_values()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorting can be relatively expensive. Are we able to avoid this?

I think this would also break sort=False.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would think that we could extract the i8 values and re-use Int64Index.sort. Is that an option?

@gfyoung gfyoung added Bug Timedelta Timedelta data type labels Mar 15, 2019
@jreback
Copy link
Contributor

jreback commented Mar 20, 2019

can you merge master

@makbigc makbigc closed this May 1, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Timedelta Timedelta data type
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: TimedeltaIndex.intersection fails for decreasing TimedeltaIndex
4 participants