BUG: #10645 in using MultiIndex.__contains__

[scari]

This PR fix a BUG #10645
@sinhrks would you review my PR?

[jreback]

this just papers over things and does not solve the issue

[jreback]

needs tests

[scari]

Thanks for mentioning about missing test. Just added. :)

[scari]

I revisit this issue and Yes, you're right @jreback I did further investigation and it seems a odd behavior of Cython. so this PR doesn't solve the issue.

I'd like to finish this issue. Do you have any idea with this? Anything would be appreciated.

[jreback]

so the indexes have different behavior if > 1M elements on lookups. so that is a different path that is taken.

[scari]

This isn't Cython issue.

https://github.com/pydata/pandas/blob/master/pandas/index.pyx#L141-L148
If > 1M elements on lookups, IndexEngine.get_loc() do _bin_search() and try to get_value_at() the loc.
If the loc is equal to len(values) (like this error case), it looks ok to raise IndexError in util.get_value_at() because it means the loc is out of bounds.

So we should handle this IndexError somewhere.
IMHO, get_loc() itself has a 'key' parameter and if it fail key lookup, it's okay to raise KeyError here.

How's your thought? @jreback

[scari]

This PR also fixes #10692
@jreback Any comments would be appreciated.

[sinhrks]

@ scari Gr8! Could you add test cases and release note for #10692? Then, pls squash to a single commit.

[jreback]

this is not correct
you are fixing a symptom not the problem
I don't think the indexer is setup correctly

[scari]

Okay. I'll take another look at this.

[kawochen]

@scari I'll submit a PR if you don't have time to come back to this.

[jreback]

replaced by #11049

[scari]

@kawochen Thanks!