BUG: duplicate indexing with embedded non-orderables

[gloryfromca]

code below is scene that issue happened.

 for user_id, row in pending_data_df.iterrows():

        df = DataFrame(
            columns=[
                Names.call.same_call_counts,
                Names.call.phonebook_detail
            ],
            index=[user_id]
        )

        phoneBookPath = row[Names.app.phonebookpath]
        call_number_list = row.get(Names.call.call_number_list)

Problem description

when I wanted to get a set from a Series , it happended.

it has worked well for a long time,but suddenly it broke out. raise Traceback like this:

Traceback (most recent call last):
File "/usr/local/lib/python3.5/threading.py", line 914, in _bootstrap_inner
self.run()
File "/home/datascience/zh/suijiesuihuan/datascience/my_threads.py", line 87, in run
ret_data, result = make_decision(int(self.layerId), df_input)
File "/home/datascience/zh/suijiesuihuan/datascience/layered_decision.py", line 52, in make_decision
data_df, result_df = load_features_group_loader(features_group_loader_id, mocking)(data_df)
File "/home/datascience/zh/suijiesuihuan/datascience/dynamic_content_loader.py", line 166, in inner
return load_and_make_decision(data_df, config_new)
File "/home/datascience/zh/suijiesuihuan/datascience/dynamic_content_loader.py", line 111, in load_and_make_decision
new_feature_group_df, error_df = _load_data(pending_df, features_loader_id, features_loader, mocking)
File "/home/datascience/zh/suijiesuihuan/datascience/dynamic_content_loader.py", line 76, in _load_data
data_df, result_df = features_loader(data_df)
File "/home/datascience/zh/suijiesuihuan/dynamic_contents/feature_group_loader/feature_group_loader_0003/feature_group_loader.py", line 55, in load
call_number_list = row.get(Names.call.call_number_list)
File "/home/datascience/zh/venv/lib/python3.5/site-packages/pandas/core/generic.py", line 1633, in get
return self[key]
File "/home/datascience/zh/venv/lib/python3.5/site-packages/pandas/core/series.py", line 611, in getitem
dtype=self.dtype).finalize(self)
File "/home/datascience/zh/venv/lib/python3.5/site-packages/pandas/core/series.py", line 227, in init
"".format(data.class.name))
TypeError: 'set' type is unordered

someone knows what happened?

[nmusolino]

First, it is not clear at all that this is a bug. A bug report typically contains a short, runnable example of the problem, and a description of why the observed behavior is wrong.

I tried to reproduce this with pandas 0.19 but could not.

In [1]: import pandas

In [2]: s = pandas.Series({'a': 0, 'b': 1})

In [3]: s.get(['b', 'a'])
Out[3]:
b    1
a    0
dtype: int64

In [4]: s.get({'b', 'a'})  # note set
Out[4]:
a    0
b    1
dtype: int64

Could you try calling get with a list argument, as in this?

        call_number_list = row.get(list(Names.call.call_number_list))

[gfyoung]

@gloryfromca : Also, could you provide version information (pandas.show_versions)?

[gloryfromca]

you can try codes below to reproduce the problem:

>> from pandas import Series >> Series_contain_duplicates = Series({'1':333,'s':set([1,2,3])}) >> Series_contain_duplicates =

Series_contain_duplicates.append(Series({'1':2}))

>> Series_contain_duplicates['s']

Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/home/zhanghui/py35/lib/python3.5/site-packages/pandas/core/series.py", line 611, in __getitem__ dtype=self.dtype).__finalize__(self) File "/home/zhanghui/py35/lib/python3.5/site-packages/pandas/core/series.py", line 227, in __init__ "".format(data.__class__.__name__)) TypeError: 'set' type is unordered It's not normal behaviour, right? I'm very confused at this TypeError at first sight. my colleague may find how the problems produce: code in row608:if not self.index.is_unique: Maybe index.is_index is a wrong determination conditions ? Thank you! 2017-09-21 22:13 GMT+08:00 Nicholas Musolino <[email protected]>:

…

First, it is not clear at all that this is a bug. A bug report typically contains a short, runnable example of the problem, and a description of why the observed behavior is wrong. I tried to reproduce this with pandas 0.19 but could not. In [1]: import pandas In [2]: s = pandas.Series({'a': 0, 'b': 1}) In [3]: s.get(['b', 'a']) Out[3]: b 1 a 0 dtype: int64 In [4]: s.get({'b', 'a'}) # note set Out[4]: a 0 b 1 dtype: int64 Could you try calling get with a list argument, as in this? call_number_list = row.get(list(Names.call.call_number_list)) — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#17610 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AWW152DsbehNPwYLz-khP8MPuCd3QXQlks5skm7rgaJpZM4PezI0> .

[gloryfromca]

Both 0.19 and 0.20 has this problems. 2017-09-22 2:23 GMT+08:00 gfyoung <[email protected]>:

…

@gloryfromca <https://github.com/gloryfromca> : Also, could you provide version information? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#17610 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AWW151q9mN8LjbWT8kHSnP4-w8NA1xqEks5skqmvgaJpZM4PezI0> .

[jreback]

I guess this is a bug, though I am not sure why you would ever do this. embedding non-scalars (e.g. a set) is non-idiomatic. Using duplicate indices requires care as well.

I'll mark it, but it would require a community pull request to fix.

In [2]: s = Series({'1':333,'s':set([1,2,3])})

In [3]: s
Out[3]: 
1          333
s    {1, 2, 3}
dtype: object

In [13]: s2 = s.append(Series({'1':2}))

In [14]: s2
Out[14]: 
1          333
s    {1, 2, 3}
1            2
dtype: object

In [15]: s2[1]
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-15-c2e7db717c6c> in <module>()
----> 1 s2[1]

~/pandas/pandas/core/series.py in __getitem__(self, key)
    629                         result = self._constructor(
    630                             result, index=[key] * len(result),
--> 631                             dtype=self.dtype).__finalize__(self)
    632 
    633             return result

~/pandas/pandas/core/series.py in __init__(self, data, index, dtype, name, copy, fastpath)
    239             elif isinstance(data, (set, frozenset)):
    240                 raise TypeError("{0!r} type is unordered"
--> 241                                 "".format(data.__class__.__name__))
    242             else:
    243 

TypeError: 'set' type is unordered

[gloryfromca]

This problem is mixture of coincidence and bad practice. I am beginner of GitHub, so What does a community pull request means? Will you fix it or Should I create a pull request for it ? 2017-10-10 7:17 GMT+08:00 Jeff Reback <[email protected]>:

…

I guess this is a bug, though I am not sure why you would ever do this. embedding non-scalars (e.g. a set) is non-idiomatic. Using duplicate indices requires care as well. I'll mark it, but it would require a community pull request to fix. In [2]: s = Series({'1':333,'s':set([1,2,3])}) In [3]: s Out[3]: 1 333 s {1, 2, 3} dtype: object In [13]: s2 = s.append(Series({'1':2})) In [14]: s2 Out[14]: 1 333 s {1, 2, 3} 1 2 dtype: object In [15]: s2[1] --------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-15-c2e7db717c6c> in <module>() ----> 1 s2[1] ~/pandas/pandas/core/series.py in __getitem__(self, key) 629 result = self._constructor( 630 result, index=[key] * len(result), --> 631 dtype=self.dtype).__finalize__(self) 632 633 return result ~/pandas/pandas/core/series.py in __init__(self, data, index, dtype, name, copy, fastpath) 239 elif isinstance(data, (set, frozenset)): 240 raise TypeError("{0!r} type is unordered" --> 241 "".format(data.__class__.__name__)) 242 else: 243 TypeError: 'set' type is unordered — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#17610 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AWW154tj1jfoiGHbNe-dK2mGMRCoIF46ks5sqqlvgaJpZM4PezI0> .

[jreback]

Should I create a pull request for it ?

sure, docs are: http://pandas.pydata.org/pandas-docs/stable/contributing.html

[gloryfromca]

I fixed the bug, and ran asv benchmarks.Then It raised information as below: SOME BENCHMARKS HAVE CHANGED SIGNIFICANTLY. Is it OK? or the way I fixed the bug is unproper? PS:attachment is photo of benchmarks detail information. 2017-10-10 11:19 GMT+08:00 Jeff Reback <[email protected]>:

…

Should I create a pull request for it ? sure, docs are: http://pandas.pydata.org/pandas-docs/stable/ contributing.html — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#17610 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AWW1562eKtoronGip1xvvx3hfa-HXacfks5squIzgaJpZM4PezI0> .