matrixQuery kernel error

[thesimonho]

I was testing the following piece of Python3 code:

df = pd.DataFrame({'idx': ['A', 'B', 'C', 'D'],
                   'quantity': [3, 4, 10, 11]})

def split_quant(df):
    quantities = ([4]*(df['quantity'].iat[0] // 4)) + [df['quantity'].iat[0] % 4]

    temp = pd.DataFrame({'idx': df['idx'].iat[0],
                         'quantity': quantities
                         }, index=range(len(quantities)))

    return temp

df_split = df[df['quantity'] > 4].groupby('idx').apply(split_quant)

output = df[df['quantity'] <= 4].append(df_split).reset_index(drop=True)

The variable inspector correctly lists 3 dataframes. df and output correctly open in new windows when theirs names are clicked. However I get an error when I click on df_split

In dev tools I see the following error:

Uncaught (in promise) Kernel error on 'matrixQuery' call!
Promise.then (async)
row.onclick @ variableinspector.js:97

Seems its a ValueError where the message is "Overlapping names between the index and columns"

When I call print(df_split) I see that the dataframe looks like this:

      idx  quantity
idx                
C   0   C         4
    1   C         4
    2   C         2
D   0   D         4
    1   D         4
    2   D         3

So the overlapping names error makes sense, but more generally, whats the plan for handling hierarchical/multi index dataframes?

[lckr]

Currently we are relying on pandas to export dataframe structures and are therefore limited by its capabilities to export to json. Sadly, there is no support for all possible multi index dataframes, e.g., the json export of dataframes with multi indices in its columns does not currently work (Related discussion).
And as you noticed pd doesn't like df_split.to_json(orient="table") either.

Nevertheless, viewing multiindexed dataframes is possible in some scenarios.
E.g. the example from the pandas documentation produces a proper gridview.

arrays = [['bar', 'bar', 'baz', 'baz', 'foo', 'foo', 'qux', 'qux'],
          ['one', 'two', 'one', 'two', 'one', 'two', 'one', 'two']]
tuples = list(zip(*arrays))
index = pd.MultiIndex.from_tuples(tuples, names=['first', 'second'])
df = pd.DataFrame(np.random.randn(8, 4), index=arrays)

I have to further investigate the limits on what is supported and what doesn't work.

[thesimonho]

would a workaround to this be to just capture str(df) and then use that to build a text representation of the multiindex df and just display that?

it might be a bit jarring from a UX perspective to see one type of df representation for normal dataframes, and another for multiindex ones. although im not sure if thats as jarring as clicking on a df and seeing nothing happen...