ENH: cross join in merge() and join()

[RomanPekar]

related here: #7203

If I haven't missed something, there's no simple way to make cartesian product method in pandas. For example, see http://stackoverflow.com/questions/19684765/pandas-extending-the-dataframe-by-adding-another-levels/19684948#19684948.
To do cross join, one have to add some keys to DataFrames, like this:

>>> dfA = pd.DataFrame(['a','b','c'],columns=['A'])
>>> dfB = pd.DataFrame(range(3),columns=['B'])
>>> dfA['key'] = 1
>>> dfB['key'] = 1
>>> pd.merge(dfB, dfA, on='key').ix[:, ('A','B')]
   A  B
0  a  0
1  b  0
2  c  0
3  a  1
4  b  1
5  c  1
6  a  2
7  b  2
8  c  2

How about adding possibility to pass how='cross' into merge() and join() methods?

[jtratner]

Why did you close this?

[RomanPekar]

Oops my bad it's totally by accident, hit the button somehow :(

[jreback]

related #3835 , there is a method: from pandas.tools.util import cartesian_product

how is this different from an outer join? (which does this and just drops the na groups)

[RomanPekar]

@jreback could you provide an example how to do outer join if there's no column on which to join? May be it's possible to do with apply somehow?

[jreback]

A bit hacky

In [28]: import pandas.tools.util as tools

In [29]: DataFrame(index=MultiIndex.from_arrays(
                  tools.cartesian_product([dfA['A'].tolist(),dfB['B'].tolist()]),
                    names=['A','B'])).reset_index()
Out[29]: 
   A  B
0  a  0
1  a  1
2  a  2
3  b  0
4  b  1
5  b  2
6  c  0
7  c  1
8  c  2

[RomanPekar]

@jreback yes, good one! But what if there're more than one column in either dataframe, or in both?

[jreback]

not sure....can you create input and output frames?

[RomanPekar]

>>> dfA = pd.DataFrame({'A':list('ab'), 'B':range(2,0,-1)})
>>> dfB = pd.DataFrame({'C':range(2), 'D':range(4, 6)})
>>> dfA
   A  B
0  a  2
1  b  1
>>> dfB
   C  D
0  0  4
1  1  5
>>> dfA['key'] = 1
>>> dfB['key'] = 1
>>> df = pd.merge(dfA, dfB, on='key')
>>> del df['key']
>>> df
   A  B  C  D
0  a  2  0  4
1  a  2  1  5
2  b  1  0  4
3  b  1  1  5

here's sql fiddle example of cross join

[dsm054]

FWIW, this is something I've often wanted too, and I think is fundamental enough an operation to deserve a better way.

[CarstVaartjes]

I use the same method as @RomanPekar regularly too (cartesian products are handy for basket analysis as they create a normalized version of a scarce matrix). Main thing I guess is that you could improve performance inside Pandas if you know you don't have to look at an index to reference back and forth but instead can multiply the dataframe for any possible combination? in regular python I would expect something like this:

list_1 = [['a', 1], ['b', 2], ['c', 3]]
list_2 = [['x', 11], ['y', 12], ['z', 13]]

list_result = []
for sub_list_1 in list_1:
    for sub_list_2 in list_2:
        list_result.append(sub_list_1 + sub_list_2)

When it comes notation I myself would like a "how='cartesian'" where you do not specify left_on and right_on...