Description
-
I have checked that this issue has not already been reported.
-
I have confirmed this bug exists on the latest version of pandas.
-
(optional) I have confirmed this bug exists on the master branch of pandas.
Currently DataFrame.apply
works asymmetrically.
>>> df = pd.DataFrame([[1,2], [3,4]], index=["a", "b"], columns=["x", "y"])
>>> df.apply(lambda s: ["one", "two"], axis=0)
x y
a one one
b two two
>>> df.apply(lambda s: pd.Series(["one", "two"], index=["c", "d"]), axis=0)
x y
c one one
d two two
>>> df.apply(lambda s: ["one", "two", "three"], axis=0)
x y
0 one one
1 two two
2 three three
Along axis=0, the native index is maintained, if the shapes match and it is not overwritten by the function result object. When shapes don't match the created index is the default RangeIndex.
>>> df.apply(lambda s: ["one", "two"], axis=1, result_type="expand")
0 1
a one two
b one two
>>> df.apply(lambda s: pd.Series(["one", "two"], index=["c", "d"]), axis=1, result_type="expand")
c d
a one two
b one two
>>> df.apply(lambda s: ["one", "two", "three"], axis=1, result_type="expand")
0 1 2
a one two three
b one two three
Along axis=1 (with result_type=expand), the native columns are overwritten by a default RangeIndex (even if it has same shape) unless overwritten by the function result object index.
Expected Output
Would anyone object to a PR that looks to unify the two cases and apply the columns index where the shapes were the same?
Currently I use the workaround:
df.apply(func, axis=1, result_type="expand") -> df.T.apply(func, axis=0).T
Bug 1
Documentation states that for result_type
arguments - These only act when axis=1 (columns):
.
This is not true since:
>>> df = pd.DataFrame([[1, 2], [3, 4]], index=["a", "b"], columns=["a", "b"])
>>> df.apply(lambda s: [1,1,1], axis=0, result_type=None) # or "expand"
a b
0 1 1
1 1 1
2 1 1
>>> df.apply(lambda s: [1,1,1], axis=0, result_type="broadcast")
ValueError: cannot broadcast result
>>> df.apply(lambda s: [1,1,1], axis=0, result_type="reduce")
a [1, 1, 1]
b [1, 1, 1]
dtype: object