BUG: Styler.apply consistently manages Series return objects aligning labels.

[attack68]

• closes Row based style.apply doesn't modify background colors #13657

This issue is due to the asymmetric application of DataFrame.apply (#42005).
The bug is that when using Styler.apply the treatment of Series return objects is applied for axis=0 and overwritten for axis=1.

df = pd.DataFrame([[1,2],[3,4]], index=["X", "Y"], columns=["X", "Y"])
s = df.style

def series_ret(s):
    return pd.Series(["color:red;", "color:blue;"], index=["Y", "X"])  # note X-Y order

Command	Current Behaviour	New Behaviour
s.apply(series_ret, axis=0)
s.apply(series_ret, axis=1)

Also the application has been expanded to allow (for axis=0 | 1) Series return objects that are not the same size, or (for axis=None) DataFrame return objects that are not the same size, if they contain only relevant index labels. This allows for more flexible user defined functions. Potentially it also increases performance by not having to loop over labels that no longer exist.

def series_ret(s):
    return pd.Series(["color:red;"], index=["Y"])  # note only returning 1 element

def array_ret(s):
    return ["color:red;"]

def ser_no_idx_ret(s):
    return pd.Series(["color:red;", "color:blue;"]) # note returning a Series of right shape with no index

Command	Current Behaviour	New Behaviour
s.apply(series_ret, axis=0)	ValueError: Function returned the wrong shape.
s.apply(series_ret, axis=1)	ValueError: Length mismatch: Expected axis has 1 elements, new values have 2 elements
s.apply(array_ret, axis=0)	ValueError: Function returned the wrong shape.	ValueError: Function created invalid index labels. Usually, this is the result of the function returning a Series which contains invalid labels, or returning an incorrectly shaped, list-like object which cannot be mapped to labels, possibly due to applying the function along the wrong axis.
s.apply(array_ret, axis=1)	ValueError: Length mismatch: Expected axis has 1 elements, new values have 2 elements	ValueError: Function created invalid index labels. Usually, this is the result of the function returning a Series which contains invalid labels, or returning an incorrectly shaped, list-like object which cannot be mapped to labels, possibly due to applying the function along the wrong axis.
s.apply(ser_no_idx_ret, axis=0)	KeyError: 0	ValueError: Function created invalid index labels. Usually, this is the result of the function returning a Series which contains invalid labels, or returning an incorrectly shaped, list-like object which cannot be mapped to labels, possibly due to applying the function along the wrong axis.
s.apply(ser_no_idx_ret, axis=1)		ValueError: Function created invalid index labels. Usually, this is the result of the function returning a Series which contains invalid labels, or returning an incorrectly shaped, list-like object which cannot be mapped to labels, possibly due to applying the function along the wrong axis.

Note all errors do report the shapes, I just didn't include in the table for brevity.

Edge Cases

The edge case here is that if a user has a DataFrame with a default range index (or a DataFrame with some indexes the same as those in default, e.g. 0 or 1) then even returning arrays of the wrong size may work, since DataFrame.apply assigns them to a RangeIndex.

To be consistent with the above this should raise, but it is impossible to hook into the return value of the user-defined function and determine if it is a Series (where non-matching shapes are OK) or list-like (where matching shapes should be enforced)

df = pd.DataFrame([[1,2],[3,4]])
s = df.style

Command	Current Behaviour	New Behaviour
s.apply(array_ret, axis=0)	ValueError: Function returned the wrong shape.
s.apply(array_ret, axis=1)	ValueError: Length mismatch: Expected axis has 1 elements, new values have 2 elements

ASV

Benchmarks unchanged.

[github-actions]

This pull request is stale because it has been open for thirty days with no activity. Please update or respond to this comment if you're still interested in working on this.

[jreback]

lgtm small comments

[jreback]

can you update the examples to show a Series return matching labels/index (e.g. not the same length)

[attack68]

added

[jreback]

thanks @attack68