Skip to content

Conversation

@attack68
Copy link
Contributor

@attack68 attack68 commented May 23, 2021

Adds an option: styler.max.elements with a default value of 2**18 = 262144.

>>> pd.options.styler.max.elements = 2**18
>>> df = pd.DataFrame(np.random.rand(10000,100))
>>> %timeit df.style.render()
4.11 s ± 30.3 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

Even though this is a 'large' dataframe it is reduced to the max elements and still renders in reasonable time (4s plus about 5s for a browser to render the date).

If the number of data elements exceeds the max then the right side, or bottom, or both of the DataFrame will be trimmed (starting with the largest axis and recursively reducing), leaving '...' filler indicators.

All reduction takes place before index or data looping so it avoids any crashes.

Screen Shot 2021-05-23 at 20 44 15

@simonjayhawkins simonjayhawkins added Bug Enhancement Styler conditional formatting using DataFrame.style labels May 24, 2021
Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not but on yet another option as we already have display.max_columns but i suppose ok

@jreback jreback added this to the 1.3 milestone Jun 3, 2021
elements in outer levels within groups).
styler.sparse.columns True "Sparsify" MultiIndex display for columns
in Styler output.
styler.max.elements 262144 Maximum number of datapoints that Styler will render
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

styler.display.max_elements ? i think something like that is more instructive no?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

or
styler.render.max_elements (as all things are display based)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed, even maybe styler.max_elements, but changed to yours

@jreback jreback merged commit 874d9a7 into pandas-dev:master Jun 4, 2021
@jreback
Copy link
Contributor

jreback commented Jun 4, 2021

thanks @attack68

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Bug Enhancement Styler conditional formatting using DataFrame.style

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ENH: add dataframe render trimming in Styler to avoid browser crash

3 participants