Skip to content

DataFrame.describe excludes top and freq for empty DataFrame #26397

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
TomAugspurger opened this issue May 14, 2019 · 4 comments · Fixed by #26474
Closed

DataFrame.describe excludes top and freq for empty DataFrame #26397

TomAugspurger opened this issue May 14, 2019 · 4 comments · Fixed by #26474
Labels
Numeric Operations Arithmetic, Comparison, and Logical operations
Milestone

Comments

@TomAugspurger
Copy link
Contributor

TomAugspurger commented May 14, 2019

Right now, the output shape and dtype of DataFrame.describe for object columns depends on whether the DataFrame is empty.

Code Sample, a copy-pastable example if possible

In [75]: x = pd.DataFrame({"A": ['a', np.nan, np.nan]})

In [76]: x.describe()
Out[76]:
        A
count   1
unique  1
top     a
freq    1

In [77]: x.iloc[:0].describe()
Out[77]:
        A
count   0
unique  0

Problem description

This leads to instability in the output dtypes and shape.

Would people prefer that we use np.NaN or None for the top and freq in this case? I believe there's no ambiguity, since we drop missing values before computing.

While the output consistency would be nice, it's not clear to me what's actually best for users here.

@WillAyd
Copy link
Member

WillAyd commented May 14, 2019

Makes sense to me

@enisnazif
Copy link
Contributor

Seems reasonable to me - would probably also need a note in the docs explaining what None means in this context. Mind if I attempt a fix?

@TomAugspurger TomAugspurger added the Numeric Operations Arithmetic, Comparison, and Logical operations label May 20, 2019
@TomAugspurger
Copy link
Contributor Author

TomAugspurger commented May 20, 2019 via email

@enisnazif
Copy link
Contributor

Thanks - opened a PR at #26474

@jreback jreback added this to the 0.25.0 milestone May 24, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Numeric Operations Arithmetic, Comparison, and Logical operations
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants