Fix the output of df.describe on an empty categorical / object column

[enisnazif]

• closes DataFrame.describe excludes top and freq for empty DataFrame #26397
• tests added / passed
• passes git diff upstream/master -u -- "*.py" | flake8 --diff
• whatsnew entry

[pep8speaks]

Hello @enisnazif! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2019-05-31 14:05:51 UTC

[codecov]

Codecov Report

Merging #26474 into master will decrease coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master   #26474      +/-   ##
==========================================
- Coverage   91.75%   91.74%   -0.01%     
==========================================
  Files         174      174              
  Lines       50765    50767       +2     
==========================================
- Hits        46578    46576       -2     
- Misses       4187     4191       +4

Flag	Coverage Δ
#multiple	90.25% <100%> (ø)	⬆️
#single	41.72% <0%> (-0.09%)	⬇️

Impacted Files	Coverage Δ
pandas/core/generic.py	93.49% <100%> (ø)	⬆️
pandas/io/gbq.py	78.94% <0%> (-10.53%)	⬇️
pandas/core/frame.py	97.02% <0%> (-0.12%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f5cc078...10df157. Read the comment docs.

[codecov]

Codecov Report

Merging #26474 into master will decrease coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master   #26474      +/-   ##
==========================================
- Coverage   91.84%   91.84%   -0.01%     
==========================================
  Files         174      174              
  Lines       50644    50646       +2     
==========================================
- Hits        46516    46514       -2     
- Misses       4128     4132       +4

Flag	Coverage Δ
#multiple	90.37% <100%> (ø)	⬆️
#single	41.71% <0%> (-0.09%)	⬇️

Impacted Files	Coverage Δ
pandas/core/arrays/categorical.py	95.91% <100%> (ø)	⬆️
pandas/core/generic.py	93.62% <100%> (ø)	⬆️
pandas/io/gbq.py	78.94% <0%> (-10.53%)	⬇️
pandas/core/frame.py	97% <0%> (-0.12%)	⬇️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7f31865...7100855. Read the comment docs.

[TomAugspurger]

The test failure is from our job against NumPy master.

        df = pd.DataFrame({"empty_col": Categorical([])})
>       result = df.describe()

pandas/tests/frame/test_analytics.py:594: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pandas/core/generic.py:9956: in describe
    ldesc = [describe_1d(s) for _, s in data.iteritems()]
pandas/core/generic.py:9956: in <listcomp>
    ldesc = [describe_1d(s) for _, s in data.iteritems()]
pandas/core/generic.py:9939: in describe_1d
    return describe_categorical_1d(data)
pandas/core/generic.py:9900: in describe_categorical_1d
    objcounts = data.value_counts()
pandas/core/base.py:1318: in value_counts
    normalize=normalize, bins=bins, dropna=dropna)
pandas/core/algorithms.py:689: in value_counts
    result = Series(values)._values.value_counts(dropna=dropna)
pandas/core/arrays/categorical.py:1480: in value_counts
    count = bincount(obs, minlength=ncat or None)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = (array([], dtype=int8),), kwargs = {'minlength': None}
relevant_args = (array([], dtype=int8), None)

>   ???
E   DeprecationWarning: 0 should be passed as minlength instead of None; this will error in future.

I would see if Categorical.value_counts can be updated to pass count = bincount(obs, minlength=ncat or 0). If that doesn't work for older numpy's then you'll need a bit more compat code.

[enisnazif]

The test failure is from our job against NumPy master.

        df = pd.DataFrame({"empty_col": Categorical([])})
>       result = df.describe()

pandas/tests/frame/test_analytics.py:594: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pandas/core/generic.py:9956: in describe
    ldesc = [describe_1d(s) for _, s in data.iteritems()]
pandas/core/generic.py:9956: in <listcomp>
    ldesc = [describe_1d(s) for _, s in data.iteritems()]
pandas/core/generic.py:9939: in describe_1d
    return describe_categorical_1d(data)
pandas/core/generic.py:9900: in describe_categorical_1d
    objcounts = data.value_counts()
pandas/core/base.py:1318: in value_counts
    normalize=normalize, bins=bins, dropna=dropna)
pandas/core/algorithms.py:689: in value_counts
    result = Series(values)._values.value_counts(dropna=dropna)
pandas/core/arrays/categorical.py:1480: in value_counts
    count = bincount(obs, minlength=ncat or None)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = (array([], dtype=int8),), kwargs = {'minlength': None}
relevant_args = (array([], dtype=int8), None)

>   ???
E   DeprecationWarning: 0 should be passed as minlength instead of None; this will error in future.

I would see if Categorical.value_counts can be updated to pass count = bincount(obs, minlength=ncat or 0). If that doesn't work for older numpy's then you'll need a bit more compat code.

Ok, thanks - I'll take a look

[TomAugspurger]

Needs a release note for 0.25.0.

What section? Are we calling this an API breaking change, or a bugfix? cc @WillAyd @jreback

[jreback]

@enisnazif can you add a note in the api breaking section

[enisnazif]

@enisnazif can you add a note in the api breaking section

Done

[jreback]

doc comment; ping on green.

[enisnazif]

doc comment; ping on green.
@jreback checks have all passed, ok to merge?

[TomAugspurger]

Pushed a small doc fix. LGTM

[enisnazif]

i'll take a look at the failing jobs

[TomAugspurger]

@enisnazif that's fixed on master. Just merged & repushed. Ping on green.

[enisnazif]

@enisnazif that's fixed on master. Just merged & repushed. Ping on green.

@jreback @TomAugspurger hey, looks like it's green. Is it good to merge?

[jreback]

thanks @enisnazif