Skip to content

Nullable integer documentation examples are incorrect #29599

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
toniprada opened this issue Nov 13, 2019 · 2 comments · Fixed by #29619
Closed

Nullable integer documentation examples are incorrect #29599

toniprada opened this issue Nov 13, 2019 · 2 comments · Fixed by #29619
Labels
Milestone

Comments

@toniprada
Copy link

toniprada commented Nov 13, 2019

Code Sample

import pandas as pd
import numpy as np

print(pd.array([23842938553820651, np.nan], dtype=pd.Int64Dtype())) # from the docs
print()
print(pd.array([23842938553820651, pd.NaT], dtype=pd.Int64Dtype()))

Problem description

Examples in the documentation of Nullable integers don't work as expected https://pandas.pydata.org/pandas-docs/stable/user_guide/integer_na.html

The integers are converted to float before casting to the new Nullable integer, so they lose precision anyway (which is the the most usual reason for using them, as stated in the docs). Using the native pd.NaT solves the problem.

Expected Output

<IntegerArray>
[23842938553820651, NaN]
Length: 2, dtype: Int64

<IntegerArray>
[23842938553820651, NaN]
Length: 2, dtype: Int64

Output of code sample

<IntegerArray>
[23842938553820652, NaN]
Length: 2, dtype: Int64

<IntegerArray>
[23842938553820651, NaN]
Length: 2, dtype: Int64

Output of pd.show_versions()

commit           : None

pandas           : 0.25.3
numpy            : 1.17.4
pytz             : 2018.9
dateutil         : 2.8.0
pip              : 19.0.3
setuptools       : 41.0.1
Cython           : None
pytest           : None
hypothesis       : None
sphinx           : None
blosc            : None
feather          : None
xlsxwriter       : None
lxml.etree       : None
html5lib         : None
pymysql          : None
psycopg2         : None
jinja2           : 2.10.1
IPython          : 7.5.0
pandas_datareader: None
bs4              : None
bottleneck       : None
fastparquet      : None
gcsfs            : None
lxml.etree       : None
matplotlib       : 3.1.1
numexpr          : None
odfpy            : None
openpyxl         : None
pandas_gbq       : None
pyarrow          : 0.14.1
pytables         : None
s3fs             : None
scipy            : None
sqlalchemy       : 1.3.2
tables           : None
xarray           : None
xlrd             : None
xlwt             : None
xlsxwriter       : None
@toniprada toniprada changed the title Nullable integer documentation are incorrect Nullable integer documentation examples are incorrect Nov 13, 2019
@TomAugspurger
Copy link
Contributor

The np.nan is causing in intermediate floating-point ndarray to be created. And that value is to large to be represented exactly as a float.

Perhaps we should update the docs to use None instead of np.nan there. That works correctly

In [3]: print(pd.array([23842938553820651, None], dtype=pd.Int64Dtype()))
<IntegerArray>
[23842938553820651, NaN]
Length: 2, dtype: Int64

OTOH, that's going to create an intermediate object-dtype ndarray, which will be slow for large arrays.

@lucassa3
Copy link
Contributor

@TomAugspurger would this simple edit be enough to make the docs coherent with expected functionality?

@jreback jreback added this to the 1.0 milestone Nov 14, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants