Skip to content

json_normalize throws AttributeError when calling with array of values #21608

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
vuminhle opened this issue Jun 23, 2018 · 6 comments
Closed
Labels
Enhancement Error Reporting Incorrect or improved errors from pandas IO JSON read_json, to_json, json_normalize Needs Discussion Requires discussion from core team before further action

Comments

@vuminhle
Copy link
Contributor

vuminhle commented Jun 23, 2018

Code Sample, a copy-pastable example if possible

xref #21605

from pandas.io.json import json_normalize

json_normalize([1, 2])

Problem description

The above code throws AttributeError:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python36\lib\site-packages\pandas\io\json\normalize.py", line 193, in json_normalize
    for x in compat.itervalues(y)] for y in data]):
  File "C:\Python36\lib\site-packages\pandas\io\json\normalize.py", line 193, in <listcomp>
    for x in compat.itervalues(y)] for y in data]):
  File "C:\Python36\lib\site-packages\pandas\compat\__init__.py", line 211, in itervalues
    return iter(obj.values(**kw))
AttributeError: 'int' object has no attribute 'values'

Expected Output

0
0 1
1 2

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.6.4.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 62 Stepping 4, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None
LOCALE: None.None

pandas: 0.23.1
pytest: 3.6.1
pip: 10.0.1
setuptools: 28.8.0
Cython: None
numpy: 1.14.2
scipy: None
pyarrow: None
xarray: None
IPython: 6.3.1
sphinx: None
patsy: None
dateutil: 2.7.2
pytz: 2018.4
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: 4.6.0
html5lib: 1.0.1
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

@WillAyd WillAyd added the IO JSON read_json, to_json, json_normalize label Jun 23, 2018
@WillAyd
Copy link
Member

WillAyd commented Jun 23, 2018

Does this throw an error when dealing with actual JSON objects instead of a list of integers?

@WillAyd WillAyd added the Needs Info Clarification about behavior needed to assess issue label Jun 23, 2018
@vuminhle
Copy link
Contributor Author

No. It does not throw if the array contains objects.
However, because an array of values is a valid JSON (https://www.json.org/), I think json_normalize should not throw an error.

@WillAyd
Copy link
Member

WillAyd commented Jun 23, 2018

I'm going to label this as an enhancement then as the documentation for json_normalize states it only works with JSON objects instead of just values.

With that said, I'm -1 on this as I think it just becomes an alternate constructor for a DataFrame. I also don't think this would be supported by read_json (correct if wrong) so it becomes kind of a nuanced corner of our JSON IO ops.

If we did want to do something like this, at the very least I think it should return a Series and not a DataFrame as AFAICT this will never be multi-dimensional (again correct if wrong).

Curious what others think

@WillAyd WillAyd added Enhancement Needs Discussion Requires discussion from core team before further action and removed Needs Info Clarification about behavior needed to assess issue labels Jun 23, 2018
@gfyoung
Copy link
Member

gfyoung commented Jun 25, 2018

I'm not particularly against having "another constructor" for a DataFrame. That being said, whether we should actually support this I'm not entirely sure.

cc @jreback

@jreback
Copy link
Contributor

jreback commented Jun 26, 2018

so we accept a dict or list of dicts for data. I don't see a reason to expand this scope. However, would take a better error message.

@jreback jreback added the Error Reporting Incorrect or improved errors from pandas label Jun 26, 2018
@vuminhle
Copy link
Contributor Author

Agree with you guys.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement Error Reporting Incorrect or improved errors from pandas IO JSON read_json, to_json, json_normalize Needs Discussion Requires discussion from core team before further action
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants