Skip to content

json_normalize throws TypeError with array of values and record_prefix #21536

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
vuminhle opened this issue Jun 19, 2018 · 2 comments · Fixed by #21540
Closed

json_normalize throws TypeError with array of values and record_prefix #21536

vuminhle opened this issue Jun 19, 2018 · 2 comments · Fixed by #21540
Labels
Bug IO Data IO issues that don't fit into a more specific label IO JSON read_json, to_json, json_normalize
Milestone

Comments

@vuminhle
Copy link
Contributor

Code Sample, a copy-pastable example if possible

from pandas.io.json import json_normalize

df = json_normalize({'A': [1, 2]}, 'A', record_prefix='Prefix.')
print(df)

Problem description

The above code throws a TypeError:

Traceback (most recent call last):
  File "c:\Users\levu\Desktop\tmp\json_normalize\main.py", line 3, in <module>
    df = json_normalize({'A': [1, 2]}, 'A', record_prefix='Prefix.')
  File "C:\Python36\lib\site-packages\pandas\io\json\normalize.py", line 262, in json_normalize
    result.rename(columns=lambda x: record_prefix + x, inplace=True)
  File "C:\Python36\lib\site-packages\pandas\util\_decorators.py", line 187, in wrapper
    return func(*args, **kwargs)
  File "C:\Python36\lib\site-packages\pandas\core\frame.py", line 3781, in rename
    return super(DataFrame, self).rename(**kwargs)
  File "C:\Python36\lib\site-packages\pandas\core\generic.py", line 973, in rename
    level=level)
  File "C:\Python36\lib\site-packages\pandas\core\internals.py", line 3340, in rename_axis
    obj.set_axis(axis, _transform_index(self.axes[axis], mapper, level))
  File "C:\Python36\lib\site-packages\pandas\core\internals.py", line 5298, in _transform_index
    items = [func(x) for x in index]
  File "C:\Python36\lib\site-packages\pandas\core\internals.py", line 5298, in <listcomp>
    items = [func(x) for x in index]
  File "C:\Python36\lib\site-packages\pandas\io\json\normalize.py", line 262, in <lambda>
    result.rename(columns=lambda x: record_prefix + x, inplace=True)
TypeError: must be str, not int

I think line 262 in normalize.py should be:

result.rename(columns=lambda x: "{p}{c}".format(p=record_prefix,c=x), inplace=True)

because x can be integer.

Expected Output

Prefix.0
0 1
1 2

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.6.4.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 62 Stepping 4, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None
LOCALE: None.None

pandas: 0.23.1
pytest: 3.6.1
pip: 10.0.1
setuptools: 28.8.0
Cython: None
numpy: 1.14.2
scipy: None
pyarrow: None
xarray: None
IPython: 6.3.1
sphinx: None
patsy: None
dateutil: 2.7.2
pytz: 2018.4
blosc: None
bottleneck: None
tables: None
numexpr: None
feather: None
matplotlib: None
openpyxl: None
xlrd: None
xlwt: None
xlsxwriter: None
lxml: None
bs4: 4.6.0
html5lib: 1.0.1
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None

@gfyoung gfyoung added IO Data IO issues that don't fit into a more specific label IO JSON read_json, to_json, json_normalize Bug labels Jun 19, 2018
@gfyoung
Copy link
Member

gfyoung commented Jun 19, 2018

That indeed looks suspicious! PR to patch is welcome!

cc @WillAyd

@gfyoung gfyoung added this to the 0.23.2 milestone Jun 19, 2018
@gfyoung
Copy link
Member

gfyoung commented Jun 19, 2018

@vuminhle : Marking this for 0.23.2, as @vuminhle has already identified a potential fix, which we can easily check and patch if this actually works if no one picks this up.

vuminhle added a commit to vuminhle/pandas that referenced this issue Jun 19, 2018
vuminhle added a commit to vuminhle/pandas that referenced this issue Jun 19, 2018
vuminhle added a commit to vuminhle/pandas that referenced this issue Jun 19, 2018
vuminhle added a commit to vuminhle/pandas that referenced this issue Jun 19, 2018
jorisvandenbossche pushed a commit that referenced this issue Jun 29, 2018
jorisvandenbossche pushed a commit that referenced this issue Jul 2, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug IO Data IO issues that don't fit into a more specific label IO JSON read_json, to_json, json_normalize
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants