Name: experiment_alias, dtype: object has no member named "iteritems"

[FernandoDuarteF]

Hi all,
It seems that the upload tool is not compatible with with the latest pandas version (2.0.3). If I run it with that version (using the example data and metadata from the repository) I get the following:

Traceback (most recent call last):
  File "/users/bi/fduarte/miniconda3/envs/ena-iupload/lib/python3.11/site-packages/genshi/template/eval.py", line 301, in lookup_attr
    val = getattr(obj, key)
          ^^^^^^^^^^^^^^^^^
  File "/users/bi/fduarte/miniconda3/envs/ena-iupload/lib/python3.11/site-packages/pandas/core/generic.py", line 5989, in __getattr__
    return object.__getattribute__(self, name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'Series' object has no attribute 'iteritems'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/users/bi/fduarte/miniconda3/envs/ena-iupload/lib/python3.11/site-packages/pandas/core/indexes/base.py", line 3653, in get_loc
    return self._engine.get_loc(casted_key)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "pandas/_libs/index.pyx", line 147, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 176, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 7080, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 7088, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'iteritems'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/users/bi/fduarte/miniconda3/envs/ena-iupload/lib/python3.11/site-packages/genshi/template/eval.py", line 307, in lookup_attr
    val = obj[key]
          ~~~^^^^^
  File "/users/bi/fduarte/miniconda3/envs/ena-iupload/lib/python3.11/site-packages/pandas/core/series.py", line 1007, in __getitem__
    return self._get_value(key)
           ^^^^^^^^^^^^^^^^^^^^
  File "/users/bi/fduarte/miniconda3/envs/ena-iupload/lib/python3.11/site-packages/pandas/core/series.py", line 1116, in _get_value
    loc = self.index.get_loc(label)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/users/bi/fduarte/miniconda3/envs/ena-iupload/lib/python3.11/site-packages/pandas/core/indexes/base.py", line 3655, in get_loc
    raise KeyError(key) from err
KeyError: 'iteritems'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/users/bi/fduarte/miniconda3/envs/ena-iupload/bin/ena-upload-cli", line 10, in <module>
    sys.exit(main())
             ^^^^^^
  File "/users/bi/fduarte/miniconda3/envs/ena-iupload/lib/python3.11/site-packages/ena_upload/ena_upload.py", line 953, in main
    schema_xmls = run_construct(
                  ^^^^^^^^^^^^^^
  File "/users/bi/fduarte/miniconda3/envs/ena-iupload/lib/python3.11/site-packages/ena_upload/ena_upload.py", line 298, in run_construct
    schema_xmls[schema] = construct_xml(schema, stream, xsds[schema])
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/users/bi/fduarte/miniconda3/envs/ena-iupload/lib/python3.11/site-packages/ena_upload/ena_upload.py", line 235, in construct_xml
    xml_string = stream.render(method='xml', encoding='utf-8')
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/users/bi/fduarte/miniconda3/envs/ena-iupload/lib/python3.11/site-packages/genshi/core.py", line 184, in render
    return encode(generator, method=method, encoding=encoding, out=out)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/users/bi/fduarte/miniconda3/envs/ena-iupload/lib/python3.11/site-packages/genshi/output.py", line 59, in encode
    return _encode(''.join(list(iterator)))
                           ^^^^^^^^^^^^^^
  File "/users/bi/fduarte/miniconda3/envs/ena-iupload/lib/python3.11/site-packages/genshi/output.py", line 243, in __call__
    for kind, data, pos in stream:
  File "/users/bi/fduarte/miniconda3/envs/ena-iupload/lib/python3.11/site-packages/genshi/output.py", line 674, in __call__
    for kind, data, pos in stream:
  File "/users/bi/fduarte/miniconda3/envs/ena-iupload/lib/python3.11/site-packages/genshi/output.py", line 779, in __call__
    for kind, data, pos in chain(stream, [(None, None, None)]):
  File "/users/bi/fduarte/miniconda3/envs/ena-iupload/lib/python3.11/site-packages/genshi/output.py", line 598, in __call__
    for ev in stream:
  File "/users/bi/fduarte/miniconda3/envs/ena-iupload/lib/python3.11/site-packages/genshi/core.py", line 292, in _ensure
    for event in stream:
  File "/users/bi/fduarte/miniconda3/envs/ena-iupload/lib/python3.11/site-packages/genshi/template/base.py", line 641, in _include
    for event in stream:
  File "/users/bi/fduarte/miniconda3/envs/ena-iupload/lib/python3.11/site-packages/genshi/template/markup.py", line 326, in _match
    for event in stream:
  File "/users/bi/fduarte/miniconda3/envs/ena-iupload/lib/python3.11/site-packages/genshi/template/base.py", line 581, in _flatten
    for kind, data, pos in stream:
  File "/users/bi/fduarte/miniconda3/envs/ena-iupload/lib/python3.11/site-packages/genshi/template/directives.py", line 369, in __call__
    iterable = _eval_expr(self.expr, ctxt, vars)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/users/bi/fduarte/miniconda3/envs/ena-iupload/lib/python3.11/site-packages/genshi/template/base.py", line 290, in _eval_expr
    retval = expr.evaluate(ctxt)
             ^^^^^^^^^^^^^^^^^^^
  File "/users/bi/fduarte/miniconda3/envs/ena-iupload/lib/python3.11/site-packages/genshi/template/eval.py", line 160, in evaluate
    return eval(self.code, _globals, {'__data__': data})
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/users/bi/fduarte/miniconda3/envs/ena-iupload/lib/python3.11/site-packages/ena_upload/templates/ENA_template_runs.xml", line 14, in <Expression 'iter(run_groups.iteritems())'>
    <py:for each="alias, experiment_alias in run_groups.iteritems()">
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/users/bi/fduarte/miniconda3/envs/ena-iupload/lib/python3.11/site-packages/genshi/template/eval.py", line 309, in lookup_attr
    val = cls.undefined(key, owner=obj)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/users/bi/fduarte/miniconda3/envs/ena-iupload/lib/python3.11/site-packages/genshi/template/eval.py", line 397, in undefined
    raise UndefinedError(key, owner=owner)
genshi.template.eval.UndefinedError: alias
run_alias_1a    [experiment_alias_7a]
run_alias_3c    [experiment_alias_9c]
Name: experiment_alias, dtype: object has no member named "iteritems"

I downgraded to pandas 1.5.3 and now it seems to work fine. I used

ena-upload-cli --action add --center 'CRG' --study ena_templates/example_tables/ENA_template_studies.tsv --sample ena_templates/example_tables/ENA_template_samples.tsv --experiment ena_templates/example_tables/ENA_template_experiments.tsv --run ena_templates/example_tables/ENA_template_runs.tsv --data ena_templates/example_data/*gz --dev --secret ena_templates/.secret.yml --draft --no_data_upload

for running the tool.
Thanks,

[bgruening]

iteritems was removed in 2.0.0 by GH45321.

If you can install pandas <2.0 it should work.

I will leave this open, as we need to port the ENA-upload-cli to pandas 2.0.

[bedroesb]

@FernandoDuarteF thanks a lot for letting us now, I will work on a patch.