numpy.AxisError: axis 1 is out of bounds for array of dimension 1

[yarikoptic]

originally appeared in

• numpy.AxisError: axis 1 is out of bounds for array of dimension 1 heudiconv#670

with traceback

  File "/src/heudiconv/heudiconv/convert.py", line 216, in prep_conversion
    seqinfo = group_dicoms_into_seqinfos(
  File "/src/heudiconv/heudiconv/dicoms.py", line 283, in group_dicoms_into_seqinfos
    mwinfo = validate_dicom(filename, dcmfilter)
  File "/src/heudiconv/heudiconv/dicoms.py", line 125, in validate_dicom
    del mw.series_signature[sig]
  File "/opt/miniconda-py39_4.12.0/lib/python3.9/site-packages/nibabel/onetime.py", line 156, in __get__
    val = self.getter(obj)
  File "/opt/miniconda-py39_4.12.0/lib/python3.9/site-packages/nibabel/nicom/dicomwrappers.py", line 625, in series_signature
    signature['image_shape'] = (self.image_shape, eq)
  File "/opt/miniconda-py39_4.12.0/lib/python3.9/site-packages/nibabel/onetime.py", line 156, in __get__
    val = self.getter(obj)
  File "/opt/miniconda-py39_4.12.0/lib/python3.9/site-packages/nibabel/nicom/dicomwrappers.py", line 545, in image_shape
    frame_indices = np.delete(frame_indices, stackid_dim_idx, axis=1)
  File "<__array_function__ internals>", line 200, in delete
  File "/opt/miniconda-py39_4.12.0/lib/python3.9/site-packages/numpy/lib/function_base.py", line 5136, in delete
    axis = normalize_axis_index(axis, ndim)
numpy.AxisError: axis 1 is out of bounds for array of dimension 1

so it feels that may be hardcoded axis=1 should be "reconsidered" there?

[effigies]

Looks like this was added in #439. Unfortunately, I don't know anything about this code. Do you have a proposal?

Possibly @moloney or @ZviBaratz would have some insight here.

[effigies]

cc also @matthew-brett who is at least more versed in DICOM than I.

[yarikoptic]

Sample tracew DICOMs were collected and available now at http://datasets.datalad.org/?dir=/dicoms/dartmouth-phantoms/Siemens-DWITrace-20230803 . But upon quick try with heudiconv - I did not reproduce this issue, so must be something special about original DICOMs.

[yarikoptic]

@effigies what could/should we do about this issue ? unfortunately people keep running into it... could we may be just safeguard the del and ignore the error??? or if we follow my summary at that point - we should look into avoiding removal of some images.

[effigies]

It would be nice if we could actually get our hands on erroring data, because I don't have any context for what this looks like under expected and erroring cases, so it's hard to know what the right thing to do is.

nibabel/nibabel/nicom/dicomwrappers.py

Lines 542 to 546 in 5d884bd

	# remove the stack id axis if present
	if stackid_tag in dim_seq:
	stackid_dim_idx = dim_seq.index(stackid_tag)
	frame_indices = np.delete(frame_indices, stackid_dim_idx, axis=1)
	dim_seq.pop(stackid_dim_idx)

Maybe it would make sense to use:

if stackid_tag in dim_seq:
    stackid_dim_idx = dim_seq.index(stackid_tag)
    dim_seq.pop(stackid_dim_idx)
    if len(frame_indices.shape) > 1:
        frame_indices = np.delete(frame_indices, stackid_dim_idx, axis=1)
    else:
        msg = (
            f'Cannot remove dimension {stackid_dim_idx} from frame_indices (shape: {frame_indices.shape}).\n'
            'Please report this at https://github.com/nipy/nibabel/issues and reference issue gh-1245'
        )
        warnings.warn(msg)

That way we can get a sense of what things look like when this is hit. I'm not sure if we have example data that hits this, but we can look for that once we have something to compare it to.

[yarikoptic]

re sample -- I did find one. I believe it is In/20_DWI_dir80_AP within https://github.com/neurolabusc/dcm_qa_xa30, so https://github.com/neurolabusc/dcm_qa_xa30/raw/main/In/20_DWI_dir80_AP/0001_1.3.12.2.1107.5.2.43.67093.2022071112140611403312307.dcm

[effigies]

Okay, at least in that case frame_indices == array([], dtype=float64). So we could check for frame_indices.size == 0 and stackid_tag in dim_seq?

Do we know if this is a bug to work around, or a particular interpretation of the standard that occurs in well-defined circumstances?

[effigies]

It looks like the bug was actually introduced in #795. That was aimed at Philips DICOMs, but causing errors on Siemens XA30 DICOMs.

@mgxd Do you remember much about the details of that one? The XA30 image has 39 ISOTROPIC frames, which are getting dropped by:

nibabel/nibabel/nicom/dicomwrappers.py

Lines 509 to 516 in 5d884bd

	if hasattr(first_frame, 'get') and first_frame.get([0x18, 0x9117]):
	# DWI image may include derived isotropic, ADC or trace volume
	try:
	self.frames = pydicom.Sequence(
	frame
	for frame in self.frames
	if frame.MRDiffusionSequence[0].DiffusionDirectionality != 'ISOTROPIC'
	)

[yarikoptic]

Should we just skip doing that on non-Philips DICOMs and be done?

[effigies]

I guess. Sorry, the details of this one won't stay in my head for more than about 20 minutes after I stop looking at it. I have a branch where I pulled in Chris' QA repository for testing. I can write the test if one of you can suggest a patch?

[mitchellxh]

I know it's low priority, but if this could patched in the next release it would be greatly appreciated. The Siemens Prisma OS upgrades are changing to this standard.

[effigies]

I think I have a fix. Please give it a test:

pip install git+https://github.com/effigies/nibabel.git@fix/dicom-missing-frame-indices

[mitchellxh]

Thank you, I can confirm this fix solves the issue.