`fileslice.is_fancy` fails for slices with numpy scalar integers

[pauldmccarthy]

This code:

import numpy as np
import nibabel as nib

shape = (10, 20, 30, 5)
randVoxels = np.vstack([np.random.randint(0, s, 10) for s in shape[:3]]).T

for x, y, z in randVoxels:
    nib.fileslice.canonical_slicers((x, y, z, slice(None, None, None)), shape)

Raises this error:

ValueError                                Traceback (most recent call last)
<ipython-input-10-1584b7a87e3e> in <module>()
      6
      7 for x, y, z in randVoxels:
----> 8     nib.fileslice.canonical_slicers((x, y, z, slice(None, None, None)), shape)
      9

/Users/paulmc/Projects/nibabel/nibabel/fileslice.pyc in canonical_slicers(sliceobj, shape, check_inds)
     80         sliceobj = (sliceobj,)
     81     if is_fancy(sliceobj):
---> 82         raise ValueError("Cannot handle fancy indexing")
     83     can_slicers = []
     84     n_dim = len(shape)

ValueError: Cannot handle fancy indexing

Due to this check in fileslice.py:

        if hasattr(slicer, 'dtype'):  # ndarray always fancy
            return True

At first, I thought this was a simple bug, that could be fixed either by making the check more specific (e.g. if hasattr(slicer, 'dtype') and not isinstance(slicer, Integral), or by removing the check entirely - the code which follows should take care of any non-integral slice object.

But, tracing back through the history, and looking at the test cases, I'm struggling to understand why this check was deemed necessary in the first place, and am wondering if I've missed some edge case or subtle piece of complexity. Can anybody shed some light on this?

[matthew-brett]

The check is because numpy fancy indexing gets triggered by any array input, and it's considerably more complicated to re-implement, so I'm just barfing when I see it. I keep forgetting how fancy indexing works in complex cases, but my memory is, it does act almost the same in the specific case of numpy array scalars. So, of course it would be possible to extend the possible indexing to the fancy indexing cases which are simple to implement, it's just I hadn't got around to doing that.

[pauldmccarthy]

Right, I think I get the problem - it's explained at test_fileslice.py:73, right? You're trying to avoid creation of copies...

Fair enough - I'll stick with pre-casting all my scalar indices to ints.

Thanks!

[matthew-brett]

Ah - no - fileslicing has to return a copy, because it's creating a new array from the data on disk. It's just that fancy indexing gets extremely complicated to implement - have a read of https://docs.scipy.org/doc/numpy/reference/arrays.indexing.html#advanced-indexing . In the simple one-dimensional index array case, it's not too bad, it wouldn't be too hard to implement - integer slicing being the simplest. It's just I haven't got round to working out all the rules and reimplementing in the general case.

[pauldmccarthy]

Aah, of course fileslice would need to create a copy, thanks for the clarification. I have been using canonical_slicers directly for other reasons, so had forgotten that, under normal circumstances, it would only be called on file accesses.