SubArray bounds checking (or not): bug or feature?

[timholy]

SubArrays currently do not check bounds upon construction:

julia> A = randn(4,8)
4x8 Float64 Array:
 -0.661697  0.150336   -0.979774   0.350783   1.97253   -0.255409   -0.464847   -0.785631
 -0.389022  0.0357723  -2.96341    0.285687   0.446166  -0.0916746  -0.64864     0.476115
 -0.972825  0.037842    0.456445   0.752897  -0.733183   0.320432   -1.00636     2.2454  
  0.964869  0.467261   -0.564566  -2.38089    1.70834    0.0517955  -0.0813409   0.302723

julia> As = sub(A, 1:3, 0:3)
3x4 SubArray of 4x8 Float64 Array:
 #undef  -0.661697  0.150336   -0.979774
 #undef  -0.389022  0.0357723  -2.96341 
 #undef  -0.972825  0.037842    0.456445

julia> As[1,1]
ERROR: BoundsError()
 in getindex at subarray.jl:172

julia> As[1,2]
-0.6616974979624464

This lack of bounds-checking turns out to be incredibly useful for certain applications, and I'm starting to write more and more code that takes advantage of it. However, I also recognize that this may be unintended behavior.

Hence, I'd like to request that we make a decision: is this a bug or a feature? I see three paths:

• It's a feature; keep it as it is. There are a couple of lines of code that will need fixing. Specifically, in a few places A.indexes[i][j] needs to become first(A.indexes[i]) + (j-1)*step(A.indexes[i]), which is mathematically equivalent except for bounds-checking.
• It should be an optional feature (introduce an extra parameter into the type definition)
• It's a bug. We will check bounds on construction, but then we also need an AbstractStridedArray type from which "private" variants can descend.

Left to my own devices I would choose the first. But I'll be happy to implement whichever of these achieves some reasonable facsimile of consensus.

[timholy]

I should also clarify that the issue of bounds-checking upon construction is separate from the issue of bounds-checking on getindex and setindex!. I think we definitely need bounds-checking on the latter, and currently that's not implemented either. Using the same array as in my demo above:

julia> As[4,2]
0.9648692808934075

That does not depend on the fact that As was created with out-of-bounds indexes:

julia> As = sub(A, 1:3, 1:3)
3x3 SubArray of 4x8 Float64 Array:
 -0.661697  0.150336   -0.979774
 -0.389022  0.0357723  -2.96341 
 -0.972825  0.037842    0.456445

julia> As[4,2]
0.4672608417658797

In other words, getindex currently behaves exactly as I implemented for get in #3814. While I still think that's the correct behavior for get, it's definitely not the right behavior for getindex.

So, to summarize: my recommendation is for no bounds-checking on construction, but have bounds-checking on getindex/setindex!. Whatever we choose to do, there's stuff that needs fixing.

[timholy]

Ah, I've realized that we can't currently add getindex/setindex! bounds-checking to SubArray without a major performance penalty.
#3503 (comment)

[JeffBezanson]

The idea of SubArrays that overlap regions outside an array is kind of neat.

[stevengj]

I see this as a feature, not a bug. It is still "safe" in the sense of throwing an exception of the bounds of the original array are exceeded.

[kmsquire]

Can you give an example of where this behavior is useful?

[timholy]

Here's an example from what I'm doing right now (this isn't quite accurate, but explaining the precise analysis would take a lot of time...): I have some movies, and I want to track movement of objects during the course of the movie. The movie is a big array, and I track an object by drawing a box around it (a SubArray). What happens when the object moves near the edge of the frame, so that my centered box would stick out beyond the edge? If the SubArray barfs on construction, then I find that I needed a fair bit more code to handle special cases. If it doesn't barf on construction, then I'm finding that it reduces the number of places in the code that I have to explicitly think about boundaries.

So, there's a real-world application showing where it's useful. That said, a person could rationally argue that such facilities don't belong in SubArray. If that's how this discussion works out, I can easily go off in my own direction with my own custom types (it may happen anyway, of course). There are pluses and minuses, for both me and Julia, either way.

[kmsquire]

Thanks for the example. How would/do you handle the #undefs in the sub-array? Display on screen seems to work fine, but any access to them throws a BoundsError.

[stevengj]

Yes, in general this is useful for defining a SubArray for an interior region that is updated, and the surrounding array elements to be "ghost cells" or "halo cells", which is a common technique to implement boundary conditions in PDEs.

[timholy]

Yep, our get command for arrays does a version of this.

The other nice property about not being picky at construction time is that shifts become commutative. Here's a 1d example:

A = rand(15)
rng = 1:4
As = sub(A, rng)

Now:

As = sub(As, rng+1)
As = sub(As, rng-1)

yields no shift, as does

As = sub(As, rng-1)
As = sub(As, rng+1)

However, this second one would give an error if we were picky at construction time.

[timholy]

OK, it sounds like this is a feature. I'll add a brief note pointing to this issue in the subarray.jl code, and fix the sub(A::SubArray, ...) constructor to be consistent with this behavior.

[ViralBShah]

I wonder if the code in examples/plife.jl can benefit from this. I do think that we should make this an intentional and well-defined feature, for the same reasons as already discussed.

[timholy]

I do think that we should make this an intentional and well-defined feature, for the same reasons as already discussed.

It's now intentional and well-defined in the sense that I've set up tests that will fail if this behavior changes.