Use with CUDAnative

[lcw]

Thanks for the cool package! This package seems to work as expected on the CPU and now I am trying to push things to the GPU.

Following the example in the README, I tried to use a StructArray in a simple CUDAnative copy kernel, see

using CUDAnative, CuArrays, StaticArrays, StructArrays                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      

c = [SHermitianCompact(@SVector(rand(3))) for i=1:5]
d = StructArray(c, unwrap = t -> t <: Union{SVector,Tuple,SHermitianCompact})

dd = replace_storage(CuArray, d)
de = similar(dd)

@show typeof(dd)

function kernel!(dest, src)
  i = (blockIdx().x-1)*blockDim().x + threadIdx().x
  if i <= length(dest)
    dest[i] = src[i]
  end
  return nothing
end

threads = 1024
blocks = cld(length(dd),threads)

@cuda threads=threads blocks=blocks kernel!(de, dd)

This resulted in the following error message

❯ julia --project=.
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.2.0 (2019-08-20)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> include("try.jl")
typeof(dd) = StructArray{SHermitianCompact{2,Float64,3},1,NamedTuple{(:lowertriangle,),Tuple{StructArray{SArray{Tuple{3},Float64,1,3},1,NamedTuple{(:data,),Tuple{StructArray{Tuple{Float64,Float64,Float64},1,Tuple{CuArray{Float64,1},CuArray{Float64,1},CuArray{Float64,1}},Int64}}},Int64}}},Int64}
ERROR: LoadError: GPU compilation of kernel!(StructArray{SHermitianCompact{2,Float64,3},1,NamedTuple{(:lowertriangle,),Tuple{StructArray{SArray{Tuple{3},Float64,1,3},1,NamedTuple{(:data,),Tuple{StructArray{Tuple{Float64,Float64,Float64},1,Tuple{CuArray{Float64,1},CuArray{Float64,1},CuArray{Float64,1}},Int64}}},Int64}}},Int64}, StructArray{SHermitianCompact{2,Float64,3},1,NamedTuple{(:lowertriangle,),Tuple{StructArray{SArray{Tuple{3},Float64,1,3},1,NamedTuple{(:data,),Tuple{StructArray{Tuple{Float64,Float64,Float64},1,Tuple{CuArray{Float64,1},CuArray{Float64,1},CuArray{Float64,1}},Int64}}},Int64}}},Int64}) failed
KernelError: passing and using non-bitstype argument

Argument 2 to your kernel function is of type StructArray{SHermitianCompact{2,Float64,3},1,NamedTuple{(:lowertriangle,),Tuple{StructArray{SArray{Tuple{3},Float64,1,3},1,NamedTuple{(:data,),Tuple{StructArray{Tuple{Float64,Float64,Float64},1,Tuple{CuArray{Float64,1},CuArray{Float64,1},CuArray{Float64,1}},Int64}}},Int64}}},Int64}.
That type is not isbits, and such arguments are only allowed when they are unused by the kernel.  .fieldarrays is of type NamedTuple{(:lowertriangle,),Tuple{StructArray{SArray{Tuple{3},Float64,1,3},1,NamedTuple{(:data,),Tuple{StructArray{Tuple{Float64,Float64,Float64},1,Tuple{CuArray{Float64,1},CuArray{Float64,1},CuArray{Float64,1}},Int64}}},Int64}}} which is not isbits.
    .lowertriangle is of type StructArray{SArray{Tuple{3},Float64,1,3},1,NamedTuple{(:data,),Tuple{StructArray{Tuple{Float64,Float64,Float64},1,Tuple{CuArray{Float64,1},CuArray{Float64,1},CuArray{Float64,1}},Int64}}},Int64} which is not isbits.
      .fieldarrays is of type NamedTuple{(:data,),Tuple{StructArray{Tuple{Float64,Float64,Float64},1,Tuple{CuArray{Float64,1},CuArray{Float64,1},CuArray{Float64,1}},Int64}}} which is not isbits.
        .data is of type StructArray{Tuple{Float64,Float64,Float64},1,Tuple{CuArray{Float64,1},CuArray{Float64,1},CuArray{Float64,1}},Int64} which is not isbits.
          .fieldarrays is of type Tuple{CuArray{Float64,1},CuArray{Float64,1},CuArray{Float64,1}} which is not isbits.
            .1 is of type CuArray{Float64,1} which is not isbits.
              .buf is of type CUDAdrv.Mem.Buffer which is not isbits.
            .2 is of type CuArray{Float64,1} which is not isbits.
              .buf is of type CUDAdrv.Mem.Buffer which is not isbits.
            .3 is of type CuArray{Float64,1} which is not isbits.
              .buf is of type CUDAdrv.Mem.Buffer which is not isbits.


Stacktrace:
 [1] check_invocation(::CUDAnative.CompilerJob, ::LLVM.Function) at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/compiler/validation.jl:70
 [2] macro expansion at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/compiler/driver.jl:187 [inlined]
 [3] macro expansion at /home/lucas/.julia/packages/TimerOutputs/7zSea/src/TimerOutput.jl:216 [inlined]
 [4] #codegen#130(::Bool, ::Bool, ::Bool, ::Bool, ::Bool, ::typeof(CUDAnative.codegen), ::Symbol, ::CUDAnative.CompilerJob) at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/compiler/driver.jl:186
 [5] #codegen at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/compiler/driver.jl:0 [inlined]
 [6] #compile#129(::Bool, ::Bool, ::Bool, ::Bool, ::Bool, ::typeof(CUDAnative.compile), ::Symbol, ::CUDAnative.CompilerJob) at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/compiler/driver.jl:47
 [7] #compile#128 at ./none:0 [inlined]
 [8] #compile at ./none:0 [inlined] (repeats 2 times)
 [9] macro expansion at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/execution.jl:389 [inlined]
 [10] #cufunction#170(::Nothing, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(cufunction), ::typeof(kernel!), ::Type{Tuple{StructArray{SHermitianCompact{2,Float64,3},1,NamedTuple{(:lowertriangle,),Tuple{StructArray{SArray{Tuple{3},Float64,1,3},1,NamedTuple{(:data,),Tuple{StructArray{Tuple{Float64,Float64,Float64},1,Tuple{CuArray{Float64,1},CuArray{Float64,1},CuArray{Float64,1}},Int64}}},Int64}}},Int64},StructArray{SHermitianCompact{2,Float64,3},1,NamedTuple{(:lowertriangle,),Tuple{StructArray{SArray{Tuple{3},Float64,1,3},1,NamedTuple{(:data,),Tuple{StructArray{Tuple{Float64,Float64,Float64},1,
Tuple{CuArray{Float64,1},CuArray{Float64,1},CuArray{Float64,1}},Int64}}},Int64}}},Int64}}}) at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/execution.jl:357
 [11] cufunction(::Function, ::Type) at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/execution.jl:357
 [12] top-level scope at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/execution.jl:174
 [13] top-level scope at gcutils.jl:87
 [14] top-level scope at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/execution.jl:171
 [15] include at ./boot.jl:328 [inlined]
 [16] include_relative(::Module, ::String) at ./loading.jl:1094
 [17] include(::Module, ::String) at ./Base.jl:31
 [18] include(::String) at ./client.jl:431
 [19] top-level scope at REPL[1]:1
in expression starting at /home/lucas/research/code/Heptapus.jl/examples/structarrays/try.jl:24

Is there a way to modify my example to be able to use the StructArray in CUDAnative? Thanks for any pointers you may have.

[lcw]

If I add

CUDAnative.cudaconvert(s::StructArray) = replace_storage(cudaconvert, s)

I get further

❯ julia --project=.
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.2.0 (2019-08-20)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> include("try.jl")
typeof(dd) = StructArray{SHermitianCompact{2,Float64,3},1,NamedTuple{(:lowertriangle,),Tuple{StructArray{SArray{Tuple{3},Float64,1,3},1,NamedTuple{(:data,),Tuple{StructArray{Tuple{Float64,Float64,Float64},1,Tuple{CuArray{Float64,1},CuArray{Float64,1},CuArray{Float64,1}},Int64}}},Int64}}},Int64}
ERROR: LoadError: InvalidIRError: compiling kernel!(StructArray{SHermitianCompact{2,Float64,3},1,NamedTuple{(:lowertriangle,),Tuple{StructArray{SArray{Tuple{3},Float64,1,3},1,NamedTuple{(:data,),Tuple{StructArray{Tuple{Float64,Float64,Float64},1,Tuple{CuDeviceArray{Float64,1,CUDAnative.AS.Global},CuDeviceArray{Float64,1,CUDAnative.AS.Global},CuDeviceArray{Float64,1,CUDAnative.AS.Global}},Int64}}},Int64}}},Int64}, StructArray{SHermitianCompact{2,Float64,3},1,NamedTuple{(:lowertriangle,),Tuple{StructArray{SArray{Tuple{3},Float64,1,3},1,NamedTuple{(:data,),Tuple{StructArray{Tuple{Float64,Float64,Float64},1,Tuple{CuDeviceArray{Float64,1,CUDAnative.AS.Global},CuDeviceArray{Float64,1,CUDAnative.AS.Global},CuDeviceArray{Float64,1,CUDAnative.AS.Global}},Int64}}},Int64}}},Int64}) resulted in invalid LLVM IR
Reason: unsupported dynamic function invocation (call to ntuple(f::F, n::Integer) where F in Base at ntuple.jl:18)
Stacktrace:
 [1] get_ith at /home/lucas/.julia/packages/StructArrays/RkrVr/src/structarray.jl:138
 [2] get_ith at /home/lucas/.julia/packages/StructArrays/RkrVr/src/structarray.jl:136
 [3] getindex at /home/lucas/.julia/packages/StructArrays/RkrVr/src/structarray.jl:153
 [4] kernel! at /home/lucas/research/code/Heptapus.jl/examples/structarrays/try.jl:18
Reason: unsupported call to the Julia runtime (call to jl_f__apply)
Stacktrace:
 [1] getindex at /home/lucas/.julia/packages/StructArrays/RkrVr/src/structarray.jl:153
 [2] kernel! at /home/lucas/research/code/Heptapus.jl/examples/structarrays/try.jl:18
Reason: unsupported dynamic function invocation (call to setproperty!)
Stacktrace:
 [1] kernel! at /home/lucas/research/code/Heptapus.jl/examples/structarrays/try.jl:18
Reason: unsupported dynamic function invocation (call to ntuple(f::F, n::Integer) where F in Base at ntuple.jl:18)
Stacktrace:
 [1] get_ith at /home/lucas/.julia/packages/StructArrays/RkrVr/src/structarray.jl:138
 [2] get_ith at /home/lucas/.julia/packages/StructArrays/RkrVr/src/structarray.jl:136
 [3] getindex at /home/lucas/.julia/packages/StructArrays/RkrVr/src/structarray.jl:153
 [4] kernel! at /home/lucas/research/code/Heptapus.jl/examples/structarrays/try.jl:19
Reason: unsupported call to the Julia runtime (call to jl_f__apply)
Stacktrace:
 [1] getindex at /home/lucas/.julia/packages/StructArrays/RkrVr/src/structarray.jl:153
 [2] kernel! at /home/lucas/research/code/Heptapus.jl/examples/structarrays/try.jl:19
Reason: unsupported dynamic function invocation (call to setproperty!)
Stacktrace:
 [1] kernel! at /home/lucas/research/code/Heptapus.jl/examples/structarrays/try.jl:19
Stacktrace:
 [1] check_ir(::CUDAnative.CompilerJob, ::LLVM.Module) at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/compiler/validation.jl:114
 [2] macro expansion at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/compiler/driver.jl:188 [inlined]
 [3] macro expansion at /home/lucas/.julia/packages/TimerOutputs/7zSea/src/TimerOutput.jl:216 [inlined]
 [4] #codegen#130(::Bool, ::Bool, ::Bool, ::Bool, ::Bool, ::typeof(CUDAnative.codegen), ::Symbol, ::CUDAnative.CompilerJob) at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/compiler/driver.jl:186
 [5] #codegen at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/compiler/driver.jl:0 [inlined]
 [6] #compile#129(::Bool, ::Bool, ::Bool, ::Bool, ::Bool, ::typeof(CUDAnative.compile), ::Symbol, ::CUDAnative.CompilerJob) at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/compiler/driver.jl:47
 [7] #compile#128 at ./none:0 [inlined]
 [8] #compile at ./none:0 [inlined] (repeats 2 times)
 [9] macro expansion at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/execution.jl:389 [inlined]
 [10] #cufunction#170(::Nothing, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(cufunction), ::typeof(kernel!), ::Type{Tuple{StructArray{SHermitianCompact{2,Float64,3},1,NamedTuple{(:lowertriangle,),Tuple{StructArray{SArray{Tuple{3},Float64,1,3},1,NamedTuple{(:data,),Tuple{StructArray{Tuple{Float64,Float64,Float64},1,Tuple{CuDeviceArray{Float64,1,CUDAnative.AS.Global},CuDeviceArray{Float64,1,CUDAnative.AS.Global},CuDeviceArray{Float64,1,CUDAnative.AS.Global}},Int64}}},Int64}}},Int64},StructArray{SHermitianCompact{2,Float64,3},1,NamedTuple{(:lowertriangle,),Tuple{StructArray{SArray{Tuple{3},Float64,1,3},1,NamedTuple{(:data,),Tuple{StructArray{Tuple{Float64,Float64,Float64},1,Tuple{CuDeviceArray{Float64,1,CUDAnative.AS.Global},CuDeviceArray{Float64,1,CUDAnative.AS.Global},CuDeviceArray{Float64,1,CUDAnative.AS.Global}},Int64}}},Int64}}},Int64}}}) at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/execution.jl:357
 [11] cufunction(::Function, ::Type) at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/execution.jl:357
 [12] top-level scope at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/execution.jl:174
 [13] top-level scope at gcutils.jl:87
 [14] top-level scope at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/execution.jl:171
 [15] include at ./boot.jl:328 [inlined]
 [16] include_relative(::Module, ::String) at ./loading.jl:1094
 [17] include(::Module, ::String) at ./Base.jl:31
 [18] include(::String) at ./client.jl:431
 [19] top-level scope at REPL[1]:1
in expression starting at /home/lucas/research/code/Heptapus.jl/examples/structarrays/try.jl:27

[lcw]

I am guessing that get_ith will need to be replaced with a generated function.

[lcw]

Okay, with lcw@efb75eb I made get_ith a generated function which fixes the previous issue. Now I see that foreachfield is a dynamic function but I don't understand why.

❯ julia --project=.
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.2.0 (2019-08-20)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> include("try.jl")
typeof(dd) = StructArray{SHermitianCompact{2,Float64,3},1,NamedTuple{(:lowertriangle,),Tuple{StructArray{SArray{Tuple{3},Float64,1,3},1,NamedTuple{(:data,),Tuple{StructArray{Tuple{Float64,Float64,Float64},1,Tuple{CuArray{Float64,1},CuArray{Float64,1},CuArray{Float64,1}},Int64}}},Int64}}},Int64}
ERROR: LoadError: InvalidIRError: compiling kernel!(StructArray{SHermitianCompact{2,Float64,3},1,NamedTuple{(:lowertriangle,),Tuple{StructArray{SArray{Tuple{3},Float64,1,3},1,NamedTuple{(:data,),Tuple{StructArray{Tuple{Float64,Float64,Float64},1,Tuple{CuDeviceArray{Float64,1,CUDAnative.AS.Global},CuDeviceArray{Float64,1,CUDAnative.AS.Global},CuDeviceArray{Float64,1,CUDAnative.AS.Global}},Int64}}},Int64}}},Int64}, StructArray{SHermitianCompact{2,Float64,3},1,NamedTuple{(:lowertriangle,),Tuple{StructArray{SArray{Tuple{3},Float64,1,3},1,NamedTuple{(:data,),Tuple{StructArray{Tuple{Float64,Float64,Float64},1,Tuple{CuDeviceArray{Float64,1,CUDAnative.AS.Global},CuDeviceArray{Float64,1,CUDAnative.AS.Global},CuDeviceArray{Float64,1,CUDAnative.AS.Global}},Int64}}},Int64}}},Int64}) resulted in invalid LLVM IR
Reason: unsupported dynamic function invocation (call to foreachfield(f, x::T, xs...) where T in StructArrays at /home/lucas/julia/dev/StructArrays/src/utils.jl:45)
Stacktrace:
 [1] setindex! at /home/lucas/julia/dev/StructArrays/src/structarray.jl:168
 [2] #61 at /home/lucas/julia/dev/StructArrays/src/structarray.jl:168
 [3] macro expansion at /home/lucas/julia/dev/StructArrays/src/utils.jl:42
 [4] foreachfield at /home/lucas/julia/dev/StructArrays/src/utils.jl:42
 [5] foreachfield at /home/lucas/julia/dev/StructArrays/src/utils.jl:45
 [6] setindex! at /home/lucas/julia/dev/StructArrays/src/structarray.jl:168
 [7] kernel! at /home/lucas/research/code/Heptapus.jl/examples/structarrays/try.jl:18
Stacktrace:
 [1] check_ir(::CUDAnative.CompilerJob, ::LLVM.Module) at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/compiler/validation.jl:114
 [2] macro expansion at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/compiler/driver.jl:188 [inlined]
 [3] macro expansion at /home/lucas/.julia/packages/TimerOutputs/7zSea/src/TimerOutput.jl:216 [inlined]
 [4] #codegen#130(::Bool, ::Bool, ::Bool, ::Bool, ::Bool, ::typeof(CUDAnative.codegen), ::Symbol, ::CUDAnative.CompilerJob) at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/compiler/driver.jl:186
 [5] #codegen at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/compiler/driver.jl:0 [inlined]
 [6] #compile#129(::Bool, ::Bool, ::Bool, ::Bool, ::Bool, ::typeof(CUDAnative.compile), ::Symbol, ::CUDAnative.CompilerJob) at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/compiler/driver.jl:47
 [7] #compile#128 at ./none:0 [inlined]
 [8] #compile at ./none:0 [inlined] (repeats 2 times)
 [9] macro expansion at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/execution.jl:389 [inlined]
 [10] #cufunction#170(::Nothing, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(cufunction), ::typeof(kernel!), ::Type{Tuple{StructArray{SHermitianCompact{2,Float64,3},1,NamedTuple{(:lowertriangle,),Tuple{StructArray{SArray{Tuple{3},Float64,1,3},1,NamedTuple{(:data,),Tuple{StructArray{Tuple{Float64,Float64,Float64},1,Tuple{CuDeviceArray{Float64,1,CUDAnative.AS.Global},CuDeviceArray{Float64,1,CUDAnative.AS.Global},CuDeviceArray{Float64,1,CUDAnative.AS.Global}},Int64}}},Int64}}},Int64},StructArray{SHermitianCompact{2,Float64,3},1,NamedTuple{(:lowertriangle,),Tuple{StructArray{SArray{Tuple{3},Float64,1,3},1,NamedTuple{(:data,),Tuple{StructArray{Tuple{Float64,Float64,Float64},1,Tuple{CuDeviceArray{Float64,1,CUDAnative.AS.Global},CuDeviceArray{Float64,1,CUDAnative.AS.Global},CuDeviceArray{Float64,1,CUDAnative.AS.Global}},Int64}}},Int64}}},Int64}}}) at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/execution.jl:357
 [11] cufunction(::Function, ::Type) at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/execution.jl:357
 [12] top-level scope at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/execution.jl:174
 [13] top-level scope at gcutils.jl:87
 [14] top-level scope at /home/lucas/.julia/packages/CUDAnative/UWBIY/src/execution.jl:171
 [15] include at ./boot.jl:328 [inlined]
 [16] include_relative(::Module, ::String) at ./loading.jl:1094
 [17] include(::Module, ::String) at ./Base.jl:31
 [18] include(::String) at ./client.jl:431
 [19] top-level scope at REPL[1]:1
in expression starting at /home/lucas/research/code/Heptapus.jl/examples/structarrays/try.jl:26

[piever]

I don't think SVector is supported as an eltype of StructArray, maybe you could try with a regular StructArray of named tuples, say:

s = StructArray(a = rand(100), b = rand(100))

Replacing SVector with a regular Tuple in your example (say c = [Tuple(rand(3) for i=1:5]) still gives:

ERROR: GPU compilation of kernel!(StructArray{Tuple{Float64,Float64,Float64},1,Tuple{CuArray{Float64,1},CuArray{Float64,1},CuArray{Float64,1}},Int64}, StructArray{Tuple{Float64,Float64,Float64},1,Tuple{CuArray{Float64,1},CuArray{Float64,1},CuArray{Float64,1}},Int64}) failed
KernelError: recursion is currently not supported

Not sure if there is an easy solution as StructArrays can be nested so getindex needs to work recursively.

[lcw]

Thanks for taking a look! SVector seems to work well on the CPU and I have made a little more progress for CUDAnative. Basically I want to remove the recursion from foreachfield and only have it happen at compile time. If we can get it to generate

    @inbounds getproperty(dest.lowertriangle.data, 1)[i] = getproperty(src.lowertriangle.data, 1)[i]
    @inbounds getproperty(dest.lowertriangle.data, 2)[i] = getproperty(src.lowertriangle.data, 2)[i]
    @inbounds getproperty(dest.lowertriangle.data, 3)[i] = getproperty(src.lowertriangle.data, 3)[i]

for

  @inbounds dest[i] = src[i]

directly without recursion I think we would be set. To this end I have written the code https://github.com/lcw/Heptapus.jl/blob/417ec8a6bd611c877c27c90d297cef5db32c17e8/examples/structarrays/lutils.jl
which takes a staticschema and gives an array of the properties. I think with this we can create the generated function foreachfield which is not recursive.

p.s. I moved the tuples out of the type domain because it was making my head hurt. But we should be able to generate the array of properties without this. Do you have something in StructArrays that might help with this?

[piever]

Could you try and make a pull request out of your code? That way it'd be easier to see what changes this requires.

One thing to check is how the broadcast machinery is implemented for CuArrays. What makes me curious is that the standard broadcast (i.e. f.(v) with v::StructArray) works when the columns are CuArrays and I wonder how that's possible if calling getindex fails in a CUDA kernel.

[lcw]

Sure, see #87. The last commit is not super clean but it seems to work. More tested should probably be done before it is merged.

[piever]

In the end, your suggestion worked. I'm changing foreachfield to do the recursion at compile time in #141 and it fixed your example.

[lcw]

In the end, your suggestion worked. I'm changing `foreachfield` to do the recursion at compile time in #141 and it fixed your example.

Awesome! Thanks for looking into it.