Internal error from a BoundsError

[EdAyers]

I was having a go at implementing an NN for representing images as a xy→rgb (eg see https://bmild.github.io/fourfeat/) in Julia using Flux. But I encountered an internal error in Julia and my understanding is that this is always a bug.

Versions:

• Julia Version 1.6.0 (2021-03-24)

[deps]
CUDA = "052768ef-5323-5732-b1bb-66c8b64840ba"
ColorTypes = "3da002f7-5984-5a60-b8a6-cbb66c0b333f"
ColorVectorSpace = "c3611d14-8923-5661-9e6a-0046d554d3a4"
FileIO = "5789e2e9-d7fb-5bc7-8068-2c6fae9b9549"
Flux = "587475ba-b771-5e3f-ad9e-33799f191a9c"
ImageDraw = "4381153b-2b60-58ae-a1ba-fd683676385f"
ImageShow = "4e3cecfd-b093-5904-9786-8bbb286a6a31"
Pluto = "c3e4b0f8-55cb-11ea-2926-15256bba5781"
Revise = "295af30f-e4ad-537b-8983-00126c2a3abe"

Reproduction code:

using ImageShow, ColorVectorSpace, ColorTypes, Flux, StaticArrays

(H,W) = (100, 200)
width = 256

col(x) = ColorTypes.RGB(x...)

model = (Chain(
		Dense(2, width, relu),
		Dense(width, width, relu),
		Dense(width, width, relu),
		Dense(width, 3, sigmoid),
	))

ι = ([SA[i, j] for i=range(0.0, 1.0, length=H), j=range(0.0, 1.0, length=W)])

imtrue(x,y) = SA[cos(x*2) ^ 2, 0, cos(x + y) ^ 2]

im3 = [imtrue(x,y) for x=range(0.0, 1.0, length=H), y=range(0.0, 1.0, length=W)]

data = [(ι, im3)]

function loss(xy, rgb)
	sum(norm.(model.(xy) .- rgb))
end

println(loss(ι, im3))

opt = Descent(0.01)
Flux.train!(loss, Flux.params(model), data, opt)

@show col.(model.(ι))

Expected output: it trains the network. I am not using cu for CUDA explicitly so my understanding is that all of the above happens on the CPU.

Output:

Internal error: encountered unexpected error in runtime:
BoundsError(a=Array{Any, (2,)}[
  Core.Compiler.VarState(typ=Zygote.Pullback{Tuple{typeof(StaticArrays._norm), StaticArrays.Size{(3,)}, StaticArrays.SArray{Tuple{3}, Float64, 1, 3}}, Any}, undef=false),
  Core.Compiler.VarState(typ=Float64, undef=false)], i=(3,))
jl_bounds_error_ints at /usr/bin/../lib/julia/libjulia-internal.so.1 (unknown line)
unknown function (ip: 0x7fdfea5605bd)
unknown function (ip: 0x7fdfea55d038)
unknown function (ip: 0x7fdfea559422)
unknown function (ip: 0x7fdfea558603)
unknown function (ip: 0x7fdfea55709a)
unknown function (ip: 0x7fdfea51effd)
unknown function (ip: 0x7fdfea51df88)
unknown function (ip: 0x7fdfea51b795)
unknown function (ip: 0x7fdfea546b0c)
unknown function (ip: 0x7fdfea54634f)
unknown function (ip: 0x7fdfea544795)
unknown function (ip: 0x7fdfea55cdb5)
unknown function (ip: 0x7fdfea559422)
unknown function (ip: 0x7fdfea558603)
unknown function (ip: 0x7fdfea55709a)
unknown function (ip: 0x7fdfea51effd)
unknown function (ip: 0x7fdfea51df88)
unknown function (ip: 0x7fdfea51b795)
unknown function (ip: 0x7fdfea546b0c)
unknown function (ip: 0x7fdfea54634f)
unknown function (ip: 0x7fdfea544795)
unknown function (ip: 0x7fdfea55cdb5)
unknown function (ip: 0x7fdfea559422)
unknown function (ip: 0x7fdfea558603)
unknown function (ip: 0x7fdfea55709a)
unknown function (ip: 0x7fdfea51effd)
unknown function (ip: 0x7fdfea51df88)
unknown function (ip: 0x7fdfea51b795)
unknown function (ip: 0x7fdfea55163b)
unknown function (ip: 0x7fdfea51880f)
unknown function (ip: 0x7fdfea27c3f0)
unknown function (ip: 0x7fdfea549347)
unknown function (ip: 0x7fdfea546a31)
unknown function (ip: 0x7fdfea54634f)
unknown function (ip: 0x7fdfea544795)
unknown function (ip: 0x7fdfea55cdb5)
unknown function (ip: 0x7fdfea559422)
unknown function (ip: 0x7fdfea558603)
unknown function (ip: 0x7fdfea55709a)
unknown function (ip: 0x7fdfea51effd)
unknown function (ip: 0x7fdfea51df88)
unknown function (ip: 0x7fdfea51b795)
unknown function (ip: 0x7fdfea549bf6)
unknown function (ip: 0x7fdfea546a31)
unknown function (ip: 0x7fdfea54634f)
unknown function (ip: 0x7fdfea544795)
unknown function (ip: 0x7fdfea55cdb5)
unknown function (ip: 0x7fdfea559422)
unknown function (ip: 0x7fdfea558603)
unknown function (ip: 0x7fdfea55709a)
unknown function (ip: 0x7fdfea51effd)
unknown function (ip: 0x7fdfea51df88)
unknown function (ip: 0x7fdfea51b795)
unknown function (ip: 0x7fdfea549bf6)
unknown function (ip: 0x7fdfea546a31)
unknown function (ip: 0x7fdfea54634f)
unknown function (ip: 0x7fdfea544795)
unknown function (ip: 0x7fdfea55de61)
unknown function (ip: 0x7fdfea559422)
unknown function (ip: 0x7fdfea558603)
unknown function (ip: 0x7fdfea55709a)
unknown function (ip: 0x7fdfea51effd)
unknown function (ip: 0x7fdfea51df88)
unknown function (ip: 0x7fdfea51b795)
unknown function (ip: 0x7fdfea55163b)
unknown function (ip: 0x7fdfea51880f)
unknown function (ip: 0x7fdfea5189b2)
unknown function (ip: 0x7fdfea540127)
unknown function (ip: 0x7fdfea549827)
unknown function (ip: 0x7fdfea546a31)
unknown function (ip: 0x7fdfea54634f)
unknown function (ip: 0x7fdfea544795)
unknown function (ip: 0x7fdfea55de61)
unknown function (ip: 0x7fdfea559422)
unknown function (ip: 0x7fdfea558603)
unknown function (ip: 0x7fdfea55709a)
unknown function (ip: 0x7fdfea51effd)
unknown function (ip: 0x7fdfea51df88)
unknown function (ip: 0x7fdfea51b795)
unknown function (ip: 0x7fdfea549bf6)
unknown function (ip: 0x7fdfea546a31)
unknown function (ip: 0x7fdfea54634f)
unknown function (ip: 0x7fdfea544795)
unknown function (ip: 0x7fdfea55cdb5)
unknown function (ip: 0x7fdfea559422)
unknown function (ip: 0x7fdfea558603)
unknown function (ip: 0x7fdfea55709a)
unknown function (ip: 0x7fdfea51effd)
unknown function (ip: 0x7fdfea51df88)
unknown function (ip: 0x7fdfea51b795)
unknown function (ip: 0x7fdfea549bf6)
unknown function (ip: 0x7fdfea546a31)
unknown function (ip: 0x7fdfea54634f)
unknown function (ip: 0x7fdfea544795)
unknown function (ip: 0x7fdfea55de61)
unknown function (ip: 0x7fdfea559422)
unknown function (ip: 0x7fdfea558603)
unknown function (ip: 0x7fdfea55709a)
unknown function (ip: 0x7fdfea51effd)
unknown function (ip: 0x7fdfea51df88)
unknown function (ip: 0x7fdfea51b795)
unknown function (ip: 0x7fdfea546b0c)
unknown function (ip: 0x7fdfea54634f)
unknown function (ip: 0x7fdfea544795)
unknown function (ip: 0x7fdfea55cdb5)
unknown function (ip: 0x7fdfea559422)
unknown function (ip: 0x7fdfea558603)
unknown function (ip: 0x7fdfea55709a)
unknown function (ip: 0x7fdfea51effd)
unknown function (ip: 0x7fdfea51df88)
unknown function (ip: 0x7fdfea51b795)
unknown function (ip: 0x7fdfea546b0c)
unknown function (ip: 0x7fdfea54634f)
unknown function (ip: 0x7fdfea544795)
unknown function (ip: 0x7fdfea55de61)
unknown function (ip: 0x7fdfea559422)
unknown function (ip: 0x7fdfea558603)
unknown function (ip: 0x7fdfea55709a)
unknown function (ip: 0x7fdfea51effd)
unknown function (ip: 0x7fdfea51df88)
unknown function (ip: 0x7fdfea51b795)
unknown function (ip: 0x7fdfea546b0c)
unknown function (ip: 0x7fdfea54634f)
unknown function (ip: 0x7fdfea544795)
unknown function (ip: 0x7fdfea55cdb5)
unknown function (ip: 0x7fdfea559422)
unknown function (ip: 0x7fdfea558603)
unknown function (ip: 0x7fdfea55709a)
unknown function (ip: 0x7fdfea51effd)
unknown function (ip: 0x7fdfea51df88)
unknown function (ip: 0x7fdfea51b795)
unknown function (ip: 0x7fdfea546b0c)
unknown function (ip: 0x7fdfea54634f)
unknown function (ip: 0x7fdfea544795)
unknown function (ip: 0x7fdfea55cdb5)
unknown function (ip: 0x7fdfea559422)
unknown function (ip: 0x7fdfea558603)
unknown function (ip: 0x7fdfea55709a)
unknown function (ip: 0x7fdfea51effd)
unknown function (ip: 0x7fdfea51df88)
unknown function (ip: 0x7fdfea51b795)
unknown function (ip: 0x7fdfea546b0c)
unknown function (ip: 0x7fdfea54634f)
unknown function (ip: 0x7fdfea544795)
unknown function (ip: 0x7fdfea55cdb5)
unknown function (ip: 0x7fdfea559422)
unknown function (ip: 0x7fdfea558603)
unknown function (ip: 0x7fdfea55709a)
unknown function (ip: 0x7fdfea51effd)
unknown function (ip: 0x7fdfea51df88)
unknown function (ip: 0x7fdfea51b795)
unknown function (ip: 0x7fdfea546b0c)
unknown function (ip: 0x7fdfea54634f)
unknown function (ip: 0x7fdfea544795)
unknown function (ip: 0x7fdfea55de61)
unknown function (ip: 0x7fdfea559422)
unknown function (ip: 0x7fdfea558603)
unknown function (ip: 0x7fdfea55709a)
unknown function (ip: 0x7fdfea51effd)
unknown function (ip: 0x7fdfea51df88)
unknown function (ip: 0x7fdfea51b795)
unknown function (ip: 0x7fdfea546b0c)
unknown function (ip: 0x7fdfea54634f)
unknown function (ip: 0x7fdfea544795)
unknown function (ip: 0x7fdfea55cdb5)
unknown function (ip: 0x7fdfea559422)
unknown function (ip: 0x7fdfea558603)
unknown function (ip: 0x7fdfea55709a)
unknown function (ip: 0x7fdfea51effd)
unknown function (ip: 0x7fdfea51df88)
unknown function (ip: 0x7fdfea51b795)
unknown function (ip: 0x7fdfea546b0c)
unknown function (ip: 0x7fdfea54634f)
unknown function (ip: 0x7fdfea544795)
unknown function (ip: 0x7fdfea55cdb5)
unknown function (ip: 0x7fdfea559422)
unknown function (ip: 0x7fdfea558603)
unknown function (ip: 0x7fdfea55709a)
unknown function (ip: 0x7fdfea51effd)
unknown function (ip: 0x7fdfea51df88)
unknown function (ip: 0x7fdfea51b795)
unknown function (ip: 0x7fdfea546b0c)
unknown function (ip: 0x7fdfea54634f)
unknown function (ip: 0x7fdfea544795)
unknown function (ip: 0x7fdfea55cdb5)
unknown function (ip: 0x7fdfea559422)
unknown function (ip: 0x7fdfea558603)
unknown function (ip: 0x7fdfea55709a)
unknown function (ip: 0x7fdfea513c35)
unknown function (ip: 0x7fdfea512cfb)
unknown function (ip: 0x7fdfea289594)
unknown function (ip: 0x7fdfea2895cf)
unknown function (ip: 0x7fdffb5bcf8d)
unknown function (ip: 0x7fdffb642b70)
unknown function (ip: 0x7fdffb5bb8df)
jl_apply_generic at /usr/bin/../lib/julia/libjulia-internal.so.1 (unknown line)
gradient at /home/edward/.julia/packages/Zygote/6HN9x/src/compiler/interface.jl:59
unknown function (ip: 0x7fdf5548d2fe)
macro expansion at /home/edward/.julia/packages/Flux/6BByF/src/optimise/train.jl:101 [inlined]
macro expansion at /home/edward/.julia/packages/Juno/n6wyj/src/progress.jl:134 [inlined]
#train!#36 at /home/edward/.julia/packages/Flux/6BByF/src/optimise/train.jl:99
train! at /home/edward/.julia/packages/Flux/6BByF/src/optimise/train.jl:97
unknown function (ip: 0x7fdf5548c381)
unknown function (ip: 0x7fdffb5d52c5)
unknown function (ip: 0x7fdffb5d4e85)
unknown function (ip: 0x7fdffb5d5a80)
unknown function (ip: 0x7fdffb5d6580)
top-level scope at /home/edward/Nerf.jl/train.jl:28
unknown function (ip: 0x7fdffb5f1777)
unknown function (ip: 0x7fdffb5f2407)
jl_toplevel_eval_in at /usr/bin/../lib/julia/libjulia-internal.so.1 (unknown line)
unknown function (ip: 0x7fdfea583877)
unknown function (ip: 0x7fdfea48090b)
unknown function (ip: 0x7fdfea480372)
unknown function (ip: 0x7fdfea21bdc9)
unknown function (ip: 0x7fdfea2207dc)
unknown function (ip: 0x7fdfea220945)
unknown function (ip: 0x7fdffb6148de)
repl_entrypoint at /usr/bin/../lib/julia/libjulia-internal.so.1 (unknown line)
main at julia (unknown line)
__libc_start_main at /usr/bin/../lib/libc.so.6 (unknown line)
_start at julia (unknown line)
ERROR: LoadError: BoundsError: attempt to access 2-element Vector{Any} at index [3]
Stacktrace:
  [1] return_type(interp::Core.Compiler.NativeInterpreter, f::Any, t::Any)
    @ Core.Compiler ./compiler/typeinfer.jl:963
  [2] return_type(f::Any, t::Any, world::UInt64)
    @ Core.Compiler ./compiler/typeinfer.jl:950
  [3] return_type(f::Any, t::Any)
    @ Core.Compiler ./compiler/typeinfer.jl:947
  [4] collect(itr::Base.Generator)
    @ Base ./array.jl:652
  [5] map(::Function, ::Matrix{typeof(∂(norm))}, ::FillArrays.Fill{Float64, 2, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}})
    @ Base ./abstractarray.jl:2383
  [6] (::Zygote.var"#1077#1084")(ȳ::Any)
    @ Zygote ~/.julia/packages/Zygote/6HN9x/src/lib/broadcast.jl:164
  [7] (::Zygote.var"#3902#back#1088")(Δ::Any)
    @ Zygote ~/.julia/packages/ZygoteRules/OjfTt/src/adjoint.jl:59
  [8] (::Zygote.var"#180#181")(Δ::Any)
    @ Zygote ~/.julia/packages/Zygote/6HN9x/src/lib/lib.jl:194
  [9] (::Zygote.var"#1689#back#182")(Δ::Any)
    @ Zygote ~/.julia/packages/ZygoteRules/OjfTt/src/adjoint.jl:59
 [10] Pullback
    @ ./broadcast.jl:1309 [inlined]
 [11] (::typeof(∂(broadcasted)))(Δ::FillArrays.Fill{Float64, 2, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}})
    @ Zygote ~/.julia/packages/Zygote/6HN9x/src/compiler/interface2.jl:0
 [12] Pullback
    @ ~/Nerf.jl/train.jl:24 [inlined]
 [13] (::typeof(∂(loss)))(Δ::Float64)
    @ Zygote ~/.julia/packages/Zygote/6HN9x/src/compiler/interface2.jl:0
 [14] (::Zygote.var"#180#181")(Δ::Any)
    @ Zygote ~/.julia/packages/Zygote/6HN9x/src/lib/lib.jl:194
 [15] (::Zygote.var"#1689#back#182")(Δ::Any)
    @ Zygote ~/.julia/packages/ZygoteRules/OjfTt/src/adjoint.jl:59
 [16] Pullback
    @ ~/.julia/packages/Flux/6BByF/src/optimise/train.jl:102 [inlined]
 [17] (::typeof(∂(λ)))(Δ::Float64)
    @ Zygote ~/.julia/packages/Zygote/6HN9x/src/compiler/interface2.jl:0
 [18] (::Zygote.var"#69#70")(Δ::Any)
    @ Zygote ~/.julia/packages/Zygote/6HN9x/src/compiler/interface.jl:252
 [19] gradient(f::Function, args::Zygote.Params)
    @ Zygote ~/.julia/packages/Zygote/6HN9x/src/compiler/interface.jl:59
 [20] macro expansion
    @ ~/.julia/packages/Flux/6BByF/src/optimise/train.jl:101 [inlined]
 [21] macro expansion
    @ ~/.julia/packages/Juno/n6wyj/src/progress.jl:134 [inlined]
 [22] train!(loss::Function, ps::Zygote.Params, data::Vector{Tuple{Matrix{SVector{2, Float64}}, Matrix{SVector{3, Float64}}}}, opt::Descent; cb::Flux.Optimise.var"#40#46")
    @ Flux.Optimise ~/.julia/packages/Flux/6BByF/src/optimise/train.jl:99
 [23] train!(loss::Function, ps::Zygote.Params, data::Vector{Tuple{Matrix{SVector{2, Float64}}, Matrix{SVector{3, Float64}}}}, opt::Descent)
    @ Flux.Optimise ~/.julia/packages/Flux/6BByF/src/optimise/train.jl:97
 [24] top-level scope
    @ ~/Nerf.jl/train.jl:28
in expression starting at /home/edward/Nerf.jl/train.jl:28

[ericphanson]

Just to say I could reproduce this on Julia 1.6.1 on MacOS, though with a slightly different looking stacktrace:

julia> Flux.train!(loss, Flux.params(model), data, opt)
Internal error: encountered unexpected error in runtime:
BoundsError(a=Array{Any, (2,)}[
  Core.Compiler.VarState(typ=Zygote.Pullback{Tuple{typeof(StaticArrays._norm), StaticArrays.Size{(3,)}, StaticArrays.SArray{Tuple{3}, Float64, 1, 3}}, Any}, undef=false),
  Core.Compiler.VarState(typ=Float64, undef=false)], i=(3,))
jl_bounds_error_ints at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
setindex! at ./essentials.jl:458 [inlined]
stupdate! at ./compiler/typelattice.jl:279
typeinf_local at ./compiler/abstractinterpretation.jl:1493
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1520
_typeinf at ./compiler/typeinfer.jl:214
typeinf at ./compiler/typeinfer.jl:209
typeinf_edge at ./compiler/typeinfer.jl:806
abstract_call_method at ./compiler/abstractinterpretation.jl:490
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:143
abstract_call at ./compiler/abstractinterpretation.jl:1054
abstract_call at ./compiler/abstractinterpretation.jl:1040
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1167
typeinf_local at ./compiler/abstractinterpretation.jl:1462
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1520
_typeinf at ./compiler/typeinfer.jl:214
typeinf at ./compiler/typeinfer.jl:209
typeinf_edge at ./compiler/typeinfer.jl:806
abstract_call_method at ./compiler/abstractinterpretation.jl:490
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:143
abstract_call at ./compiler/abstractinterpretation.jl:1054
abstract_call at ./compiler/abstractinterpretation.jl:1040
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1167
typeinf_local at ./compiler/abstractinterpretation.jl:1462
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1520
_typeinf at ./compiler/typeinfer.jl:214
typeinf at ./compiler/typeinfer.jl:209
typeinf_edge at ./compiler/typeinfer.jl:806
abstract_call_method at ./compiler/abstractinterpretation.jl:490
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:143
abstract_call_known at ./compiler/abstractinterpretation.jl:1033
abstract_call at ./compiler/abstractinterpretation.jl:1056
abstract_apply at ./compiler/abstractinterpretation.jl:738
abstract_call_known at ./compiler/abstractinterpretation.jl:952
abstract_call at ./compiler/abstractinterpretation.jl:1056
abstract_call at ./compiler/abstractinterpretation.jl:1040
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1167
typeinf_local at ./compiler/abstractinterpretation.jl:1462
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1520
_typeinf at ./compiler/typeinfer.jl:214
typeinf at ./compiler/typeinfer.jl:209
typeinf_edge at ./compiler/typeinfer.jl:806
abstract_call_method at ./compiler/abstractinterpretation.jl:490
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:143
abstract_call_known at ./compiler/abstractinterpretation.jl:1033
abstract_call at ./compiler/abstractinterpretation.jl:1056
abstract_call at ./compiler/abstractinterpretation.jl:1040
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1167
typeinf_local at ./compiler/abstractinterpretation.jl:1462
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1520
_typeinf at ./compiler/typeinfer.jl:214
typeinf at ./compiler/typeinfer.jl:209
typeinf_edge at ./compiler/typeinfer.jl:806
abstract_call_method at ./compiler/abstractinterpretation.jl:490
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:143
abstract_call_known at ./compiler/abstractinterpretation.jl:1033
abstract_call at ./compiler/abstractinterpretation.jl:1056
abstract_call at ./compiler/abstractinterpretation.jl:1040
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1167
typeinf_local at ./compiler/abstractinterpretation.jl:1447
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1520
_typeinf at ./compiler/typeinfer.jl:214
typeinf at ./compiler/typeinfer.jl:209
typeinf_edge at ./compiler/typeinfer.jl:806
abstract_call_method at ./compiler/abstractinterpretation.jl:490
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:143
abstract_call_known at ./compiler/abstractinterpretation.jl:1033
abstract_call at ./compiler/abstractinterpretation.jl:1056
jfptr_abstract_call_11621 at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/sys.dylib (unknown line)
jl_apply_generic at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
return_type_tfunc at ./compiler/tfuncs.jl:1584
abstract_call_known at ./compiler/abstractinterpretation.jl:982
abstract_call at ./compiler/abstractinterpretation.jl:1056
abstract_call at ./compiler/abstractinterpretation.jl:1040
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1167
typeinf_local at ./compiler/abstractinterpretation.jl:1447
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1520
_typeinf at ./compiler/typeinfer.jl:214
typeinf at ./compiler/typeinfer.jl:209
typeinf_edge at ./compiler/typeinfer.jl:806
abstract_call_method at ./compiler/abstractinterpretation.jl:490
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:143
abstract_call_known at ./compiler/abstractinterpretation.jl:1033
abstract_call at ./compiler/abstractinterpretation.jl:1056
abstract_call at ./compiler/abstractinterpretation.jl:1040
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1167
typeinf_local at ./compiler/abstractinterpretation.jl:1462
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1520
_typeinf at ./compiler/typeinfer.jl:214
typeinf at ./compiler/typeinfer.jl:209
typeinf_edge at ./compiler/typeinfer.jl:806
abstract_call_method at ./compiler/abstractinterpretation.jl:490
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:143
abstract_call_known at ./compiler/abstractinterpretation.jl:1033
abstract_call at ./compiler/abstractinterpretation.jl:1056
abstract_call at ./compiler/abstractinterpretation.jl:1040
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1167
typeinf_local at ./compiler/abstractinterpretation.jl:1447
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1520
_typeinf at ./compiler/typeinfer.jl:214
typeinf at ./compiler/typeinfer.jl:209
typeinf_edge at ./compiler/typeinfer.jl:806
abstract_call_method at ./compiler/abstractinterpretation.jl:490
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:143
abstract_call at ./compiler/abstractinterpretation.jl:1054
abstract_call at ./compiler/abstractinterpretation.jl:1040
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1167
typeinf_local at ./compiler/abstractinterpretation.jl:1462
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1520
_typeinf at ./compiler/typeinfer.jl:214
typeinf at ./compiler/typeinfer.jl:209
typeinf_edge at ./compiler/typeinfer.jl:806
abstract_call_method at ./compiler/abstractinterpretation.jl:490
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:143
abstract_call at ./compiler/abstractinterpretation.jl:1054
abstract_call at ./compiler/abstractinterpretation.jl:1040
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1167
typeinf_local at ./compiler/abstractinterpretation.jl:1447
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1520
_typeinf at ./compiler/typeinfer.jl:214
typeinf at ./compiler/typeinfer.jl:209
typeinf_edge at ./compiler/typeinfer.jl:806
abstract_call_method at ./compiler/abstractinterpretation.jl:490
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:143
abstract_call at ./compiler/abstractinterpretation.jl:1054
abstract_call at ./compiler/abstractinterpretation.jl:1040
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1167
typeinf_local at ./compiler/abstractinterpretation.jl:1462
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1520
_typeinf at ./compiler/typeinfer.jl:214
typeinf at ./compiler/typeinfer.jl:209
typeinf_edge at ./compiler/typeinfer.jl:806
abstract_call_method at ./compiler/abstractinterpretation.jl:490
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:143
abstract_call at ./compiler/abstractinterpretation.jl:1054
abstract_call at ./compiler/abstractinterpretation.jl:1040
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1167
typeinf_local at ./compiler/abstractinterpretation.jl:1462
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1520
_typeinf at ./compiler/typeinfer.jl:214
typeinf at ./compiler/typeinfer.jl:209
typeinf_edge at ./compiler/typeinfer.jl:806
abstract_call_method at ./compiler/abstractinterpretation.jl:490
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:143
abstract_call at ./compiler/abstractinterpretation.jl:1054
abstract_call at ./compiler/abstractinterpretation.jl:1040
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1167
typeinf_local at ./compiler/abstractinterpretation.jl:1462
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1520
_typeinf at ./compiler/typeinfer.jl:214
typeinf at ./compiler/typeinfer.jl:209
typeinf_edge at ./compiler/typeinfer.jl:806
abstract_call_method at ./compiler/abstractinterpretation.jl:490
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:143
abstract_call at ./compiler/abstractinterpretation.jl:1054
abstract_call at ./compiler/abstractinterpretation.jl:1040
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1167
typeinf_local at ./compiler/abstractinterpretation.jl:1447
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1520
_typeinf at ./compiler/typeinfer.jl:214
typeinf at ./compiler/typeinfer.jl:209
typeinf_edge at ./compiler/typeinfer.jl:806
abstract_call_method at ./compiler/abstractinterpretation.jl:490
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:143
abstract_call at ./compiler/abstractinterpretation.jl:1054
abstract_call at ./compiler/abstractinterpretation.jl:1040
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1167
typeinf_local at ./compiler/abstractinterpretation.jl:1462
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1520
_typeinf at ./compiler/typeinfer.jl:214
typeinf at ./compiler/typeinfer.jl:209
typeinf_edge at ./compiler/typeinfer.jl:806
abstract_call_method at ./compiler/abstractinterpretation.jl:490
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:143
abstract_call at ./compiler/abstractinterpretation.jl:1054
abstract_call at ./compiler/abstractinterpretation.jl:1040
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1167
typeinf_local at ./compiler/abstractinterpretation.jl:1462
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1520
_typeinf at ./compiler/typeinfer.jl:214
typeinf at ./compiler/typeinfer.jl:209
typeinf_edge at ./compiler/typeinfer.jl:806
abstract_call_method at ./compiler/abstractinterpretation.jl:490
abstract_call_gf_by_type at ./compiler/abstractinterpretation.jl:143
abstract_call at ./compiler/abstractinterpretation.jl:1054
abstract_call at ./compiler/abstractinterpretation.jl:1040
abstract_eval_statement at ./compiler/abstractinterpretation.jl:1167
typeinf_local at ./compiler/abstractinterpretation.jl:1462
typeinf_nocycle at ./compiler/abstractinterpretation.jl:1520
_typeinf at ./compiler/typeinfer.jl:214
typeinf at ./compiler/typeinfer.jl:209
typeinf_ext at ./compiler/typeinfer.jl:892
typeinf_ext_toplevel at ./compiler/typeinfer.jl:925
typeinf_ext_toplevel at ./compiler/typeinfer.jl:921
jfptr_typeinf_ext_toplevel_12207 at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/sys.dylib (unknown line)
jl_apply_generic at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_type_infer at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_generate_fptr at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_compile_method_internal at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_apply_generic at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
gradient at /Users/eph/.julia/packages/Zygote/6HN9x/src/compiler/interface.jl:59
unknown function (ip: 0x16748c594)
macro expansion at /Users/eph/.julia/packages/Flux/6BByF/src/optimise/train.jl:101 [inlined]
macro expansion at /Users/eph/.julia/packages/Juno/n6wyj/src/progress.jl:134 [inlined]
#train!#36 at /Users/eph/.julia/packages/Flux/6BByF/src/optimise/train.jl:99
train! at /Users/eph/.julia/packages/Flux/6BByF/src/optimise/train.jl:97
unknown function (ip: 0x16748b50e)
jl_apply_generic at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
do_call at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
eval_body at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_interpret_toplevel_thunk at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
top-level scope at REPL[24]:1
jl_toplevel_eval_flex at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_toplevel_eval_flex at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_toplevel_eval_flex at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
eval_body at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
eval_body at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_interpret_toplevel_thunk at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
top-level scope at /Users/eph/.julia/packages/CUDA/Px7QU/src/initialization.jl:81
jl_toplevel_eval_flex at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_toplevel_eval_in at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
eval at ./boot.jl:360 [inlined]
eval_user_input at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:139
repl_backend_loop at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:200
start_repl_backend at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:185
#run_repl#42 at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:317
run_repl at /Users/julia/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.6/REPL/src/REPL.jl:305
jl_apply_generic at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
#874 at ./client.jl:387
jfptr_YY.874_25971 at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/sys.dylib (unknown line)
jl_apply_generic at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
jl_f__call_latest at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
#invokelatest#2 at ./essentials.jl:708 [inlined]
invokelatest at ./essentials.jl:706 [inlined]
run_main_repl at ./client.jl:372
exec_options at ./client.jl:302
_start at ./client.jl:485
jfptr__start_32022 at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/sys.dylib (unknown line)
jl_apply_generic at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
true_main at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
repl_entrypoint at /Applications/Julia-1.6.app/Contents/Resources/julia/lib/julia/libjulia-internal.1.6.dylib (unknown line)
ERROR: BoundsError: attempt to access 2-element Vector{Any} at index [3]
Stacktrace:
  [1] setindex!
    @ ./essentials.jl:458 [inlined]
  [2] stupdate!(state::Nothing, changes::Core.Compiler.StateUpdate)
    @ Core.Compiler ./compiler/typelattice.jl:279
  [3] typeinf_local(interp::Core.Compiler.NativeInterpreter, frame::Core.Compiler.InferenceState)
    @ Core.Compiler ./compiler/abstractinterpretation.jl:1493
  [4] typeinf_nocycle(interp::Core.Compiler.NativeInterpreter, frame::Core.Compiler.InferenceState)
    @ Core.Compiler ./compiler/abstractinterpretation.jl:1520
  [5] _typeinf(interp::Core.Compiler.NativeInterpreter, frame::Core.Compiler.InferenceState)
    @ Core.Compiler ./compiler/typeinfer.jl:214
  [6] typeinf(interp::Core.Compiler.NativeInterpreter, frame::Core.Compiler.InferenceState)
    @ Core.Compiler ./compiler/typeinfer.jl:209
  [7] typeinf_edge(interp::Core.Compiler.NativeInterpreter, method::Method, atypes::Any, sparams::Core.SimpleVector, caller::Core.Compiler.InferenceState)
    @ Core.Compiler ./compiler/typeinfer.jl:806
  [8] abstract_call_method(interp::Core.Compiler.NativeInterpreter, method::Method, sig::Any, sparams::Core.SimpleVector, hardlimit::Bool, sv::Core.Compiler.InferenceState)
    @ Core.Compiler ./compiler/abstractinterpretation.jl:490
  [9] abstract_call_gf_by_type(interp::Core.Compiler.NativeInterpreter, f::Any, argtypes::Vector{Any}, atype::Any, sv::Core.Compiler.InferenceState, max_methods::Int64)
    @ Core.Compiler ./compiler/abstractinterpretation.jl:143
 [10] abstract_call(interp::Core.Compiler.NativeInterpreter, fargs::Vector{Any}, argtypes::Vector{Any}, sv::Core.Compiler.InferenceState, max_methods::Int64)
    @ Core.Compiler ./compiler/abstractinterpretation.jl:1054
 [11] abstract_call(interp::Core.Compiler.NativeInterpreter, fargs::Vector{Any}, argtypes::Vector{Any}, sv::Core.Compiler.InferenceState)
    @ Core.Compiler ./compiler/abstractinterpretation.jl:1040
 [12] abstract_eval_statement(interp::Core.Compiler.NativeInterpreter, e::Any, vtypes::Vector{Any}, sv::Core.Compiler.InferenceState)
    @ Core.Compiler ./compiler/abstractinterpretation.jl:1167
 [13] typeinf_local(interp::Core.Compiler.NativeInterpreter, frame::Core.Compiler.InferenceState)
    @ Core.Compiler ./compiler/abstractinterpretation.jl:1462
--- the last 10 lines are repeated 1 more time ---
 [24] typeinf_nocycle(interp::Core.Compiler.NativeInterpreter, frame::Core.Compiler.InferenceState)
    @ Core.Compiler ./compiler/abstractinterpretation.jl:1520
 [25] _typeinf(interp::Core.Compiler.NativeInterpreter, frame::Core.Compiler.InferenceState)
    @ Core.Compiler ./compiler/typeinfer.jl:214
 [26] typeinf(interp::Core.Compiler.NativeInterpreter, frame::Core.Compiler.InferenceState)
    @ Core.Compiler ./compiler/typeinfer.jl:209
 [27] typeinf_edge(interp::Core.Compiler.NativeInterpreter, method::Method, atypes::Any, sparams::Core.SimpleVector, caller::Core.Compiler.InferenceState)
    @ Core.Compiler ./compiler/typeinfer.jl:806
 [28] abstract_call_method(interp::Core.Compiler.NativeInterpreter, method::Method, sig::Any, sparams::Core.SimpleVector, hardlimit::Bool, sv::Core.Compiler.InferenceState)
    @ Core.Compiler ./compiler/abstractinterpretation.jl:490
 [29] abstract_call_gf_by_type(interp::Core.Compiler.NativeInterpreter, f::Any, argtypes::Vector{Any}, atype::Any, sv::Core.Compiler.InferenceState, max_methods::Int64)
    @ Core.Compiler ./compiler/abstractinterpretation.jl:143
 [30] abstract_call_known(interp::Core.Compiler.NativeInterpreter, f::Any, fargs::Nothing, argtypes::Vector{Any}, sv::Core.Compiler.InferenceState, max_methods::Int64)
    @ Core.Compiler ./compiler/abstractinterpretation.jl:1033
 [31] abstract_call(interp::Core.Compiler.NativeInterpreter, fargs::Nothing, argtypes::Vector{Any}, sv::Core.Compiler.InferenceState, max_methods::Int64)
    @ Core.Compiler ./compiler/abstractinterpretation.jl:1056
 [32] abstract_apply(interp::Core.Compiler.NativeInterpreter, itft::Any, aft::Any, aargtypes::Vector{Any}, sv::Core.Compiler.InferenceState, max_methods::Int64)
    @ Core.Compiler ./compiler/abstractinterpretation.jl:738
 [33] abstract_call_known(interp::Core.Compiler.NativeInterpreter, f::Any, fargs::Vector{Any}, argtypes::Vector{Any}, sv::Core.Compiler.InferenceState, max_methods::Int64)
    @ Core.Compiler ./compiler/abstractinterpretation.jl:952
 [34] abstract_call(interp::Core.Compiler.NativeInterpreter, fargs::Vector{Any}, argtypes::Vector{Any}, sv::Core.Compiler.InferenceState, max_methods::Int64)
    @ Core.Compiler ./compiler/abstractinterpretation.jl:1056
 [35] abstract_call(interp::Core.Compiler.NativeInterpreter, fargs::Vector{Any}, argtypes::Vector{Any}, sv::Core.Compiler.InferenceState)
    @ Core.Compiler ./compiler/abstractinterpretation.jl:1040
 [36] abstract_eval_statement(interp::Core.Compiler.NativeInterpreter, e::Any, vtypes::Vector{Any}, sv::Core.Compiler.InferenceState)
    @ Core.Compiler ./compiler/abstractinterpretation.jl:1167
 [37] typeinf_local(interp::Core.Compiler.NativeInterpreter, frame::Core.Compiler.InferenceState)
    @ Core.Compiler ./compiler/abstractinterpretation.jl:1462
 [38] typeinf_nocycle(interp::Core.Compiler.NativeInterpreter, frame::Core.Compiler.InferenceState)
    @ Core.Compiler ./compiler/abstractinterpretation.jl:1520
 [39] _typeinf(interp::Core.Compiler.NativeInterpreter, frame::Core.Compiler.InferenceState)
    @ Core.Compiler ./compiler/typeinfer.jl:214
 [40] typeinf(interp::Core.Compiler.NativeInterpreter, frame::Core.Compiler.InferenceState)
    @ Core.Compiler ./compiler/typeinfer.jl:209
 [41] typeinf_edge(interp::Core.Compiler.NativeInterpreter, method::Method, atypes::Any, sparams::Core.SimpleVector, caller::Core.Compiler.InferenceState)
    @ Core.Compiler ./compiler/typeinfer.jl:806
 [42] abstract_call_method(interp::Core.Compiler.NativeInterpreter, method::Method, sig::Any, sparams::Core.SimpleVector, hardlimit::Bool, sv::Core.Compiler.InferenceState)
    @ Core.Compiler ./compiler/abstractinterpretation.jl:490
 [43] abstract_call_gf_by_type(interp::Core.Compiler.NativeInterpreter, f::Any, argtypes::Vector{Any}, atype::Any, sv::Core.Compiler.InferenceState, max_methods::Int64)
    @ Core.Compiler ./compiler/abstractinterpretation.jl:143
 [44] abstract_call_known(interp::Core.Compiler.NativeInterpreter, f::Any, fargs::Vector{Any}, argtypes::Vector{Any}, sv::Core.Compiler.InferenceState, max_methods::Int64)
    @ Core.Compiler ./compiler/abstractinterpretation.jl:1033
--- the last 11 lines are repeated 1 more time ---
 [56] abstract_call(interp::Core.Compiler.NativeInterpreter, fargs::Vector{Any}, argtypes::Vector{Any}, sv::Core.Compiler.InferenceState, max_methods::Int64)
    @ Core.Compiler ./compiler/abstractinterpretation.jl:1056
 [57] abstract_call(interp::Core.Compiler.NativeInterpreter, fargs::Vector{Any}, argtypes::Vector{Any}, sv::Core.Compiler.InferenceState)
    @ Core.Compiler ./compiler/abstractinterpretation.jl:1040
 [58] abstract_eval_statement(interp::Core.Compiler.NativeInterpreter, e::Any, vtypes::Vector{Any}, sv::Core.Compiler.InferenceState)
    @ Core.Compiler ./compiler/abstractinterpretation.jl:1167
 [59] typeinf_local(interp::Core.Compiler.NativeInterpreter, frame::Core.Compiler.InferenceState)
    @ Core.Compiler ./compiler/abstractinterpretation.jl:1447
 [60] typeinf_nocycle(interp::Core.Compiler.NativeInterpreter, frame::Core.Compiler.InferenceState)
    @ Core.Compiler ./compiler/abstractinterpretation.jl:1520
 [61] _typeinf(interp::Core.Compiler.NativeInterpreter, frame::Core.Compiler.InferenceState)
    @ Core.Compiler ./compiler/typeinfer.jl:214
 [62] typeinf(interp::Core.Compiler.NativeInterpreter, frame::Core.Compiler.InferenceState)
    @ Core.Compiler ./compiler/typeinfer.jl:209
 [63] typeinf
    @ ./compiler/typeinfer.jl:8 [inlined]
 [64] typeinf_type(interp::Core.Compiler.NativeInterpreter, method::Method, atypes::Any, sparams::Core.SimpleVector)
    @ Core.Compiler ./compiler/typeinfer.jl:914
 [65] return_type(interp::Core.Compiler.NativeInterpreter, f::Any, t::Any)
    @ Core.Compiler ./compiler/typeinfer.jl:963
 [66] return_type(f::Any, t::Any, world::UInt64)
    @ Core.Compiler ./compiler/typeinfer.jl:950
 [67] return_type(f::Any, t::Any)
    @ Core.Compiler ./compiler/typeinfer.jl:947
 [68] collect(itr::Base.Generator)
    @ Base ./array.jl:652
 [69] map(::Function, ::Matrix{typeof(∂(norm))}, ::FillArrays.Fill{Float64, 2, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}})
    @ Base ./abstractarray.jl:2383
 [70] (::Zygote.var"#1077#1084")(ȳ::Any)
    @ Zygote ~/.julia/packages/Zygote/6HN9x/src/lib/broadcast.jl:164
 [71] (::Zygote.var"#3902#back#1088")(Δ::Any)
    @ Zygote ~/.julia/packages/ZygoteRules/OjfTt/src/adjoint.jl:59
 [72] (::Zygote.var"#180#181")(Δ::Any)
    @ Zygote ~/.julia/packages/Zygote/6HN9x/src/lib/lib.jl:194
 [73] (::Zygote.var"#1689#back#182")(Δ::Any)
    @ Zygote ~/.julia/packages/ZygoteRules/OjfTt/src/adjoint.jl:59
 [74] Pullback
    @ ./broadcast.jl:1309 [inlined]
 [75] (::typeof(∂(broadcasted)))(Δ::FillArrays.Fill{Float64, 2, Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}}})
    @ Zygote ~/.julia/packages/Zygote/6HN9x/src/compiler/interface2.jl:0
 [76] Pullback
    @ ./REPL[22]:2 [inlined]
 [77] (::typeof(∂(loss)))(Δ::Float64)
    @ Zygote ~/.julia/packages/Zygote/6HN9x/src/compiler/interface2.jl:0
 [78] (::Zygote.var"#180#181")(Δ::Any)
    @ Zygote ~/.julia/packages/Zygote/6HN9x/src/lib/lib.jl:194
 [79] (::Zygote.var"#1689#back#182")(Δ::Any)
    @ Zygote ~/.julia/packages/ZygoteRules/OjfTt/src/adjoint.jl:59
 [80] Pullback
    @ ~/.julia/packages/Flux/6BByF/src/optimise/train.jl:102 [inlined]
 [81] (::typeof(∂(λ)))(Δ::Float64)
    @ Zygote ~/.julia/packages/Zygote/6HN9x/src/compiler/interface2.jl:0
 [82] (::Zygote.var"#69#70")(Δ::Any)
    @ Zygote ~/.julia/packages/Zygote/6HN9x/src/compiler/interface.jl:252
 [83] gradient(f::Function, args::Zygote.Params)
    @ Zygote ~/.julia/packages/Zygote/6HN9x/src/compiler/interface.jl:59
 [84] macro expansion
    @ ~/.julia/packages/Flux/6BByF/src/optimise/train.jl:101 [inlined]
 [85] macro expansion
    @ ~/.julia/packages/Juno/n6wyj/src/progress.jl:134 [inlined]
 [86] train!(loss::Function, ps::Zygote.Params, data::Vector{Tuple{Matrix{SVector{2, Float64}}, Matrix{SVector{3, Float64}}}}, opt::Descent; cb::Flux.Optimise.var"#40#46")
    @ Flux.Optimise ~/.julia/packages/Flux/6BByF/src/optimise/train.jl:99
 [87] train!(loss::Function, ps::Zygote.Params, data::Vector{Tuple{Matrix{SVector{2, Float64}}, Matrix{SVector{3, Float64}}}}, opt::Descent)
    @ Flux.Optimise ~/.julia/packages/Flux/6BByF/src/optimise/train.jl:97
 [88] top-level scope
    @ REPL[24]:1

[DhairyaLGandhi]

Interesting indeed. Could you try without static arrays?

[EdAyers]

Yeah it works without the static arrays

[DhairyaLGandhi]

A quick follow up, does it also work with cuda (without involving static arrays)?

[EdAyers]

I couldn't get it to work with CUDA without static arrays because it complained about not being a bits type or something. I am a complete beginner at Julia so I don't really know.

[DhairyaLGandhi]

Ah yeah, CUDA doesn't work with non bitstype objects indeed.

[ericphanson]

As a workaround, it seems to not error if I do

square(x) = x^2
my_norm(vec) = sqrt(sum(square, vec))
function loss(xy, rgb)
	sum(my_norm.(model.(xy) .- rgb))
end

as suggested by @mcabbott on Slack.

[simeonschaub]

MWE is

Zygote._pullback(Zygote.Context(), StaticArrays._norm, Size(3), SA[1., 2, 3])[2](1.)

Looks like Zygote.Pullback is not handling generated functions correctly somewhere.

[mcabbott]

Or less ambitiously, gradient(norm, SA[1, 2, 3])[1] ≈ gradient(norm, 1:3.0)[1].

[simeonschaub]

We should definitely be able to handle recursing into _norm as well though, since the actual code generated is not hard to differentiate.

[EdAyers]

A link to this issue #860 arose in the slack discussion