Unexpected Union type given constructor

[timholy]

Here's another inference surprise triggered by #7568:

julia> Base.return_types(*, (typeof(AfM), typeof(BfM)))
1-element Array{Any,1}:
 Union(FixedArrayM{Float64,2,SZ},Array{Float64,2})

There's no way that function can return an Array{Float64,2}.

[simonster]

This is somewhat related to #7569. If type inference can't figure out SZ, then it has to consider the possibility that the generic variant of A_mul_B! could be called rather than the variant you've defined. However, that wouldn't introduce type instability if it weren't for this line:

julia/base/linalg/matmul.jl

Line 277 in 4cfb9a1

if mA == 0 || nA == 0 || nB == 0; return zeros(T, mA, nB); end

[Jutho]

Why is that line like this. Can't one just check C to be of the correct size and then return fill!(C,0).

[simonster]

@Jutho Yes, I think that's the right thing to do.