Permit `ComponentArray`s as simulation condition (u0 + ps)

[TorkelE]

Component arrays got tags for each element. This could be used by modelling toolkit to determine individual parameters and initial condition values (in a similar way as done for e.g. a map).

While this does not directly add anything, Lux uses ComponentArrays to represent neural network parameters. If MTK could use these, it should be relatively strightforward to work with UDEs in MTK (as one could simply merge the ComponentArray representing neural network parameters and those representing otehr parameter values and initial conditions. Next, Optimization.jl can run directly using ComponentArrays as well, so these could be used unmodified in the loss function.

This or something equivalently convenident might actually already by possible, but if so, I am not aware of it.

@SebastianM-C

[ChrisRackauckas]

I'm not sure this is actually easy. It would only really be possible for the tunables. For large models it could make there be a pretty large compilation time, since there can be thousands of names?

[TorkelE]

Ok, that sounds like a problem. In that case it might be better to work with what we got in MTK, and see if there are adaptations/additions that can be made in ModelingToolkitNeuralNets.jl to make things easier to work with. At some point I plan to make a short write-up on various usability ideas that potentially could make MTK UDEs easier to work with.

[ChrisRackauckas]

Let's follow up with this after the next wave of major changes. FYI, SymbolicUtils.jl v4 is making Symbolics fully type-stable and precompilation work well, so that's a massive improvement (landing this week?). Then following that we're going to rewrite the codegen to make better use of symbolic array expressions, so A*x is fully preserved through ModelingToolkit (except in specific DAE scenarios where it requires cutting, etc.). And then have an optional Reactant setup which automatically optimizes a lot of the linear algebra. That's set to be done in like the next month. When we are there, we should revisit the interface on top of it.

[TorkelE]

Sounds good, maybe something we could look at at towards the end of year?

[ChrisRackauckas]

Yes