Allow for a named tuple for `initial_params`

[DominiqueMakowski]

My understanding is that currently initial_params must be a vector of the length of the parameters, but this can become a bit tricky for models with lots of parameters.

Would it be possible to allow for passing, for instance, a named tuple (initial_params=(μ=2.5, σ=1.0)) to set initial values on desired parameters?

[Red-Portal]

I would also love to see named tuple generally supported across the Turing ecosystem, but that would probably need some work. It's also somewhat dependent on this PR

[torfjelde]

Would it be possible to allow for passing, for instance, a named tuple (initial_params=(μ=2.5, σ=1.0)) to set initial values on desired parameters?

Definitively possible! I made a quick PoC concept here: TuringLang/DynamicPPL.jl#632

I'm somewhat limited in time these days, but maybe someone can complete it (shouldn't be much work).

I would also love to see named tuple generally supported across the Turing ecosystem, but that would probably need some work. It's also somewhat dependent on JuliaStats/Distributions.jl#1803

Supporting initial_params::NamedTuple would actually be quite a bit simpler than supporting NamedTuple across the entire TuringLang ecosystem:)

[Red-Portal]

@torfjelde We don't yet officially support passing NamedTuples to a LogDensityFunction right?

[torfjelde]

We don't yet officially support passing NamedTuples to a LogDensityFunction right?

No, but we don't need this to support initial_params::NamedTuple.

[DominiqueMakowski]

Another potential feature for TuringLang/DynamicPPL.jl#632 would be to set all/some of these initial parameters as functions that are then applied the prior. For instance, mean() or mode() (I reckon this would be quite convenient to use, like sample(..., init_params=mean(), to take the mean of all the priors as starting params)

[torfjelde]

Though I agree that seems convenient, it's just a bit too much hassle to maintain as it really only saves a single call from the user perspective:

init_parmas=mean(rand(Vector, model))`

(though this is somewhat "new" / not well-documented tbh)

[sunxd3]

I agree with Tor

[torfjelde]

This has now been solved by TuringLang/DynamicPPL.jl#632 I believe? @sunxd3 ?:)

[DominiqueMakowski]

Cheers for the work!

[DominiqueMakowski]

Small thing, but mean(rand(Vector, model)) does not return the mean for each parameter (but just the mean of a random draw from all the parameters). Thus I'm not sure how to initialize using the mean of the prior distribution of each parameter?

[sunxd3]

I am probably understanding wrongly, but for

@model function f()
    x ~ Normal(0, 1)
    y ~ Normal(x, 1)
end

mean of x is 0, is this what you mean? And yes, what would the mean of y is?

[torfjelde]

Small thing, but mean(rand(Vector, model)) does not return the mean for each parameter (but just the mean of a random draw from all the parameters). Thus I'm not sure how to initialize using the mean of the prior distribution of each parameter?

Ah sorry, yes that's very true. I wrote that a bit too quickly trying to convey that you can take the mean of smaoples from the model to get it.

You can do:

chain_prior = sample(model, Prior(), 1000)

and then extract from this:)

[DominiqueMakowski]

Well yes but my point was about some syntactic sugar to conveniently (and efficiently) set the initial parameters to the prior means, and sampling from the priors first appears as overkill given that the distributions usually have analytically defined means.

Hence the initial_params=:mean proposition or ``initial_params=mean` (i.e., passing a function that would be applied to each prior distribution).

Alternatively, is there a way to extract the prior distributions as a vector, for instance:

priors = get_priors(model)  
priors

(mu=Normal(0, 1), sigma=Gamma(2, 2))

One could then run:

initial_params=mean.(priors)