How to keep weights of parts of a model fixed under Flux.train!

[fps]

Hi, maybe I'm just too dense in reading the docs, but I couldn't quite figure out a nice way to do this. Let's say I have a model

m = Flux.Chain(Flux.Conv(...), Flux.Conv(...), Flux.Conv(...), etc.. pp)

and I wanted to keep the parameters of the first chain entry fixed under Flux.train! how would I best go about it? The first thought was to simply have a "wrapper" like

struct Fixed
  m
end
(f::Fixed)(x) = f.m(x)

which is explicitly not a Flux.@functor and just use that in m:

m = Flux.Chain(Fixed(Flux.Conv(...)), Flux.Conv(...), Flux.Conv(...), etc.. pp)

But sadly that breaks moving to the GPU. I tried reading the Flux.@functor code, but sadly it is a little bit over my head. I guess another approach would be to pick apart the Flux.params(m) return value and take out everything I don't want updated, but I'm sure there's a nicer way that composes more intuitively.

[DhairyaLGandhi]

An easy enough way is to only pass the Params that you do want to train. That is, try params(model[2:end]) while passing into to train!

[fps]

Thanks for your suggestion. Where this gets unwieldy though is once you compose these things again. I.e. I have a "Layer" which really is a Flux.Chain with some fixed components and I construct a bigger model out of these layers.

While it's still possible to collect the params with special knowledge about the component, I would prefer a way to make this layer reusable without the user having to know about the internals. So the original question still stands..

[DhairyaLGandhi]

Another way might be to declare the Layer with
@functor Layer (a,b,...) where the tuple contains the fields that would be collected with the call to params.

[fps]

Hmm, I will try that. Though doesn't functor play a double role here? In that it also facilitates moving the parameters of the Layer to the GPU?

[CarloLucibello]

duplicate of #1022