Deprecate remaining VI methods

[ricardoV94]

#4582 left some old VI methods marked as NotImplemented

Normalizing flows:

pymc/pymc/variational/approximations.py

Line 363 in ef5f91b

raise NotImplementedInference("Normalizing flows are not yet ported to v4")

And these:

pymc/pymc/variational/opvi.py

Lines 959 to 962 in e987950

	if self.local:
	raise NotImplementedInference("Local inferene aka AEVB is not supported in v4")
	if self.batched:
	raise NotImplementedInference("Batched inferene is not supported in v4")

We should decide whether we want to refactor or remove them

[ricardoV94]

CC @ferrine, @fonnesbeck, @twiecki

[twiecki]

My vote is on removing them, don't think anyone uses those. If they do, they'll probably complain. We could ask on discourse too.

[fonnesbeck]

I'm pretty sure OVPI can go, but normalizing flows are something we would want. They are a pretty useful and are available in most VI implementations these days (e.g. pyro, TF). That said, they can always be added back if anyone ever wants to implement them.

[ferrine]

I'd vote for removal, agree with @fonnesbeck. This functionality did not prove to be super useful.

[fonnesbeck]

@ferrine do you think it will be straightforward to get normalizing flows ported? Were they fully functional in v3?

[ferrine]

they were, but that time I struggled with shape inference for parameter creation. In v3 I relied on test values for NF, now test values are less strict and it broke the internals. To fix this issue new mechanism has to be implemented to initialize normalizing flows.

[ferrine]

To summarize there are 3 blocks of features

• AEVB that is all self.local occasions. It is rarely used, and not PyMC focus
• batched block sparse inference. All self.batched occasions. Was used to improve UX of AEVB
• Normalizing flows. Normalizing flows API is nice but stays very far from PyMC focus. I'm ok to just delete it

[ricardoV94]

So shall we delete everything and open an issue to re-implement normalizing flows in the future (either here or in pymc-experimental)?

[twiecki]

That sounds good to me.