shape attribute not updated when using it with a shared variable

[denadai2]

I have a problem with the shape parameter, that doesn't get updated when I update a shared variable. This is a toy example: it doesn't make so sense by itself but it is to reproduce the error.

from pymc3.distributions import distribution
from theano import shared


class CAR2(distribution.Continuous):
	def __init__(self, mu, *args, **kwargs):
		super(CAR2, self).__init__(*args, **kwargs)

		self.mean = mu

	def logp(self, x):
		return tt.sum(continuous.Normal.dist(mu=0, tau=0.001).logp(x))


df = pd.read_csv('data_bug.csv')[['land_use_mix3', 'y']]

train_index = list(range(df.values.shape[0]-1))
test_index = [df.values.shape[0]-1]

X_shared = shared(df[['land_use_mix3']].values[train_index, :][:, 0])
y_shared = shared(df['y'].values[train_index])


with pm.Model() as pooled_model:
    coeffs = pm.Normal('asd', mu=0, tau=0.001)
    mu_phi = CAR2('mu_phi', shape=X_shared.get_value().shape[0], mu=tt.zeros(X_shared.get_value().shape[0]))
    
    mu = tt.exp(X_shared.ravel() * coeffs + mu_phi)
    
    obs = pm.Poisson('obs', mu=mu, observed=y_shared)
    
    pooled_trace = pm.sample(100, njobs=1, tune=100)
    
X_shared.set_value(df[['land_use_mix3']].values[test_index, :][:, 0])
y_shared.set_value(df['y'].values[test_index])
    
ppc = pm.sample_ppc(pooled_trace, samples=1000, model=pooled_model)

the problem is with shape parameter of CAR2, which stays = to 76 but it should be shape=1 after the update.

ValueError: Input dimension mis-match. (input[0].shape[0] = 1, input[2].shape[0] = 76)
Apply node that caused the error: Elemwise{Composite{exp(((i0 * i1) + i2))}}(<TensorType(float64, vector)>, InplaceDimShuffle{x}.0, mu_phi)
Toposort index: 1
Inputs types: [TensorType(float64, vector), TensorType(float64, (True,)), TensorType(float64, vector)]
Inputs shapes: [(1,), (1,), (76,)]
Inputs strides: [(8,), (8,), (8,)]
Inputs values: [array([-1.30698299]), array([-1.67981393]), 'not shown']
Outputs clients: [['output']]

file:

data_bug.csv.zip

[aseyboldt]

You set the shape of mu_phi statically to the original shape. X_shared.get_value() returns a numpy array. What happens if you use

mu_phi = CAR2('mu_phi', mu=tt.zeros_like(X_shared))

or maybe

mu_phi = CAR2('mu_phi', mu=tt.zeros_like(X_shared), shape=X_shared.shape)

[denadai2]

mhh ok! with the first solution I have an error (I have to specify shape)

with the second one:

TypeError: Expected int elements in shape

Is there a way to "dynamically" get the shape as int?

[aseyboldt]

Hm. This is more complicated than I thought. I think the problem is that the bijection object for converting the model variables between array and dict expects constant shapes. At first I thought this shouldn't be a problem because we are doing something similar for minibatches, but I think that only applies to observed variables. So I don't expect that to change soon. In general it seems dynamically changing shapes are difficult to deal with.

[junpenglao]

@denadai2 I think about it a bit more as well: In principle you should have a RV that can be broadcasted to a dynamic shape for this to work. Something along the line of b ~ Normal(), Y ~ Normal(X*b, sigma) where the shape of X and Y can be dynamic. Notice that if you rewrite is as Y = X*b + eps, eps ~ Normal(0, sigma) it is the same but if you want to make the eps has a dynamic shape is much more difficult.

So in your case, can you reparameterize the model?

[denadai2]

@aseyboldt mhh ok. The problem is that this appears when you have to do predictions with holdout data and multivariate models like this one :(
I don't know what to do

@junpenglao how could I? Any suggestion? In the "real" problem I have, CAR is a set of parameters extracted from a Z (shared) variable. Since Z changes, I have to change the CAR shape as well...

[junpenglao]

You can have a look at the sample_gp function. The idea is to reapply CAR on the new Z using some matrix multiplication:
https://github.com/pymc-devs/pymc3/blob/master/pymc3/gp/gp.py#L79

Another idea is that, below your training model, add a generation model (similar to Stan):

with pm.Model():
    [...]
    para = CAR(**args, shape=Xtrain.shape)
    likelihood = dist(**args, observed=Ytrain)
    [...]
    para_new = CAR(**args, shape=Xtest.shape)
    y_predict = dist(**args)

And then compare the y_predict with Ytest for your purpose.

[denadai2]

@junpenglao I tried to think about a possible solution, but CAR is a vector estimated from an adjacency matrix, so it is not easy to do a matrix multiplication (I think!) because the new parameters depend on the matrix of X_test.

For the generation model, do you mean something like this?

with pm.Model():
    [...]
    beta = pm.Normal("beta1", mu=0, tau=0.01)
    para = CAR(**args, shape=Xtrain.shape)
    likelihood = Poisson("outcome", mu = tt.exp(para + X_train.dot(beta)), observed=Ytrain)
    [...]
    para_new = CAR(**args, shape=Xtest.shape)
    y_predict = Poisson("outcome_test", mu = tt.exp(para_new + X_test.dot(beta)))
    pooled_trace = pm.sample(1000, njobs=1, tune=1000)

I didn't understand the meaning so much...

Thanks for your patience in advance :)

[junpenglao]

@denadai2 Yep that's exactly what I meant ;-)