Question about Interpreting Bert Layers in Interpreting qa with Bert tutorial

[xiaoda99]

In Interpreting Bert Layers Section (cell 26):

lc = LayerConductance(squad_pos_forward_func, model.bert.encoder.layer[i])
layer_attributions_start = lc.attribute(inputs=input_embeddings, baselines=ref_input_embeddings, additional_forward_args=(token_type_ids, position_ids,attention_mask, 0))[0]
layer_attributions_end = lc.attribute(inputs=input_embeddings, baselines=ref_input_embeddings, additional_forward_args=(token_type_ids, position_ids,attention_mask, 1))[0]

where input_embeddings and ref_input_embeddings are computed in construct_whole_bert_embeddings (cell 7):

input_embeddings = interpretable_embedding.indices_to_embeddings(input_ids, token_type_ids=token_type_ids, position_ids=position_ids)
ref_input_embeddings = interpretable_embedding.indices_to_embeddings(ref_input_ids, token_type_ids=token_type_ids, position_ids=position_ids)
 
interpretable_embedding = configure_interpretable_embedding_layer(model, 'bert.embeddings')  (cell 25)

To my understanding, input_embeddings and ref_input_embeddings thus obtained are outputs of BertEmbeddings layer, i.e. summation of input_embedding, position_embedding and token_type_embedding. They are not the "inputs" arg of squad_pos_forward_func, which should be input_ids. If this is true, why are token_type_ids and position_ids still needed in additional_forward_args?

• Da Xiao

[NarineK]

Hi @xiaoda99, that's a good question. I think that those arguments might still be used in the forward pass outside computing bert embeddings. I have to rerun and check again - I don't remember exactly. You can try without passing token_type_ids and position_ids as additional arguments and see if it works as expected.

[xiaoda99]

Thank you. Another thing that confuses me is: why can lc.attribute's "inputs" arg receive embeddings (real-valued vectors)? Shoudn't it receive input_ids (integer indices)? It is determined by the signature of model's forward method:
model(inputs, token_type_ids=token_type_ids,
position_ids=position_ids, attention_mask=attention_mask, )

[NarineK]

Hi @xiaoda99, it is the case because Layer Conductance computes the gradients of selected output w.r.t. scaled inputs.
https://github.com/pytorch/captum/blob/master/captum/attr/_core/layer/layer_conductance.py#L335
If we scale token indices it won't make sense, that's why we scale real-valued embeddings.

We could potentially, take the input indices, compute embedding output internally, but, for simplicity, it is currently not implemented that way.

[NarineK]

@xiaoda99 is this still an open issue ?