Skip to content

Conversation

@OliBomby
Copy link
Contributor

What does this PR do?

Fixes KeyError in 4D attention mask creation for encoder-decoder models when generating with attention mask and static cache.

This occurs because inputs_embeds does not get added to model_inputs for encoder-decoder models, like Whisper. Subsequently it tries to check if model_inputs["inputs_embeds"] is none and throws a KeyError.

cc @gante

@gante
Copy link
Contributor

gante commented May 22, 2025

@OliBomby good catch! Thank you for opening the PR :D

This and other encoder-decoder issues are being addressed in #38235 and #38108, so I'm not going to merge this PR to avoid creating merge conflicts 🤗 (cc @vasqu)

@OliBomby
Copy link
Contributor Author

Ah in that case I'll close this PR. Hope to see the others merged soon!

@OliBomby OliBomby closed this May 22, 2025
@OliBomby OliBomby deleted the patch-1 branch May 22, 2025 09:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants