mps cross-attention hack: don't crash on fp16 #2258
Conversation
|
The documentation is not available anymore as the PR was closed or merged. |
| @@ -234,7 +234,7 @@ def prepare_attention_mask(self, attention_mask, target_length): | |||
| # HACK: MPS: Does not support padding by greater than dimension of input tensor. | |||
| # Instead, we can manually construct the padding tensor. | |||
| padding_shape = (attention_mask.shape[0], attention_mask.shape[1], target_length) | |||
| padding = torch.zeros(padding_shape, device=attention_mask.device) | |||
| padding = torch.zeros(padding_shape).to(attention_mask) | |||
There was a problem hiding this comment.
| padding = torch.zeros(padding_shape).to(attention_mask) | |
| padding = torch.zeros(padding_shape).to(attention_mask.device) |
no?
There was a problem hiding this comment.
I recently learned that doing it that way changes both the device and the dtype at once. But it may not be clear, so I'll change it.
There was a problem hiding this comment.
Ah interesting! Good to keep in mind :-)
In general, it's better IMO to stick to our existing API usage, e.g. we always use f-strings, ... so even if it requires more code IMO we should stick to what we have currently and change it only if we change the whole codebase so that our code design stays consistent.
Also for such cases we should also be loosely aware of whether it's a very recent feature of PyTorch or already there since ~2 years. If it's there since ~2 years then happy to ad
There was a problem hiding this comment.
It's been there for several versions, I verified :) But I do agree that it's nowhere in our codebase, so it's better to err on the explicit side.
|
Ah, thanks @pcuenca ! |
* mps cross-attention hack: don't crash on fp16 * Make conversion explicit.
* mps cross-attention hack: don't crash on fp16 * Make conversion explicit.
* mps cross-attention hack: don't crash on fp16 * Make conversion explicit.
Found while testing #1791.
The pipeline doesn't crash, but it still doesn't work.