Gradient not calculated for transforms.matrix_to_quaternion

[m-nez]

It looks like the gradient is not calculated for transforms.matrix_to_quaternion.

import pytorch3d.transforms
import torch

matrix = torch.eye(3, requires_grad=True)
target = torch.Tensor([0.984808, 0, 0.174, 0])

optimizer = torch.optim.Adam([matrix], lr=0.05)
optimizer.zero_grad()
q = pytorch3d.transforms.matrix_to_quaternion(
    matrix
)
loss = torch.sum((q - target)**2)
loss.backward()
optimizer.step()

print(matrix)

After running the code above the matrix after optimizer step shouldn't contain NaNs, but it does.

[gkioxari]

You're absolutely correct! After some investigation, the issue comes from the somewhat unexpected backward behaviour of torch.sqrt(torch.max(zero, x)) when x < 0 in the implementation in L99 - L102. We are working on a fix but in the meantime you can unblock yourself by adding torch.sqrt(torch.max(zero, x) + eps) where eps = 1e-8.

[bottler]

This should now be fixed by the mentioned commit.