In Autograd:

If x is a Tensor that has x.requires_grad=True then x.grad is another Tensor holding the gradient of x with respect to some scalar value.

should be

If x is a Tensor that has x.requires_grad=True then x.grad is another Tensor holding the gradient of scalar (usually loss) with respect to x.