Rotating a 3D direction vector upwards with `glm::rotate` and quaternions

Question

Given the following coordinate system¹, where positive z goes towards the ceiling:

I have a glm::vec3 called dir representing a (normalized) direction between two points A and B in 3D space²:

The two points A and B happen to be on the same plane, so the z coordinate for dir is zero. Now, given an angle α, I would like to rotate dir towards the ceiling by the specified amount. As an example, if α is 45 degrees, I would like dir to point in the same x/y direction, but 45 degrees towards the ceiling³:

My original idea was to calculate the "right" vector of dir, and use that as a rotation axis. I have attempted the following:

glm::vec3 rotateVectorUpwards(const glm::vec3& input, const float aRadians)
{
    const glm::vec3 up{0.0, 0.0, 1.0};
    const glm::vec3 right = glm::cross(input, glm::normalize(up));

    glm::mat4 rotationMatrix(1); // Identity matrix
    rotationMatrix = glm::rotate(rotationMatrix, aRadians, right);
    return glm::vec3(rotationMatrix * glm::vec4(input, 1.0));
}

I would expect that invoking rotateVectorUpwards(dir, glm::radians(45)) would return a vector representing my desired new direction, but it always returns a vector with a zero z component.

I have also attempted to represent the same rotation with quaternions:

glm::quat q;
q = glm::rotate(q, aRadians, right);
return q * input;

But, again, the resulting vector always seems to have a zero z component.

What am I doing wrong?

Am I misunderstanding what the "axis of rotation" means?
Is my right calculation incorrect?

How can I achieve my desired result?

Can you try `glm::normalize(input + glm::vec3{0.0, 0.0, tan(aRadians)})`? It should do the same. — Nelfeal, May 21 '20 at 11:13
normalize the right vector before constructing your rotation matrix? — noobius, May 21 '20 at 16:03
@noobius, `input` is already normalized - wouldn't `right` be normalized as well? — Vittorio Romeo, May 21 '20 at 16:34
@VittorioRomeo, not necessary that only is true if your input and up vectors are orthogonal and are unit vectors. If they're not orthogonal then you would need to normalize the result. I don't know what your input direction vector may be, so it's probably safe to normalize the result after doing a cross product. — noobius, May 21 '20 at 16:50

score 0 · Answer 1 · answered Sep 26 '20 at 09:06

You don't need to normalize your up vector because you defined it to be a unit vector, but you do need to normalize your right vector.

However, while I am unfamiliar with glm, I suspect the problem is you are rotating the matrix (or quaternion) around your axis rather than creating a matrix/quaternion that represents a rotation around your axis. taking a quick look at the docs, it looks like you might want to use:

glm::mat4 rotationMatrix = glm::rotate(radians, right);

Rotating a 3D direction vector upwards with `glm::rotate` and quaternions

1 Answers1