Unity RotateAround Clamp Dead Zone Issue

Question

For the past 3 days, I have tried numerous methods and read dozens of questions on forums in order to achieve the desired effect of rotating a camera on the Y axis using mouse rotations, and clamping the rotation within a certain range. The amount of rotation is dependent on how close the mouse is to the edge of the screen.

I was unable to find a solution, but I was able to learn enough to have my own honest attempt at it, and I came quite close. The prominent issue I faced was being able to properly clamp the RotateAround() inside of the desired range. This is because of eulerAngles being 0 - 360, and having the min and max of the clamp transition to the other side of the spectrum.

By clamping the rotation degrees before calling RotateAround, I was able to get the clamping to work, though I had to use Booleans to get them to work properly.

My current issue is that when the initial Y rotation(anchor) is slightly below 360, the camera fails to clamp on the left and will clamp back around to the right side of the range. In all other situations, the clamping works just fine:

• Initial Y rotation slightly above 0
• Not within the dead zone

While debugging, I find that this is because rightOverFlag is true. When I set this bool to true manually in situations clamping normally works fine, the clamping will no longer work on the left. I can't seem to find how this bool being true causes something like this to happen, so I am hoping some fresh eyes and seasoned advice can help me learn from this. Thank you.

public gameState gameState;
public openCams openCams;

private GameObject GameStateManager;
[SerializeField]
private GameObject Player;

[SerializeField]
private Camera cam;

// Rotation Variables
private Quaternion initialRotation;

private float initialYRotation = 0f;

[SerializeField]
private float angleRange;

private float anchorY = 0f;

public bool leftOverFlag = false;
public bool rightOverFlag = false;

void Start()
{
    cam = Camera.main;

    Cursor.lockState = CursorLockMode.None;
}

public void OrientControls() 
{
    initialRotation = Player.transform.rotation;
    initialYRotation = Player.transform.eulerAngles.y;

    anchorY = initialYRotation;

    if (anchorY > 360)
    {
        anchorY -= 360;
    }

    rightOverFlag = false;
    leftOverFlag = false;
    if ((anchorY + angleRange) > 360)
    {
        rightOverFlag = true;
    }
    else if ((anchorY - angleRange) <= 0)
    {
        leftOverFlag = true;
    }
}

void Update()
{
    if (openCams.GetMonitorState() == false)
    {
        mousePos = Input.mousePosition;

        float rotateDegrees = 0f;

        //Debug.Log(Player.transform.eulerAngles.y);

        // THERE IS CODE HERE THAT INCREASES ROTATE DEGREES BASED ON MOUSE POSITION. I removed it for the sake of readability, because it was quite long and unrelated to my issue.

        float angleFromInitial = Quaternion.Angle(initialRotation, Player.transform.rotation);

        float currentPlayerYRot = Player.transform.eulerAngles.y;

        if (currentPlayerYRot > 360)
        {
            currentPlayerYRot -= 360;
        }

        bool currentRightOverageFlag = false;
        bool currentLeftOverageFlag = false;
        if (rightOverFlag)
        {
            if ((currentPlayerYRot) < (anchorY - (angleRange - 5))) // 
            {
                currentRightOverageFlag = true;
            }
        }
        if (leftOverFlag)
        {
            if ((currentPlayerYRot) > (anchorY + (angleRange + 5)))
            {
                currentLeftOverageFlag = true;
            }
        }
        

        // !!! - For some reason, when rightOverFlag is enabled, the clamp does not work on the left side. In all other situations, the clamp works perfectly.- !!!

        if (!currentLeftOverageFlag && !currentRightOverageFlag)
        {
           if (currentPlayerYRot < anchorY) // Regular
           {
               angleFromInitial *= -1;
           }
        
        }
        else if (currentLeftOverageFlag && !currentRightOverageFlag)
        {
            if (currentPlayerYRot > anchorY) // If over the left line
            {
                angleFromInitial *= -1;
            }
        }
        else if (!currentLeftOverageFlag && currentRightOverageFlag)
        {
            if (currentPlayerYRot > anchorY) // If over the right line
            {
                angleFromInitial *= -1;
            }
        }
        else
        {
            Debug.Log("staticPersonController: ERROR => Cannot have current left and right overage flags enabled at the same time.");
        }

        currentLeftOverageFlag = false;
        currentRightOverageFlag = false;

        float newAngle = Mathf.Clamp(angleFromInitial + rotateDegrees, -angleRange, angleRange);

        rotateDegrees = newAngle - angleFromInitial;

        Player.transform.RotateAround(Player.transform.position, Vector3.up, rotateDegrees);


    }
}

}

Answer 1

Assume you have these two variables set.

// This is the initial value of the player's Y rotation. Set only once.
float initialRotY = Player.transform.eulerAngles.y;

// This is the offset of the player's Y rotation based on the cursor position
// e.g. -90 when the cursor is on the left edge and 90 on the right
float mouseBasedRotY;

In the Update method, just do this.

Player.transform.rotation = Quaternion.Euler(0f, initialRotY + mouseBasedRotY, 0f);

You don't have to clamp the value, because the degree is always limited in [initialRotY-90, initialRotY+90]