NGINX 404 not found when trying to reverse proxy a NoVNC server

Question

So I am currently setting up a default.conf file in nginx to reverse proxy an IP that is currently setup using NoVNC to have a virtual machine running on it, but when making the config file (I am very new to this go easy on me) I cannot seem to get it to direct to the specific IP I am looking for,

upstream vnc_proxy {
       server destination_IP:6080;
       
  
    }
    server {
    listen       443 ssl;
    ssl_certificate       /etc/ssl/certs/nginx-selfsigned.crt; 
    ssl_certificate_key  /etc/ssl/private/nginx-selfsigned.key;
    
    
    location /websockify {
     proxy_http_version 1.1;
          proxy_pass http://vnc_proxy/;
          proxy_set_header Upgrade $http_upgrade;
          proxy_set_header Connection "upgrade";

          # VNC connection timeout
          proxy_read_timeout 61s;

          # Disable cache
          proxy_buffering off;
    }

    location my_IP {
            index vnc.html;
            alias /path/to/noVNC/;
            try_files $uri $uri/ /vnc.html;
    } 
    
}

This is my current config file, I would like to direct the bottom location IP to proxy to the upstream vnc_proxy IP:6080. I had it working earlier however I realized that NoVNC requires some extra work with websockify and ssl certs

Following the above code I expected my_IP to then be proxy'd to destination_IP:6080 but instead whenever I try to open a web browser I get a 404 not found error from NGINX. Any advice on the small tweaks I need to fix this? Again very new to both NGINX and NoVNC so go easy on me!