Nginx still redirects even though I removed the rule from the conf

Question

I was using this block for redirecting the website's IP to the actual URL:

# IP to domain redirect
server {
        # tell on which port this server listens
        listen [::]:80;
        listen 80;

        # listen on the IP
        server_name xx.xx.xx.xxx;

        # and redirect to the domain
        return 301 $scheme://example.com$request_uri;
}

However I decided to remove it but the redirection doesn't go away. I'm aware that my browser had already hard-cached that redirection and I won't see the change so I used curl -I xx.xx.xx.xxx via SSH to see the response, where xx.xx.xx.xxx is the server's Public IP:

HTTP/1.1 301 Moved Permanently
Server: nginx
Date: Tue, 10 Jan 2017 14:06:01 GMT
Content-Type: text/html
Content-Length: 178
Connection: keep-alive
Location: http://example.com/
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
X-UA-Compatible: IE=Edge

As you can see the redirection is still there. In addition I pinged the IP from third party sources and they confirm that the 301 redirect is there. So I'm really confused how is this possible when there's no redirection rule at all.

Other things I've done:

• I've checked my iptables and there are no PREROUTING chains of any kind.
• I didn't forget to restart Nginx and I even rebooted the server.
• Since I'm using FastCGI Caching I've purged the whole FastCGI Nginx cache.
• Since the server is behind CloudFlare I've purged the whole cache on their end too. But it is obvious that the above 301 response is not coming from CloudFlare because they set easily recognizable headers and those are not theirs.

Here's the Nginx conf. Obviously in production this block is segmented in several files but for the purpose of this question I paste it in one block.

user www-data;
worker_processes 4;
pid /run/nginx.pid;

events {
        worker_connections 1024;
        multi_accept on;
}

worker_rlimit_nofile 10000;

http {
        ##
        # Basic Settings
        ##

        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;

        client_body_timeout 12;
        client_header_timeout 12;
        keepalive_timeout 15;
        send_timeout 10;

        open_file_cache          max=10000 inactive=5m;
        open_file_cache_valid    2m;
        open_file_cache_min_uses 1;
        open_file_cache_errors   on;

        types_hash_max_size 2048;
        server_tokens off;

        server_names_hash_bucket_size 64;
        server_name_in_redirect off;

        include /etc/nginx/mime.types;
        default_type application/octet-stream;

        ##
        # Logging Settings
        ##

        access_log /var/log/nginx/access.log;
        error_log /var/log/nginx/error.log;

        ##
        # Gzip Settings
        ##

        gzip on;
        gzip_disable "msie6";

        gzip_vary on;
        gzip_proxied any;
        gzip_comp_level 6;
        gzip_min_length 256;
        gzip_buffers 16 8k;
        gzip_http_version 1.1;
        gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

        ##
        # Virtual Host Configs
        ##

        fastcgi_cache_path /var/run/nginx-cache levels=1:2 keys_zone=WORDPRESS:800m inactive=4h;
        fastcgi_cache_key "$scheme$request_method$host$request_uri";
        fastcgi_cache_use_stale error timeout invalid_header http_500;
        fastcgi_ignore_headers Cache-Control Expires Set-Cookie;
        add_header X-Cache $upstream_cache_status;
        add_header X-Content-Type-Options nosniff;
        add_header X-XSS-Protection "1; mode=block";
        add_header "X-UA-Compatible" "IE=Edge";

        # www to non-www redirect
        server {
                # don't forget to tell on which port this server listens
                listen [::]:80;
                listen 80;

                # listen on the 'www' host
                server_name www.example.com;

                # and redirect to the non-www host (declared below)
                return 301 $scheme://example.com$request_uri;
        }

        # Begin the server config
        server {
                listen [::]:80;
                listen 80;

                # The host name to respond to
                server_name example.com;

                # The location of access and error logs (symlinked to /var/www/example.com/logs/)
                access_log /var/log/nginx/example.com.access.log;
                error_log /var/log/nginx/example.com.error.log;

                # Root for static files
                root /var/www/example.com/htdocs;
                index index.php index.html index.htm;

                # Find and replace CDN urls
                subs_filter example.com/wp-content/uploads/         cdn.example.com/wp-content/uploads/;
                subs_filter example.com/wp-content/themes/          cdn.example.com/wp-content/themes/;
                subs_filter example.com/wp-content/plugins/         cdn.example.com/wp-content/plugins/;
                subs_filter example.com/apple-touch-icon.png        cdn.example.com/apple-touch-icon.png;

                # Cache every page
                set $skip_cache 0;

                # POST requests and urls with a query string should always go to PHP
                if ($request_method = POST) {
                        set $skip_cache 1;
                }

                # Don't cache queries
                if ($query_string) {
                        set $skip_cache 1;
                }

                # Cache only these queries
                if ($query_string ~* "orderby|r_sortby|results") {
                        set $skip_cache 0;
                }

                # Don't cache uris containing the following segments
                if ($request_uri ~* "/wp-admin/|/xmlrpc.php|/opcache.php|wp-.*.php|/feed/|index.php|sitemap(_index)?.xml") {
                        set $skip_cache 1;
                }

                # Don't use the cache for logged in users or recent commenters
                if ($http_cookie ~* "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_no_cache|wordpress_logged_in") {
                        set $skip_cache 1;
                }

                # Don't log robots.txt and favicon.ico
                location ~ /(robots.txt|favicon.ico) { 
                        access_log off; 
                        log_not_found off; 
                }

                # Prevent clients from accessing hidden files (starting with a dot) - .htaccess or .htpasswd
                location ~ /\. { 
                        deny all; 
                        access_log off; 
                        log_not_found off; 
                }

                # Prevent clients from accessing backup/config/source files
                location ~* (?:\.(?:bak|config|sql|fla|psd|ini|log|sh|inc|swp|dist)|~)$ { 
                        deny all; 
                        access_log off;
                        log_not_found off;
                }

                # Don't allow access to files in /wp-admin/includes folder
                location ~* /wp-admin/includes { 
                        deny all;
                        access_log off;
                        log_not_found off;
                }

                # Don't allow php execution in wp-includes folder
                location ~* /wp-includes/.*.php$ {
                        deny all;
                        access_log off;
                        log_not_found off;
                }

                # Don't allow php execution in wp-content folder
                location ~* /wp-content/.*.php$ {
                        deny all;
                        access_log off;
                        log_not_found off;
                }

                # Don't allow php execution in uploads and files folders
                location ~* /(?:uploads|files)/.*\.php$ { 
                        deny all; 
                        access_log off;
                        log_not_found off;
                }

                # Deny access to XML-RPC requests
                # location = /xmlrpc.php {
                #       deny all;
                #       access_log off;
                #       log_not_found off;
                # }

                # Deny access to sensitive files
                location ~ /(wp-config.php|install.php|readme.html|licence.txt) { 
                        deny all;
                        access_log off;
                        log_not_found off;
                }

                # Redirect feed to FeedBurner
                if ($http_user_agent !~ "FeedBurner|FeedValidator") {
                        rewrite ^/feed/?.*$ http://feeds.example.com/example redirect;
                }

                # Strip/Redirect these queries
                if ($query_string ~* "fb_xd_fragment|fb_action_ids|iframe") {
                        rewrite ^(.*)$ $1? redirect;
                }

                rewrite ^/sitemap(-+([a-zA-Z0-9_-]+))?\.xml$ "/index.php?xml_sitemap=params=$2" last;
                rewrite ^/sitemap(-+([a-zA-Z0-9_-]+))?\.xml\.gz$ "/index.php?xml_sitemap=params=$2;zip=true" last;
                rewrite ^/sitemap(-+([a-zA-Z0-9_-]+))?\.html$ "/index.php?xml_sitemap=params=$2;html=true" last;
                rewrite ^/sitemap(-+([a-zA-Z0-9_-]+))?\.html.gz$ "/index.php?xml_sitemap=params=$2;html=true;zip=true" last;

                location / {
                        try_files $uri $uri/ /index.php?$args;
                }

                location ~ \.php$ {
                        # try_files $uri =404;
                        # try_files $uri =404 /index.php?$args;
                        try_files $uri /index.php?$args;
                        fastcgi_split_path_info ^(.+\.php)(/.+)$;

                        fastcgi_pass unix:/run/php/php7.0-fpm.sock;
                        include fastcgi_params;

                        # Combat (110: Connection timed out) while reading response header from upstream
                        fastcgi_read_timeout 120;

                        fastcgi_cache_bypass $skip_cache;
                        fastcgi_no_cache $skip_cache;

                        fastcgi_cache WORDPRESS;
                        fastcgi_cache_valid 4h;

                        # combat error 'upstream sent too big header...' 
                        fastcgi_buffers 32 64k;
                        fastcgi_buffer_size 64k;
                }

                # Purge ULR only within localhost
                location ~ /purge(/.*) {
                        allow 127.0.0.1;
                        deny all;
                        fastcgi_cache_purge WORDPRESS "$scheme$request_method$host$1";
                }       

                # cache.appcache, your document html and data
                location ~* \.(?:manifest|appcache|html?|xml|json)$ {
                        expires -1;
                }

                # Feed
                location ~* \.(?:rss|atom)$ {
                        add_header Pragma public;
                        add_header Cache-Control "public";
                        expires 4h;
                }

                # Media: images, icons, video, audio, HTC, CSS, Javascript, and WebFonts
                location ~ \.(css|js|htc|ogg|ogv|svg|svgz|eot|otf|woff|mp4|ttf|rss|atom|jpg|jpeg|gif|png|ico|zip|tgz|gz|rar|bz2|doc|xls|exe|ppt|tar|mid|midi|wav|bmp|rtf)$ {
                        access_log off;
                        log_not_found off;
                        add_header Pragma public;
                        add_header Cache-Control "public";
                        expires max;
                }
        }

        server {
                listen 127.0.0.1:80;
                server_name 127.0.0.1;
                location /nginx_status {
                        stub_status on;
                        access_log off;
                        allow 127.0.0.1;
                        deny all;
                }
        }
}

I've also done curl -I 127.1and the response is the following:

HTTP/1.1 200 OK
Server: nginx
Date: Wed, 11 Jan 2017 14:09:08 GMT
Content-Type: text/html
Content-Length: 612
Last-Modified: Tue, 26 Apr 2016 13:31:19 GMT
Connection: keep-alive
Vary: Accept-Encoding
ETag: "571f6da7-264"
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
X-UA-Compatible: IE=Edge
Accept-Ranges: bytes

# Test 1: See what happens when you delete 301 directive for other URLs

1. I've redirected one non-existent test URL to a real URL via Nginx conf. I reloaded Nginx. I visited the test URL and by doing that I cached the redirection into the browser and thus probably at CloudFlare and FastCGI Cache (that is if FastCGI even caches redirection). After that I curl-ed that test URL from SSH and got 301 response. Now I got one example of a solid redirection.

2. Then I removed the redirection from Nginx conf, reloaded Nginx and visited the test URL via browser. Redirection is still there. However when I curl-ed the test URL via SSH I got no redirection but proper 404 Not Found status. Redirection at Nginx server level was gone after removing the redirection code from the nginx conf.

# Test 2: Change public IP redirection to 302

I moved back the IP-to-domain redirection directive to Nginx conf but this time with return 302.

# IP to domain redirect
server {
        # tell on which port this server listens
        listen [::]:80;
        listen 80;

        # listen on the IP
        server_name xx.xx.xx.xxx;

        # and redirect to the domain
        return 302 $scheme://example.com$request_uri;
}

I've reloaded Nginx and curl-ed the IP. I was surprised to see 302 Moved Temporarily response:

HTTP/1.1 302 Moved Temporarily
Server: nginx
Date: Thu, 12 Jan 2017 14:46:49 GMT
Content-Type: text/html
Content-Length: 154
Connection: keep-alive
Location: http://example.com/
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
X-UA-Compatible: IE=Edge

After removing this very same directive altogether and reloading Nginx the response via CURL is back to 301 Moved Permanently.

This test tells me that the problem is probably not a caching issue. It MUST be some other rule somewhere in the conf files or the server's architecture in general that makes the public IP to 301 redirect to the domain.

# Test 3: Put site into development mode at CloudFlare and purge all the cache.

I've put the site into development mode at CF and I've purged everything. I've done curl from both the local machine on the same network and from my PC at home. The response in both cases was 301.

# Test 4: Check if ipv6 redirects too.

I've done curl to the server's public ipv6.

curl -I -g "http://[server:public:ipv6:here]/"

The response is 301 Moved Permanently pointing to the website's domain which is even more puzzling and almost unbelievable. This ipv6 redirection was never ever set on my part to begin with, unless the redirection server block posted at the beginning of this question encompasses ipv6 redirection too.

Any thoughts?

Answer 1

Due to the result of curl 127.1 that you tested and mentioned in your question, you are getting 200 HTTP Status Code - whilst using curl locally, and by using 127.1 instead of your Public IP Address.

The reason you are getting 200 Status Code locally, is that you could've successfully removed the 301 redirection line and your NGINX is working fine after restarting the daemon. So don't you worry about NGINX, the problem is something else caching the redirection.

How to find out what is causing the issue, and where it is from ?

There might be two things causing problem - depending on if you are using your own servers or you are using VPSs provided by somewhere like GoDaddy, OVH, or etc. :

1. Some thing in your own architecture is caching odds and sods
2. Something in your VPS Provider is caching odds and sods

---

If you have multiple servers, you can use the steps below to find out which one of the told possibilities is the reason, and if you do not, I recommend you contact with the provider or if you are not using VPS at all and you're using your own servers, check if you are using something like Proxy, Cloud services, Caching Server, or whatsoever that might cache stuff - (Consider that the reason might be caused by both of the told possibilities) ↴

1. Login via SSH to one of the other servers in the same VLAN that you own, for instance your Storage server.

2. Guess that your Storage is using 192.168.1.122 and your WebServer is using 192.168.1.125 as their Private IP Address.

3. From Storage Server, use curl 192.168.1.125 to see if you are still getting 200 or you are getting 301.

4. If 301, this is something in your architecture that is caching stuff, it might be some service or some other servers you may own like a LoadBalancer.

5. And if you are getting 200, this is not your architecture, so contact your provider.

What is the Reason 301 Status Code is causing problem ?

301 Redirection is called PERMANENT Redirection! So everything receiving it - including most of the browsers available today - will consider it PERMANENT until it gets expired! This is an expected thing and nothing is acting weird, this is PERMANENT THING's nature!

301 is a one-way street.

CONCLUSION :

NEVER, NEVER EVER use permanent stuff IF you are not 100% - personally would say 500% - sure! Somewhere like Google, like Apple, like RedHat, and other big big companies and manufacturers can use 301 since they know what they are doing, and they own most of the technologies everyone would use, and ALSO, they know how their plan will be going to be in the next 5 years ahead. You can use 301 for simple redirecting that you may use every day like what you have used it for (www to non-www). But never use it permanently until you get sure about the next five years of your project - or at least 2 years.

I hope you can fix your problem soon, and I hope I could help you with the answer.

Cheers mate.

Answer 2

you can use permanently redirect every time. for un permanenly or remove redirect, just remove the line of redirect statement.

if you have domain like this ci.com and will redirect to laravel.com

and you remove redirect statement, u open the browser you still open redirect page. that because browser cached.

try using curl and it will show default domain because curl not cached.

Conclusion remove cache on browser