Why I cannot connect to my website in HTTP using curl?

Question

I have a web site @ www.eshipp.com, all the domain is secured by SSL so any HTTP traffic is redirected by NGINX to HTTPS equivalent URL.

However it seems that some users are not redirected and then getting a "not accessible" http error.

As I don't have the error myself on the browser it's hard to debug, but I luckily found that using Curl make this error happen :

$ curl -v http://eshipp.com/
* Hostname was NOT found in DNS cache
*   Trying 198.199.96.110...
* connect to 198.199.96.110 port 80 failed: Connection timed out
* Failed to connect to eshipp.com port 80: Connection timed out
* Closing connection 0
curl: (7) Failed to connect to eshipp.com port 80: Connection timed out

The command above does not work, but the one below works :

curl -v https://eshipp.com/

* Hostname was NOT found in DNS cache
*   Trying 198.199.96.110...
* Connected to eshipp.com (198.199.96.110) port 443 (#0)
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* SSLv3, TLS handshake, Client hello (1):
* SSLv3, TLS handshake, Server hello (2):
* SSLv3, TLS handshake, CERT (11):
* SSLv3, TLS handshake, Server key exchange (12):
* SSLv3, TLS handshake, Server finished (14):
* SSLv3, TLS handshake, Client key exchange (16):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSLv3, TLS change cipher, Client hello (1):
* SSLv3, TLS handshake, Finished (20):
* SSL connection using ECDHE-RSA-AES128-GCM-SHA256
* Server certificate:
*        subject: OU=Domain Control Validated; OU=Gandi Standard SSL; CN=eshipp.com
*        start date: 2015-07-13 00:00:00 GMT
*        expire date: 2016-07-13 23:59:59 GMT
*        subjectAltName: eshipp.com matched
*        issuer: C=FR; ST=Paris; L=Paris; O=Gandi; CN=Gandi Standard SSL CA 2
*        SSL certificate verify ok.
> GET / HTTP/1.1
> User-Agent: curl/7.35.0
> Host: eshipp.com
> Accept: */*
> 
< HTTP/1.1 200 OK
* Server nginx is not blacklisted
< Server: nginx
< Date: Sat, 08 Aug 2015 22:10:56 GMT
< Content-Type: text/html; charset=utf-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< Vary: Accept-Encoding
< Strict-Transport-Security: max-age=31536000; includeSubdomains
< 
<!DOCTYPE html>

Here is the NGINX config :

# HTTP
server {
        listen 80 default_server;
        listen [::]:80 default_server;
        server_name www.eshipp.com eshipp.com;
        return 301 https://$server_name$request_uri;
}

# HTTPS
server {
        listen 443 ssl spdy;
        server_name www.eshipp.com eshipp.com;
        keepalive_timeout 10m;

        # Certificats SSL
        ssl_certificate /etc/nginx/ssl/eshipp.com.crt;
        ssl_certificate_key /etc/nginx/ssl/eshipp.com.key;

        # Amélioration des performances SSL
        ssl_stapling on;
        ssl_session_cache shared:SSL:10m;
        ssl_session_timeout 10m;

        # Amélioration de la sécurité SSL
        ssl_prefer_server_ciphers on;
        ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers 'ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SH$

        # Active le HSTS to avoid SSL stripping
        add_header Strict-Transport-Security "max-age=31536000; includeSubdomains";

        # If your application is not compatible with IE <= 10, this will redirect visitors to a page advising a browser update
        # This works because IE 11 does not present itself as MSIE anymore
        if ($http_user_agent ~ "MSIE" ) {
                return 303 https://browser-update.org/update.html;
        }

        location / {
                proxy_pass http://127.0.0.1:3000;
                proxy_http_version 1.1;
                proxy_set_header Upgrade $http_upgrade; # allow websockets
                proxy_set_header Connection $connection_upgrade;
                proxy_set_header X-Forwarded-For $remote_addr; # preserve client IP

                # this setting allows the browser to cache the application in a way compatible with Meteor
                # on every application update the name of CSS and JS file is different, so they can be cache infinitely (here: 30 da$
                # the root path (/) MUST NOT be cached
                if ($uri != '/') {
                        expires 30d;
                }
        }
}

EDIT

Okay I found that, the problem comes from a rule in the Firewall but I don't know which one is blocking HTTP traffic, could someone help me ?

# INPUT rules
iptables -A INPUT -i lo -j ACCEPT
iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -m limit --limit 50/second --limit-burst 50 -j ACCEPT
iptables -A INPUT -p icmp -m limit --limit 1/sec -j ACCEPT
iptables -A INPUT -p tcp --dport 22 -j ACCEPT
iptables -A INPUT -p tcp --dport 80 -m state --state NEW -m limit --limit 50/minute --limit-burst 200 -j ACCEPT
iptables -A INPUT -p tcp --dport 443 -m state --state NEW -m limit --limit 50/minute --limit-burst 200 -j ACCEPT
iptables -A INPUT -j LOG
iptables -A INPUT -j DROP

EDIT 2

I had to add this rule to unlock the traffic, but it's a problem because I don't want to expose the port 3000 on Internet, it's only used in local by a NodeJS app.. I tried with -i lo but it does not work..

iptables -A INPUT -i eth0 -p tcp --dport 3000 -j ACCEPT

Where's the rest of the curl output from the failing run? It should be reporting some sort of error when it fails. — womble, Aug 08 '15 at 22:48
@womble, there's no error, you can test yourself, it just blocks on Trying 198.199.96.110... — Karl.S, Aug 08 '15 at 22:53
@womble, I havn't wait enough, Shi added the missing error : it's connection timeout, so how do I fix that in Nginx ? — Karl.S, Aug 08 '15 at 23:07

Paul · Accepted Answer · 2015-08-09T07:07:47.620

The Reason your CURL command is failing, is because you're not following redirects: http://curl.haxx.se/docs/faq.html#How_do_I_tell_curl_to_follow_HTT

Try: curl -Lv http://eshipp.com/

Looking at your IP table rules, you're rate limiting connections to 443 and 80 pretty aggressively. That might be the reason some of your clients aren't able to collect.

Also, in these 2 rules, you're only allowing NEW connection (and rate limiting them) I suggest you change them to include ESTABLISHED as well as NEW.

iptables -A INPUT -p tcp --dport 80 -m state --state NEW -m limit --limit 50/minute --limit-burst 200 -j ACCEPT iptables -A INPUT -p tcp --dport 443 -m state --state NEW -m limit --limit 50/minute --limit-burst 200 -j ACCEPT

change --state NEW to be --state NEW,ESTABLISHED

FWIW: I was able to curl the URI just fine

And Additional FYI, SPDY support is being remove by google as it's functionality is built into HTTP/2 http://blog.chromium.org/2015/02/hello-http2-goodbye-spdy-http-is_9.html

Why I cannot connect to my website in HTTP using curl?

1 Answers1