Getting http traffic to redirect to https

Question

I'm trying to figure out how to redirect http traffic to https. I running apache2 with varnish. Someone more knowledgeable than me has set this up for me on other websites. I've set up a new site to mimic the apache config for one of the known good sites that has redirection working but I'm not having any luck getting it to work for the new site; http traffic just pulls up as regular http traffic in the browser. Here is my apache config file:

<VirtualHost *>

ServerName thesite.org
ServerAlias *.thesite.org
Use letsencrypt-challenge thesite.org
Use ssl-upgrade thesite.org

# replace the brackets and string with the appropriate string as indicated
DocumentRoot /home/sites/wp_sites/thesite
ScriptAlias /cgi-bin/ /home/thesite/cgi-bin/
ScriptAlias /cgi /home/thesite/cgi-bin/

# not a typo, the user name is entered twice in the line below
SuexecUserGroup thesite thesite

# THIS HAS TO BE CORRECT. Double and triple check your typing here!
Use site_logging thesite

# These settings are the same for every site
ErrorDocument 404 /index.php
Use default_expire
Options IncludesNOEXEC FollowSymLinks
Use default_deny
Use default_php_fastcgi
</VirtualHost>


<VirtualHost *:443>
  ServerName thesite.org
  ServerAlias *.thesite.org
    Use letsencrypt-challenge thesite.org
    Use letsencrypt-ssl thesite.org
    Use ssl-proxy thesite.org
#ProxyPass / http://thesite.org/
#  ProxyPassReverse / http://thesite.org/
</VirtualHost>

And here are the relevant macros:

<Macro ssl-redirect $server_redirect>
    RewriteCond expr "%{HTTP_HOST} != '$server_redirect'"
    RewriteRule /?(.*) "https://$server_redirect/$1" [L,NE]
</Macro>
<Macro ssl-upgrade $server_upgrade>
    Use ssl-redirect "$server_upgrade"
    <If "! req_novary('X-Forwarded-Proto') =~ /https/">
        Redirect / "https://$server_upgrade/"
    </If>
</Macro>
<Macro ssl-proxy $server_proxy>
    Use letsencrypt-challenge "$server_proxy"
    Use letsencrypt-ssl "$server_proxy"
    Use ssl-redirect "$server_proxy"
    ProxyPass / "http://$server_proxy/"
    ProxyPassReverse / "http://$server_proxy/"
</Macro>
<Macro letsencrypt-ssl $server>
    SSLEngine On
    SSLProtocol all -SSLv2 -SSLv3
    SSLHonorCipherOrder on
    SSLCipherSuite "EECDH+ECDSA+AESGCM EECDH+aRSA+AESGCM EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256 EECDH+aRSA+RC4 EECDH EDH+aRSA RC4 !aNULL !eNULL !LOW !3DES !MD5 !EXP !PSK !SRP !DSS"
    SSLCertificateFile "/etc/letsencrypt/live/$server/fullchain.pem"
    SSLCertificateKeyFile "/etc/letsencrypt/live/$server/privkey.pem"
    RequestHeader set X-Forwarded-Proto "https"
</Macro>
<Macro letsencrypt-challenge $server>
  <Directory /etc/apache2/letsencrypt/$server/.well-known/acme-challenge>
    Require all granted
  </Directory>
  RewriteEngine On
  RewriteRule /.well-known/acme-challenge/(.*) /etc/apache2/letsencrypt/$server/.well-known/acme-challenge/$1 [L]
  </Macro>

Answer 1

The easiest way to achieve your goal is by adding the following code to the .htaccess file in your document root:

SetEnvIf X-Forwarded-Proto "https" HTTPS=on
Header append Vary: X-Forwarded-Proto

<IfModule mod_rewrite.c>
  RewriteEngine on
  RewriteCond %{HTTPS} !=on
  RewriteCond %{HTTP:X-Forwarded-Proto} !https [NC]
  RewriteRule ^ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]
</IfModule>

What makes these rewrite rules so specific, is the fact that they are protocol aware. This is important for Varnish.

Varnish & TLS

Varnish only processes HTTP traffic and communicates with Apache in plain HTTP. You'll probably terminate the TLS connection elsewhere.

So even if you're performing an HTTPS request, Varnish will do an HTTP request do Apache to fetch the content. This will always result in a 301 redirect, regardless of protocol that was used for the initial connection.

The end result is an endless loop, which we want to avoid. That's why it's good practice to expose an X-Forwarded-Proto header in the vhost of the system that terminates TLS.

Setting the X-Forwarded-Proto header

The following snippet of code seems to be responsible for TLS termination. However, the part where the HTTPS traffic is proxied to Varnish is commented out.

<VirtualHost *:443>
  ServerName thesite.org
  ServerAlias *.thesite.org
    Use letsencrypt-challenge thesite.org
    Use letsencrypt-ssl thesite.org
    Use ssl-proxy thesite.org
#ProxyPass / http://thesite.org/
#  ProxyPassReverse / http://thesite.org/
</VirtualHost>

You're probably going to have to fix this. The following expression should also be added, in order to expose the incoming request protocol:

RequestHeader set "X-Forwarded-Proto" expr=%{REQUEST_SCHEME}

Please make sure mod_headers is enabled if you want to set headers .

If an HTTPS connection is made, Apache will return the following response header:

X-Forwarded-Proto: https

The rewrite rule that I shared takes this into account and will only redirect if X-Forwarded-Proto is http or if the header is not set.

Creating a cache variation for Varnish

This conditional redirection strategy will have an impact on your cache: Varnish will store the 301 redirect and serve it to all clients.

The problem is that Varnish is not aware of the difference in protocol. Everything is plain HTTP for Varnish. Apache needs to inform Varnish that it should story a copy for the HTTP version and a copy for the HTTPS version.

This is called a cache variation and HTTP allows you to use the Vary header for that. In this case were going to vary on the different values that X-Forwarded-Proto has.

That's why you need Header append Vary: X-Forwarded-Proto in your .htacccess file.

Varnish will interpret this and create 2 versions of this page in cache. If you don't specify a Vary: X-Forwarded-Proto, Varnish will either always redirect to HTTPS, even for HTTPS request, or never, even for plain HTTP requests.

Conclusion

Make Apache protocol aware by leveraging X-Forwarded-Proto that should be set in the Apache vhost where you terminate TLS.

And finally, make Varnish protocol aware as well, by using the X-Forwarded-Proto header as a Vary value.