0

I'm trying to set up a Varnish Cache for Magento using reverse proxy (on Apache) to get https cached. I can't get it to work. So far I've got ERR_TOO_MANY_REDIRECTS and NET::ERR_CERT_AUTHORITY_INVALID while setting up reverse proxy (and 503 when trying to set up Varnish cache to respond directly on http/80).

Technology stack is as follows:

  • Dedicated server (insane specs)
  • Centos
  • Apache
  • Direct Admin (yeah, I know)
  • Magento

My understanding of Varnish Cache is as follows:

  1. HTTP: User->Varnish[:80]->Apache[:8080]
  2. HTTPS: User->Apache[:443]->Varnish[:80]->Apache[:8080] (reverse proxy).

Magento settings are as follow:

web/seo/use_rewrites - 1
web/unsecure/base_url - https://www.{domain}.com/  //Previously it didn't work when I set this up with http. That's why it's https. Tested with http and no difference now.
web/unsecure/base_link_url - {{unsecure_base_url}}
web/unsecure/base_static_url -
web/unsecure/base_media_url -
web/secure/base_url - https://www.{domain}.com/
web/secure/use_in_frontend - 1  //tested 0, no difference
web/secure/use_in_adminhtml - 1 //tested 0, no difference
web/secure/base_link_url - {{secure_base_url}}
web/secure/base_static_url -
web/secure/base_media_url -
web/secure/offloader_header - SSL_OFFLOADED
web/url/use_store - 0
web/url/redirect_to_base - 1 //tested 0, infinite loading
web/default/front - cms
web/default/cms_home_page - home
web/default/no_route - cms/noroute/index
web/default/cms_no_route - no-route
web/default/cms_no_cookies - enable-cookies
web/default/show_cms_breadcrumbs - 1
web/cookie/cookie_lifetime - 3600
web/cookie/cookie_path -
web/cookie/cookie_domain -
web/cookie/cookie_httponly - 0
web/cookie/cookie_restriction - 0
web/session/use_remote_addr - 0
web/session/use_http_via - 0
web/session/use_http_x_forwarded_for - 0
web/session/use_http_user_agent - 0
web/session/use_frontend_sid - 1
web/browser_capabilities/cookies - 1
web/browser_capabilities/javascript - 1
web/browser_capabilities/local_storage - 0
web/default_layouts/default_product_layout -
web/default_layouts/default_category_layout -
web/default_layouts/default_cms_layout - 1column

httpd.conf:

    ServerRoot "/etc/httpd"
    Listen 8080
    <IfModule unixd_module>
    User apache
    Group apache
    </IfModule>
    LoadModule php7_module        /usr/lib/apache/libphp7.so
    Include /etc/httpd/conf/extra/httpd-phpmodules.conf
    DocumentRoot "/home/admin/domains/{domain}.com/public_html"
    <IfModule dir_module>
        Include /etc/httpd/conf/extra/httpd-directoryindex.conf
    </IfModule>
    <Files ".ht*">
        Require all denied
    </Files>
    <Files ".user.ini">
        Require all denied
    </Files>
    LogLevel warn

    <IfModule log_config_module>
        #replace %b with %O for more accurate logging
        <IfModule mod_logio.c>
          LogFormat "%a %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined
          LogFormat "%a %l %u %t \"%r\" %>s %O" common
          LogFormat "%O %I" bytes

          LogFormat "%a %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
        </IfModule>

        CustomLog /var/log/httpd/access_log common
    </IfModule>

    <IfModule alias_module>
        # Include some DirectAdmin alias
        Include conf/extra/httpd-alias.conf
    </IfModule>
    <IfModule mime_module>
        TypesConfig conf/mime.types
    AddType application/x-gzip .tgz
    AddEncoding x-compress .Z
    AddEncoding x-gzip .gz .tgz
    AddType application/x-compress .Z
    AddType application/x-gzip .gz .tgz
    AddHandler cgi-script .cgi
    AddHandler type-map var
    AddType text/html .shtml
    AddOutputFilter INCLUDES .shtml
        AddType video/x-ms-asf .avi
        AddType video/mpeg .mpg
        AddType video/mpeg .mpeg
        AddType video/quicktime .mov
        AddType video/x-ms-wmv .wmv
    </IfModule>

    incldes x20
    Include conf/extra/directadmin-vhosts.conf
    <IfModule ssl_module>
    SSLRandomSeed startup builtin
    SSLRandomSeed connect builtin
    </IfModule>


!!!-----------------------------And here comes DirectAdmin httpd.conf:----------------------------!!!



[root@  ~]# cat /usr/local/directadmin/data/users/admin/httpd.conf
<Directory "/home/admin/public_html">
        <IfModule mod_ruid2.c>
                RMode config
                RUidGid admin admin
                #RGroups apache access
                RGroups @none
        </IfModule>
        <IfModule mod_fcgid.c>
                SuexecUserGroup admin admin
        </IfModule>
                php_admin_flag engine ON
                php_admin_value sendmail_path '/usr/sbin/sendmail -t -i -f admin@{domain}.com'
                php_admin_value mail.log /home/admin/.php/php-mail.log
                php_admin_value open_basedir /home/admin/:/tmp:/var/tmp:/opt/alt/php72/usr/share/pear/:/dev/urandom:/usr/local/lib/php/:/usr/local/php72/lib/php/
</Directory>

<VirtualHost {ip}:8080 >
        ServerName www.{domain}.com
#        ServerAlias www.{domain}.com {domain}.com //tested "on" - see errors
        ServerAdmin webmaster@{domain}.com
        DocumentRoot /home/admin/domains/{domain}.com/public_html
#        UseCanonicalName OFF //tested uncommenting - see errors
        <IfModule !mod_ruid2.c>
                SuexecUserGroup admin admin
        </IfModule>
        <IfModule mod_ruid2.c>
                RMode config
                RUidGid admin admin
                #RGroups apache access
                RGroups @none
        </IfModule>
        CustomLog /var/log/httpd/domains/{domain}.com.bytes bytes
        CustomLog /var/log/httpd/domains/{domain}.com.log combined
        ErrorLog /var/log/httpd/domains/{domain}.com.error.log
        <Directory /home/admin/domains/{domain}.com/public_html>
                AllowOverride AuthConfig FileInfo Indexes Limit Options=Indexes,IncludesNOEXEC,MultiViews,SymLinksIfOwnerMatch,FollowSymLinks,None
                Options -ExecCGI -Includes +IncludesNOEXEC
                php_admin_flag engine ON
                php_admin_value sendmail_path '/usr/sbin/sendmail -t -i -f admin@{domain}.com'
                php_admin_value mail.log /home/admin/.php/php-mail.log
                php_admin_value open_basedir /home/admin/:/tmp:/var/tmp:/opt/alt/php72/usr/share/pear/:/dev/urandom:/usr/local/lib/php/:/usr/local/php72/lib/php/
        </Directory>
</VirtualHost>

<VirtualHost {ip}:443 >
        SSLEngine on
        SSLCertificateFile /usr/local/directadmin/data/users/admin/domains/{domain}.com.cert.combined
        SSLCertificateKeyFile /usr/local/directadmin/data/users/admin/domains/{domain}.com.key
        SSLCACertificateFile /usr/local/directadmin/data/users/admin/domains/{domain}.com.cacert
        ServerName www.{domain}.com


    ProxyPreserveHost On
    ProxyPass / http://127.0.0.1:80/
    ProxyPassReverse / http://127.0.0.1:80/

    RequestHeader set X-Forwarded-Port "443"
    RequestHeader set X-Forwarded-Proto "https"

        ServerAlias www.{domain}.com {domain}.com
#       ServerAdmin webmaster@{domain}.com
        DocumentRoot /home/admin/domains/{domain}.com/private_html
#       UseCanonicalName OFF
#       <IfModule !mod_ruid2.c>
#               SuexecUserGroup admin admin
#       </IfModule>
#       <IfModule mod_ruid2.c>
#               RMode config
#               RUidGid admin admin
#               #RGroups apache access
#               RGroups @none
#       </IfModule>
        CustomLog /var/log/httpd/domains/{domain}.com.bytes bytes
        CustomLog /var/log/httpd/domains/{domain}.com.log combined
        ErrorLog /var/log/httpd/domains/{domain}.com.error.log
#       <Directory /home/admin/domains/{domain}.com/private_html>
#               AllowOverride AuthConfig FileInfo Indexes Limit Options=Indexes,IncludesNOEXEC,MultiViews,SymLinksIfOwnerMatch,FollowSymLinks,None
#               Options -ExecCGI -Includes +IncludesNOEXEC
#               php_admin_flag engine ON
#               php_admin_value sendmail_path '/usr/sbin/sendmail -t -i -f admin@{domain}.com'
#               php_admin_value mail.log /home/admin/.php/php-mail.log
#               php_admin_value open_basedir /home/admin/:/tmp:/var/tmp:/opt/alt/php72/usr/share/pear/:/dev/urandom:/usr/local/lib/php/:/usr/local/php72/lib/php/
#       </Directory>
</VirtualHost>

PLEASE NOTE: I just want to get this to work, I will transfer all the changes to a seperate file that will not get overwritten by DA.

netstat:

[root@  ~]# netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 127.0.0.1:953           0.0.0.0:*               LISTEN      3037/named
tcp        0      0 0.0.0.0:25              0.0.0.0:*               LISTEN      2203/exim
tcp        0      0 0.0.0.0:993             0.0.0.0:*               LISTEN      2208/dovecot
tcp        0      0 127.0.0.1:6082          0.0.0.0:*               LISTEN      19248/varnishd
tcp        0      0 0.0.0.0:995             0.0.0.0:*               LISTEN      2208/dovecot
tcp        0      0 0.0.0.0:587             0.0.0.0:*               LISTEN      2203/exim
tcp        0      0 0.0.0.0:110             0.0.0.0:*               LISTEN      2208/dovecot
tcp        0      0 0.0.0.0:143             0.0.0.0:*               LISTEN      2208/dovecot
tcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      1/systemd
***tcp     0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      19248/varnishd
tcp        0      0 0.0.0.0:465             0.0.0.0:*               LISTEN      2203/exim
tcp        0      0 {ip}:53                 0.0.0.0:*               LISTEN      3037/named
tcp        0      0 127.0.0.1:53            0.0.0.0:*               LISTEN      3037/named
tcp        0      0 0.0.0.0:21              0.0.0.0:*               LISTEN      2226/pure-ftpd
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      2201/sshd
tcp6       0      0 ::1:953                 :::*                    LISTEN      3037/named
***tcp6    0      0 :::443                  :::*                    LISTEN      19143/httpd
tcp6       0      0 :::3306                 :::*                    LISTEN      2860/mysqld
tcp6       0      0 :::2222                 :::*                    LISTEN      19694/directadmin
tcp6       0      0 :::111                  :::*                    LISTEN      1/systemd
***tcp6    0      0 :::80                   :::*                    LISTEN      19248/varnishd
***tcp6    0      0 :::8080                 :::*                    LISTEN      19143/httpd
tcp6       0      0 :::53                   :::*                    LISTEN      3037/named
tcp6       0      0 :::21                   :::*                    LISTEN      2226/pure-ftpd
tcp6       0      0 :::22                   :::*                    LISTEN      2201/sshd
udp        0      0 {ip}:53                 0.0.0.0:*                           3037/named
udp        0      0 127.0.0.1:53            0.0.0.0:*                           3037/named
udp        0      0 0.0.0.0:111             0.0.0.0:*                           1/systemd
udp        0      0 127.0.0.1:323           0.0.0.0:*                           1341/chronyd
udp        0      0 0.0.0.0:659             0.0.0.0:*                           1356/rpcbind
udp6       0      0 :::53                   :::*                                3037/named
udp6       0      0 :::111                  :::*                                1/systemd
udp6       0      0 ::1:323                 :::*                                1341/chronyd
udp6       0      0 :::659                  :::*                                1356/rpcbind

Now the problems (with above settings):

  • https://{domain}.com/ causes NET::ERR_CERT_AUTHORITY_INVALID
  • https://www.{domain}.com/ causes ERR_TOO_MANY_REDIRECTS
  • https://{domain}.com/test.php causes NET::ERR_CERT_AUTHORITY_INVALID
  • https://www.{domain}.com/test.php works fine (tho not served by Varnish)

with server alias on:

  • https://{domain}.com/ infinite loading
  • https://www.{domain}.com/ infinite loading
  • https://{domain}.com/test.php works fine (tho not served by Varnish)
  • https://www.{domain}.com/test.php works fine (tho not served by Varnish)

I don't think it makes sense to paste every single combination of settings and errors thus I would like to ask you if you can see where the problem might be (other than the use of DA).

Since I think that it is before it gets to varnish.params and varnish.vcl I didn't paste those files. I somewhat suspect .htaccess but you will tell me what other files I should paste.

Sergiusz Olszewski
  • 1

1 Answers1

0

I noticed you do set RequestHeader set X-Forwarded-Proto "https" in your Apache configuration in the port 443 vhost.

This should announce to your backend that although the connection is done in plain HTTP, it was originally terminated as an HTTPS connection.

Currently your Magento configuration uses SSL_OFFLOADED as the ofloading header, however your Apache config uses X-Forwarded-Proto.

Can you try updating your Magento config as follows?

web/secure/offloader_header - X-FORWARDED-PROTO

This should help Magento look for the right header and realize it doesn't need to redirect back to HTTPS, since the page was already terminated as HTTPS.

Thijs Feryn
  • 3,982
  • 1
  • 5
  • 10