1

On our Apache web server cluster, we have mod_pagespeed setup with memcached and domain sharding to cookieless domain.

Our web servers sits behind a loadbalancer with non-sticky sessions, and SSL offloading. With SSL offloading the loadbalancer requests web servers on regular http always, but sends the respective X-Forwarded headers. The Apache web servers are not configured for SSL/https.

Everything works great with mod_pagespeed for regular http, but not for https. Accessing website through https though:

  • CSS gets tagged, combined and so forth
  • JS does NOT get tagged and combined
  • Some media (png, jpeg, gif) are tagged, some are not
  • Resources are not directed to the sharded domains

I have attached my various configurations, note that mod_pagespeed is enabled on individual vhost - I have stripped non-relevant vhosts from the config below.

In the config below I have changed our primary domain to example.com and our cookieless domain to nocookie.com

Apache

Version     : 2.2.15
Release     : 39.el6.centos

mod_pagespeed version:

1.9.32.14

/etc/httpd/conf.d/vhost.conf

## Virtual Hosts
#

<Directory "/hostroot/www/vhost/*/httpdocs">
    Order allow,deny
    Allow from all
    Options FollowSymLinks
    AllowOverride none
</Directory>

NameVirtualHost *:80

<VirtualHost _default_:80>
    ServerAdmin noreply@example.com
    DocumentRoot "/hostroot/www/vhost/default/httpdocs"
    ServerName webserver.example.com

    ErrorLog "/var/log/httpd/vhost/default/error_log"
#    CustomLog "/var/log/httpd/vhost/default/access_log" common
    CustomLog "/var/log/httpd/vhost/default/access_log" combined env=!forwarded
    CustomLog "/var/log/httpd/vhost/default/access_log" proxy env=forwarded
</VirtualHost>



<VirtualHost *:80>
    ServerAdmin noreply@example.com
    ServerName example.com
    ServerAlias www.example.com
    ServerAlias www1.example.com
    ServerAlias www2.example.com
    ServerAlias www3.example.com
    ServerAlias www4.example.com
    ServerAlias www5.example.com
    ServerAlias www6.example.com
    ServerAlias www7.example.com
    ServerAlias www8.example.com
    ServerAlias www9.example.com
    ServerAlias m.example.com
    ServerAlias wap.example.com
    ServerAlias mobil.example.com
    ServerAlias mob.example.com
    ServerAlias app.example.com
    ServerAlias ap.example.com

    DocumentRoot "/hostroot/www/vhost/example_com/httpdocs/public/default"
    DirectoryIndex index.php

    # This should be omitted in the production environment
    SetEnv APPLICATION_DOMAIN www.example.com
#    SetEnv APPLICATION_ENV production
#    SetEnv APPLICATION_LAYOUT default

#    SetEnvIf X-Forwarded-Proto https HTTPS=on

    ErrorLog "/var/log/httpd/vhost/example_com/error_log"
#    CustomLog "/var/log/httpd/vhost/example_com/access_log" common
    CustomLog "/var/log/httpd/vhost/example_com/access_log" combined env=!forwarded
    CustomLog "/var/log/httpd/vhost/example_com/access_log" proxy env=forwarded

    ModPagespeed on
    ModPagespeedDomain www.example.com
    ModPagespeedLoadFromFileMatch "^(http|https)://www.example.com/(img|lib|css|swg)/" "/hostroot/www/vhost/example_com/httpdocs/public/default/\\2/"
    ModPagespeedShardDomain nocookie.com s1.nocookie.com,s2.nocookie.com,s3.nocookie.com
    ModPagespeedMapRewriteDomain nocookie.com www.example.com
    ModPagespeedRespectXForwardedProto on
#    ModPagespeedEnableFilters insert_image_dimensions
    ModPagespeedDisableFilters convert_png_to_jpeg,inline_images,convert_jpeg_to_webp
    Header unset ETag
    FileETag None

    # Enable expirations
    ExpiresActive On
    ExpiresDefault "access plus 1 month"
    ExpiresByType image/x-icon "access plus 1 year"
    ExpiresByType image/gif "access plus 1 month"
    ExpiresByType image/png "access plus 1 month"
    ExpiresByType image/jpg "access plus 1 month"
    ExpiresByType image/jpeg "access plus 1 month"
    ExpiresByType text/css "access plus 1 month"
    ExpiresByType application/javascript "access plus 1 year"

    <Directory "/hostroot/www/vhost/example_com/httpdocs/public/default">
        RewriteEngine On

        # Redirect to www.example.com if no-sub or sub is not www, stop further rewrites
        RewriteCond %{HTTP_HOST} !^www([0-9]*)\.example\.com [NC]
        RewriteRule ^(.*)$ http://www.example.com/$1 [L,R=301]

        # If actual resource serve it and stop further rewrites
        RewriteCond %{REQUEST_FILENAME} -s [OR]
        RewriteCond %{REQUEST_FILENAME} -l [OR]
        RewriteCond %{REQUEST_FILENAME} -d [OR]
        RewriteCond %{REQUEST_FILENAME} \.(gif|jpe?g|png|js|css|swf|php|ico|txt|pdf|xml|woff|ttf|eot|svg)$
        RewriteRule ^.*$ - [NC,L]

        # Remove trailing slash, set permanent redirect and stop further rewrites
        # Condition is only needed, if directories is not handled by previous rewrites
        #RewriteCond %{REQUEST_FILENAME} !-d
#        RewriteRule ^(.*)/$ /$1 [R=301,L]

        # Bootstrap to index.php
        RewriteRule ^.*$ index.php [NC,L]
    </Directory>
    <IfModule mod_alias.c>
        Alias /apple-touch-icon-57x57.png /hostroot/www/vhost/example_com/favicons/apple-touch-icon-57x57.png
        Alias /apple-touch-icon-60x60.png /hostroot/www/vhost/example_com/favicons/apple-touch-icon-60x60.png
        Alias /apple-touch-icon-72x72.png /hostroot/www/vhost/example_com/favicons/apple-touch-icon-72x72.png
        Alias /apple-touch-icon-76x76.png /hostroot/www/vhost/example_com/favicons/apple-touch-icon-76x76.png
        Alias /apple-touch-icon-114x114.png /hostroot/www/vhost/example_com/favicons/apple-touch-icon-114x114.png
        Alias /apple-touch-icon-120x120.png /hostroot/www/vhost/example_com/favicons/apple-touch-icon-120x120.png
        Alias /apple-touch-icon-144x144.png /hostroot/www/vhost/example_com/favicons/apple-touch-icon-144x144.png
        Alias /apple-touch-icon-152x152.png /hostroot/www/vhost/example_com/favicons/apple-touch-icon-152x152.png
        Alias /apple-touch-icon-180x180.png /hostroot/www/vhost/example_com/favicons/apple-touch-icon-180x180.png
        Alias /apple-touch-icon-precomposed.png /hostroot/www/vhost/example_com/favicons/apple-touch-icon-precomposed.png
        Alias /apple-touch-icon.png /hostroot/www/vhost/example_com/favicons/apple-touch-icon.png
        Alias /browserconfig.xml /hostroot/www/vhost/example_com/favicons/browserconfig.xml
        Alias /crossdomain.xml /hostroot/www/vhost/example_com/favicons/crossdomain.xml
        Alias /favicon-16x16.png /hostroot/www/vhost/example_com/favicons/favicon-16x16.png
        Alias /favicon-32x32.png /hostroot/www/vhost/example_com/favicons/favicon-32x32.png
        Alias /favicon-96x96.png /hostroot/www/vhost/example_com/favicons/favicon-96x96.png
        Alias /favicon-160x160.png /hostroot/www/vhost/example_com/favicons/favicon-160x160.png
        Alias /favicon-192x192.png /hostroot/www/vhost/example_com/favicons/favicon-192x192.png
        Alias /favicon.ico /hostroot/www/vhost/example_com/favicons/favicon.ico
        Alias /mstile-70x70.png /hostroot/www/vhost/example_com/favicons/mstile-70x70.png
        Alias /mstile-144x144.png /hostroot/www/vhost/example_com/favicons/mstile-144x144.png
        Alias /mstile-150x150.png /hostroot/www/vhost/example_com/favicons/mstile-150x150.png
        Alias /mstile-310x150.png /hostroot/www/vhost/example_com/favicons/mstile-310x150.png
        Alias /mstile-310x310.png /hostroot/www/vhost/example_com/favicons/mstile-310x310.png
    </IfModule>
</VirtualHost>

<VirtualHost *:80>
    ServerAdmin noreply@example.com
    ServerName nocookie.com
    ServerAlias *.nocookie.com

    DocumentRoot "/hostroot/www/vhost/example_com/httpdocs/public/default"
    DirectoryIndex nocookie-index.htm

    ErrorLog "/var/log/httpd/vhost/nocookie_com/error_log"
#    CustomLog "/var/log/httpd/vhost/nocookie_com/access_log" common
    CustomLog "/var/log/httpd/vhost/nocookie_com/access_log" combined env=!forwarded
    CustomLog "/var/log/httpd/vhost/nocookie_com/access_log" proxy env=forwarded

    ModPagespeed on
    ModPagespeedDomain nocookie.com
    ModPagespeedLoadFromFileMatch "^(http|https)://s[0-9]+.nocookie.com/(img|lib|css|swg)/" "/hostroot/www/vhost/example_com/httpdocs/public/default/\\2/"
    ModPagespeedRespectXForwardedProto on
#    ModPagespeedEnableFilters insert_image_dimensions
    ModPagespeedDisableFilters convert_png_to_jpeg,inline_images
    Header unset ETag
    FileETag None

    # Enable expirations
    ExpiresActive On
    ExpiresDefault "access plus 1 month"
    ExpiresByType image/x-icon "access plus 1 year"
    ExpiresByType image/gif "access plus 1 month"
    ExpiresByType image/png "access plus 1 month"
    ExpiresByType image/jpg "access plus 1 month"
    ExpiresByType image/jpeg "access plus 1 month"
    ExpiresByType text/css "access plus 1 month"
    ExpiresByType application/javascript "access plus 1 year"

    # Disable PHP
    php_admin_flag engine off

    # CORS setting
    <FilesMatch "\.(ttf|otf|eot|woff)$">
        SetEnvIf Origin "^http(s)?://(.+\.)?(example|nocookie)\.com(:\d+)?$" AccessControlAllowOrigin=$0
        Header set Access-Control-Allow-Origin %{AccessControlAllowOrigin}e env=AccessControlAllowOrigin
    </FilesMatch>

    <Directory "/hostroot/www/vhost/example_com/httpdocs/public/default">
        RewriteEngine On

        # If actual resource serve it and stop further rewrites
        RewriteCond %{REQUEST_FILENAME} \.(gif|jpe?g|png|js|css|swf|php|ico|txt|pdf|xml|woff|ttf|eot|svg)$
        RewriteRule ^.*$ - [NC,L]

        # Bootstrap to index.php
        RewriteRule ^.*$ http://www.example.com/ [NC,L]
    </Directory>
    <IfModule mod_alias.c>
        Alias /apple-touch-icon-57x57.png /hostroot/www/vhost/example_com/favicons/apple-touch-icon-57x57.png
        Alias /apple-touch-icon-60x60.png /hostroot/www/vhost/example_com/favicons/apple-touch-icon-60x60.png
        Alias /apple-touch-icon-72x72.png /hostroot/www/vhost/example_com/favicons/apple-touch-icon-72x72.png
        Alias /apple-touch-icon-76x76.png /hostroot/www/vhost/example_com/favicons/apple-touch-icon-76x76.png
        Alias /apple-touch-icon-114x114.png /hostroot/www/vhost/example_com/favicons/apple-touch-icon-114x114.png
        Alias /apple-touch-icon-120x120.png /hostroot/www/vhost/example_com/favicons/apple-touch-icon-120x120.png
        Alias /apple-touch-icon-144x144.png /hostroot/www/vhost/example_com/favicons/apple-touch-icon-144x144.png
        Alias /apple-touch-icon-152x152.png /hostroot/www/vhost/example_com/favicons/apple-touch-icon-152x152.png
        Alias /apple-touch-icon-180x180.png /hostroot/www/vhost/example_com/favicons/apple-touch-icon-180x180.png
        Alias /apple-touch-icon-precomposed.png /hostroot/www/vhost/example_com/favicons/apple-touch-icon-precomposed.png
        Alias /apple-touch-icon.png /hostroot/www/vhost/example_com/favicons/apple-touch-icon.png
        Alias /browserconfig.xml /hostroot/www/vhost/example_com/favicons/browserconfig.xml
        Alias /crossdomain.xml /hostroot/www/vhost/example_com/favicons/crossdomain.xml
        Alias /favicon-16x16.png /hostroot/www/vhost/example_com/favicons/favicon-16x16.png
        Alias /favicon-32x32.png /hostroot/www/vhost/example_com/favicons/favicon-32x32.png
        Alias /favicon-96x96.png /hostroot/www/vhost/example_com/favicons/favicon-96x96.png
        Alias /favicon-160x160.png /hostroot/www/vhost/example_com/favicons/favicon-160x160.png
        Alias /favicon-192x192.png /hostroot/www/vhost/example_com/favicons/favicon-192x192.png
        Alias /favicon.ico /hostroot/www/vhost/example_com/favicons/favicon.ico
        Alias /mstile-70x70.png /hostroot/www/vhost/example_com/favicons/mstile-70x70.png
        Alias /mstile-144x144.png /hostroot/www/vhost/example_com/favicons/mstile-144x144.png
        Alias /mstile-150x150.png /hostroot/www/vhost/example_com/favicons/mstile-150x150.png
        Alias /mstile-310x150.png /hostroot/www/vhost/example_com/favicons/mstile-310x150.png
        Alias /mstile-310x310.png /hostroot/www/vhost/example_com/favicons/mstile-310x310.png
    </IfModule>
</VirtualHost>

/etc/httpd/conf.d/pagespeed.conf

<IfModule !mod_version.c>
  LoadModule version_module /usr/lib64/httpd/modules/mod_version.so
</IfModule>

<IfVersion < 2.4>
  LoadModule pagespeed_module /usr/lib64/httpd/modules/mod_pagespeed.so
</IfVersion>
<IfVersion >= 2.4.2>
  LoadModule pagespeed_module /usr/lib64/httpd/modules/mod_pagespeed_ap24.so
</IfVersion>
<IfModule !mod_deflate.c>
 LoadModule deflate_module /usr/lib64/httpd/modules/mod_deflate.so
</IfModule>
<IfModule pagespeed_module>
    ModPagespeed off
    ModPagespeedInheritVHostConfig on
    AddOutputFilterByType MOD_PAGESPEED_OUTPUT_FILTER text/html
    ModPagespeedFileCachePath "/var/cache/mod_pagespeed/"
    ModPagespeedLogDir "/var/log/pagespeed"
    # ModPagespeedSslCertDirectory "/etc/pki/tls/certs"
    # ModPagespeedSslCertFile /etc/pki/tls/cert.pem
    ModPagespeedMemcachedServers "10.220.30.70:11211,10.220.30.71:11211"
    ModPagespeedCreateSharedMemoryMetadataCache "/var/cache/mod_pagespeed/" 51200
    # ModPagespeedRewriteLevel PassThrough
    # ModPagespeedDisableFilters rewrite_images
    # ModPagespeedEnableFilters rewrite_javascript,rewrite_css
    # ModPagespeedEnableFilters collapse_whitespace,elide_attributes
    # ModPagespeedForbidFilters rewrite_images
    # ModPagespeedRewriteDeadlinePerFlushMs 10

    ModPagespeedXHeaderValue "enabled"
    ModPagespeedImplicitCacheTtlMs 2592000000

    # ModPagespeedDomain
    # ModPagespeedDownstreamCachePurgeLocationPrefix
    # ModPagespeedDownstreamCachePurgeMethod PURGE
    # ModPagespeedDownstreamCacheRewrittenPercentageThreshold 95
    # ModPagespeedDownstreamCacheRebeaconingKey
    # ModPagespeedFileCacheSizeKb          102400
    # ModPagespeedFileCacheCleanIntervalMs 3600000
    # ModPagespeedLRUCacheKbPerProcess     1024
    # ModPagespeedLRUCacheByteLimit        16384
    # ModPagespeedCssFlattenMaxBytes       102400
    # ModPagespeedCssInlineMaxBytes        2048
    # ModPagespeedCssImageInlineMaxBytes   0
    # ModPagespeedImageInlineMaxBytes      3072
    # ModPagespeedJsInlineMaxBytes         2048
    # ModPagespeedCssOutlineMinBytes       3000
    # ModPagespeedJsOutlineMinBytes        3000
    # ModPagespeedMaxCombinedCssBytes      -1
    # ModPagespeedMaxCombinedJsBytes       92160
    ModPagespeedFileCacheInodeLimit        500000
    # ModPagespeedImageMaxRewritesAtOnce      8
    # ModPagespeedNumRewriteThreads 4
    # ModPagespeedNumExpensiveRewriteThreads 4
    # ModPagespeedRewriteRandomDropPercentage 90
    # ModPagespeedJsPreserveURLs on
    # ModPagespeedImagePreserveURLs on
    # ModPagespeedCssPreserveURLs on
    # ModPagespeedFilters in_place_optimize_for_browser
    # ModPagespeedPrivateNotVaryForIE on
    # ModPagespeedImageRecompressionQuality 85
    # ModPagespeedJpegRecompressionQuality -1
    # ModPagespeedJpegRecompressionQualityForSmallScreens 70
    # ModPagespeedWebpRecompressionQuality 80
    # ModPagespeedWebpRecompressionQualityForSmallScreens 70
    # ModPagespeedWebpTimeoutMs 5000
    # ModPagespeedImageLimitOptimizedPercent 100
    # ModPagespeedImageLimitResizeAreaPercent 100
    # ModPagespeedMaxInlinedPreviewImagesIndex -1
    # ModPagespeedMinImageSizeLowResolutionBytes 3072
    # ModPagespeedMaxSegmentLength 250
    # ModPagespeedCombineAcrossPaths off
    # ModPagespeedAvoidRenamingIntrospectiveJavascript off
    # ModPagespeedEnableFilters canonicalize_javascript_libraries
    # ModPagespeedLibrary 43 1o978_K0_LNE5_ystNklf http://www.modpagespeed.com/rewrite_javascript.js
    # ModPagespeedLoadFromFile "http://example.com/static/" "/hostroot/www/static/"
    # ModPagespeedEnableFilters add_instrumentation
    # ModPagespeedReportUnloadTime on
    # ModPagespeedRespectVary on
    # ModPagespeedStatistics off
    <Location /pagespeed_admin>
        Order deny,allow
        Deny from all
        SetEnvIF X-Forwarded-For "10.10.200.2" AllowIP
        Allow from env=AllowIP
        Allow from 192.168.1.0/24
        SetHandler pagespeed_admin
    </Location>
    <Location /pagespeed_global_admin>
        Order deny,allow
        Deny from all
        SetEnvIF X-Forwarded-For "10.10.200.2" AllowIP
        Allow from env=AllowIP
        Allow from 192.168.1.0/24
        SetHandler pagespeed_global_admin
    </Location>
    ModPagespeedStatisticsLogging on
    ModPagespeedMessageBufferSize 100000
</IfModule>
Phliplip
  • 541
  • 8
  • 22

1 Answers1

0

Ok, by some trial and error I managed to solve this.

If have changed my ModPagespeedLoadFromFileMatch, this takes care off the non-tagged/combined JS and also the some media file that didn't match, ie. from the root of the domain.

ModPagespeedLoadFromFileMatch "^(http|https)://www\.example\.com(.*)(gif|jpe?g|png|js|css|swf|ico|txt|pdf|xml|woff|ttf|eot|svg)$" "/hostroot/www/vhost/example_com/httpdocs/public/default\\2\\3"

...

I have forced the domain sharding to https with the following updated rule, on the primary vhost:

ModPagespeedShardDomain nocookie.com https://s1.nocookie.com,https://s2.nocookie.com,https://s3.nocookie.com

...

I added (uncommented) following rule, on both vhosts:

SetEnvIf X-Forwarded-Proto https HTTPS=on

I removed the following rule, on both vhosts:

ModPagespeedRespectXForwardedProto on
Phliplip
  • 541
  • 8
  • 22