how to deny all unexcepted host headers 403 forbidden? Apache 2.4

Question

Hi I am trying to deny all unexcepted host headers to stop them from appearing in the mod pagespeed cache folder.

I tried to implement the following but on an apache 2.4 server -

you can lock down your server by specifying server names for all your virtual hosts and then adding a catchall block that gives 403-forbidden to everyone. For example, I just set ngxpagespeed.com to have:

   server {
        listen 80;
        location / {
           deny all;
        }
   }
   server {
        listen       80;
        server_name  ngxpagespeed.com www.ngxpagespeed.com;
        pagespeed on;
        ...
    }

This is what I tried adding to my apache server at the top of this file /etc/apache2/sites-enabled/000-default-le-ssl.conf

<VirtualHost *:80>
    ServerName catchall
    <Location />
        Require all denied
    </Location>
    <Location /var/www/html/>
        Require all denied
    </Location>
</VirtualHost>

When I try

wget --header="Host: example.com" http://demo.mysite.com

I got this response -

Resolving demo.mysite.com (demo.mysite.com)... 142.41.74.25
Connecting to demo.mysite.com (demo.mysite.com)|142.41.74.25|:443... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://demo.mysite.com/ [following]
--2022-09-05 10:57:46--  https://demo.mysite.com/
Reusing existing connection to demo.mysite.com:443.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://demo.mysite.com/ [following]
20 redirections exceeded.

Why is the output 301 and not 403 Forbidden?

How can I get it to 403 all unknown host headers?

This is what my mod_pagespeed cache folder looks like and I would like to stop it creating these random folders which aren't on my server -

/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/%2C2Fwp-content
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/%2C2Fwp-includes
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/%2C2Fwp-json
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/%2C3F3x%3D3x
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/%2C3Fa%3Dfetch%2C26content%3D%2C3Cphp%2C3Edie%2C28%2C40md5%2C28HelloThinkCMF%2C29%2C29%2C3C
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/%2C3Frest_route%3D
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/.git
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/.well-known
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/1phpmyadmin
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/2022
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/2phpmyadmin
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/_
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/_ignition
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/_phpMyAdmin
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/_phpmyadmin
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/_phpmyadmin_
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/_profiler
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/actuator
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/admin
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/administrator
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/assets
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/Autodiscover
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/blog
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/c
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/cart
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/cgi-bin
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/checkout
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/console
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/contact
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/cookies
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/css
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/database
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/db
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/dbadmin
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/download
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/feed
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/flu
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/index.php%2C3Frest_route%3D
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/my-account
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/MyAdmin
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/myadmin
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/mysql
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/mysql-admin
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/mysqladmin
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/mysqlmanager
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/p-content
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/photo
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/php-my-admin
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/php-myadmin
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpmy
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpmy-admin
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpMyAdmin
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpMyadmin
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpmyAdmin
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpmyadmin
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpMyAdmin-3
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpMyAdmin-4
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpMyAdmin-4.9.7
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpMyAdmin-5
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpMyAdmin-5.1.0
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpMyAdmin-5.1.1
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpMyAdmin-5.1.2
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpMyAdmin-5.1.3
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpMyAdmin-5.2.0
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpMyAdmin1
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpmyadmin1
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpMyAdmin2
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpmyadmin2
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpmyadmin2011
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpmyadmin2012
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpmyadmin2013
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpmyadmin2014
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpmyadmin2015
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpmyadmin2016
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpmyadmin2017
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpmyadmin2018
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpmyadmin2019
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpmyadmin2020
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpmyadmin2021
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpmyadmin2022
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpMyAdmin3
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpmyadmin3
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpMyAdmin4
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpmyadmin4
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpMyAdmin5
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpmyadmin5
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpMyAdmin5.1
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpMyAdmin5.2
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpMyAdmin_
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phpmyadmin_
/var/cache/mod_pagespeed/v3/mysite.com/https,3A/,2Fdemo.mysite.com/phppma

Thank you if anyone can help!

updated

Here's how my conf files look after implementing Robbie's suggestions -

/etc/apache2/sites-enabled/000-default.conf

# Added to mitigate CVE-2017-8295 vulnerability
UseCanonicalName On

<VirtualHost *:80>

    ServerName catchall

</VirtualHost>

<VirtualHost *:443>

    ServerName catchall

    SSLCertificateFile /etc/letsencrypt/live/demo.mysite.com/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/demo.mysite.com/privkey.pem
    Include /etc/letsencrypt/options-ssl-apache.conf

</VirtualHost>

/etc/apache2/sites-enabled/001-demo.mysite.com.conf

<VirtualHost *:80>
    ServerName demo.mysite.com
    ServerAlias demo.mysite.com

        <Directory /var/www/html/>
            Options FollowSymLinks
            AllowOverride All
            Require all granted
        </Directory>

        RewriteEngine on
        RewriteCond %{SERVER_NAME} =demo.mysite.com [OR]
        RewriteCond %{SERVER_NAME} =www.demo.mysite.com
        RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]

</VirtualHost>

<IfModule mod_ssl.c>
<VirtualHost *:443>
        ServerAdmin webmaster@localhost
        ServerName demo.mysite.com
        ServerAlias www.demo.mysite.com

        UseCanonicalName On
        UseCanonicalPhysicalPort On

        Protocols h2 http/1.1

        DocumentRoot /var/www/html

        <Directory /var/www/html/>
            Options FollowSymLinks
            AllowOverride All
            Require all granted
        </Directory>

        ErrorLog ${APACHE_LOG_DIR}/error.log
        CustomLog ${APACHE_LOG_DIR}/access.log combined

    SSLCertificateFile /etc/letsencrypt/live/demo.mysite.com/fullchain.pem
    SSLCertificateKeyFile /etc/letsencrypt/live/demo.mysite.com/privkey.pem
    Include /etc/letsencrypt/options-ssl-apache.conf
</VirtualHost>
</IfModule>

There are at least two missing chunks in this puzzle: a) have you enabled vhosts, or is it still running config in the main httpd.conf? b) What is redirecting from "80" (http) to 443 (https). That is when the 301 is: and once that redirect happens, the above config is irrelevant. — Robbie, Sep 12 '22 at 00:05
https://httpd.apache.org/docs/2.4/mod/core.html#servername: _"If you are using name-based virtual hosts, the ServerName inside a section specifies what hostname must appear in the request's Host: header to match this virtual host."_ - so unless the host name in the request is `catchall`, your VH does not apply to begin with. — CBroe, Sep 12 '22 at 07:58
Hi @Robbie, a) I don't have a httpd.conf file but I have apache2.conf. I think vhosts is enabled otherwise the SSL wouldn't work. b) I think the redirect happens in this file **000-default.conf** on this line `RewriteRule ^ https://%{SERVER_NAME}%{REQUEST_URI} [END,NE,R=permanent]`. I've updated my post above with contents of both 000-default.conf and **000-default-le-ssl.conf**. I've tried moving the catchall code to the top of this file **000-default.conf** but nothing changed. — wheretheresawill, Sep 12 '22 at 10:34
That question makes no sense; you don't want this to apply to a _specific_ unwanted host name, but to _all_ of them. You need a default VH that does _not_ have any ServerName or ServerAlias set as the first one, https://stackoverflow.com/a/5427520/1427878 — CBroe, Sep 12 '22 at 10:43
Hi @CBroe, thanks sorry for asking stupid questions. I think I tried applying what you recommended to **000-default-le-ssl.conf** but still the same result. This is what the top of the file looks like now - `NameVirtualHost * ServerName catchall Require all denied Require all denied ` Any idea why its not working? — wheretheresawill, Sep 12 '22 at 11:44
What part of "you must _not_ set a ServerName" do you not understand? — CBroe, Sep 12 '22 at 11:49
Hi @CBroe I've removed ServerName catchall now. Still the same result. I've also tried removing ServerName demo.mysite.com from the 443 block in 000-default-le-ssl.conf and from 000-default.conf. Still the same result. What am I doing wrong? Thanks for your help — wheretheresawill, Sep 12 '22 at 12:12
The "wildcard" VHost needs to be the first one, not sure whether that is actually the case when you are using the default `/sites-enabled/` mechanism. — CBroe, Sep 12 '22 at 12:49
Hi @CBroe if I understand right I've now moved this block to the top of both files. I still get the same result. — wheretheresawill, Sep 12 '22 at 12:56

score 1 · Accepted Answer · answered Sep 13 '22 at 05:27

Too long for a comment, so it's coming in as an answer.

I think CBroe's comment response is slightly confusing, but also correct in a way.

To clarify how vhosts blocks work, if the request host (ServerName) does not match any ServerName or ServerAlias declarations, then the FIRST vhost block (matched by *:80, or *:443) is used. So if you only have one vhost declaration it does not matter what you have in the ServerName or ServerAlias section. In this case, "catchall" is perfectly fine.

Three examples:

<VirtualHost *:80>
    // This is the first block, so is default.
    // Matches anything (as there are no other blocks)
    // ServerName can be anything you like.
    ServerName catchall
</VirtualHost>

Compared to

<VirtualHost *:80>
    // This is the first block, so is default.
    // Matches anything that is NOT "mydomain.com" / "www.mydomain.com"
    ServerName catchall
</VirtualHost>
<VirtualHost *:80>
    // Matches only "mydomain.com" / "www.mydomain.com"
    ServerName mydomain.com
    ServerAlias www.mydomain.com
</VirtualHost>

Compared to:

<VirtualHost *:80>
    // This is the first block, so is default.
    // Matches anything that is not "catchall"
    ServerName mydomain.com
    ServerAlias www.mydomain.com
</VirtualHost>
<VirtualHost *:80>
    // This block is a total waste of time as "catchall" is not a valid public host (unless local DNS etc)
    ServerName catchall
</VirtualHost>

In your example, you actually have two *:80 vhosts blocks, one in 000-default.conf and the other in 000-default-le-ssl.conf. The critical question is which is loaded first. They are loaded in "alphanumeric" order, but if you're not sure if "." comes before "-" then I suggest you rename "000" and "001" as that what those numbers are there for; to control loading order.

So what you should have, that will solve your problem, is two conf files, each with 80 and 443 declared, correctly ordered as follows:

000-default.conf

<VirtualHost *:80>
    // This is the first block, so is default.
    // Matches anything that is NOT "mydomain.com" / "www.mydomain.com"
    ServerName catchall
    // Do not add to cache, reject, throw error etc.
</VirtualHost>
<VirtualHost *:443>
    // This is the first block, so is default.
    // Matches anything that is NOT "mydomain.com" / "www.mydomain.com"
    ServerName catchall
    // Do not add to cache, reject, throw error etc.
    // Add your cert details, but users will get a cert error here anyway as name will not match.
</VirtualHost>

001-mydomain.com.conf

<VirtualHost *:80>
    // This is the second block, so must match the host (otherwise, will hit "default")
    ServerName mydomain.com
    ServerAlias www.mydomain.com 
    <Directory..... etc</Directory>
</VirtualHost>
<VirtualHost *:443>
    // This is the second block, so must match the host (otherwise, will hit "default")
    ServerName mydomain.com
    ServerAlias www.mydomain.com 
    <Directory..... etc</Directory>
    // Add your cert details
</VirtualHost>

Hi @Robbie thanks so much for your detailed answer. I think that has solved it! I renamed the files like you recommended and added what you suggested. I've deleted all the cache folders and will keep an eye on it to see if any random folders are added. When I perform **wget --header="Host: example.com" http://demo.mysite.com** I still get the same response **301 Moved Permanently** but the folder example.com is no longer created. Do you know why it says 301 and not 403? I am curious how to make it return 403. — wheretheresawill, Sep 13 '22 at 09:52
I also added this to my .htaccess file to tell the ModPagespeed configuration to explicitly list what you want to allow - ` ModPagespeed on ModPagespeedDisallow *; ModPagespeedAllow *.mysite.com; ` — wheretheresawill, Sep 13 '22 at 09:52
Hi @Robbie, I've updated my post with how the .conf files look now. If you see anything out of place please let me know. Thanks again! — wheretheresawill, Sep 13 '22 at 10:04
The ServerAlias (second part) is missing www (but that's probably typo). You don't need the "RewriteConds" in the https redirect, as the user can't get to that block unless the hosts match. Oherwise it _should_ do what you need (assuming vhosts is turned on in apache.conf, and those conf files are being read, as mentioned earlier). — Robbie, Sep 14 '22 at 02:30
Hi @Robbie, I actually removed www as I thought it wasn't needed as its on a sub domain but I've put it back now. I was looking at the error logs and this message appears *[ssl:warn] [pid 48356:tid 139682146331712] AH01909: catchall:443:0 server certificate does NOT include an ID which matches the server name*. Should it be of concern? When I remove the cert details from 000-default.conf the error goes but the cache folder gets random folders again. Thanks for all your help. — wheretheresawill, Sep 14 '22 at 08:43
Just leave it with the invalid cert, it's a warning but you know it's invalid for that configuration, so all good. Will only appear once (per apache restart) and allows it to work. — Robbie, Sep 14 '22 at 12:54
Hi @Robbie, is it possible to fix this message? **AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1. Set the 'ServerName' directive globally to suppress this message** I tried adding ServerName demo.mysite.com to the main apache.conf file but the random cache folders started appearing again! — wheretheresawill, Sep 15 '22 at 11:06
"ServerName localhost" - but you should be asking these as new questions as it may help others. (Although this is the wrong forum: you want the server forum for that!) — Robbie, Sep 15 '22 at 14:07
Thanks @Robbie. Sorry I will use the server forum next time. Thanks again for your help. — wheretheresawill, Sep 16 '22 at 00:25

how to deny all unexcepted host headers 403 forbidden? Apache 2.4

updated

1 Answers1