Apache htaccess ruleset: Try request as: 1) file as-is 2) file + .html suffix 3) DirectoryIndex 4) Else to index.php CMS router

Question

I'd like to express this ruleset

If the request is /hello then try the following in the given order:

1. /hello — File of that name exists (file without file extension).
   • Edit: Not a necessity. Only a possibility/compromise. Drop if too complicated.
2. /hello.html - File of that name plus .html extension exists.
3. /hello/index.(htm|html|php) — Folder of that name with index file exists. Note: /hello/ directory listing shall explicitly be forbidden
4. /index.php — If nothing of the above matched hand over to CMS index.php (e.g. Wordpress)

My .htaccess in the domain root folder of my shared hosting account:

## Request without file extension
# e.g. "/hello"

### First look for DirectoryIndex files (with mod_dir)
# e.g. "/hello" shall serve "/hello/index.(html|htm|php)" if present
# Explicitly forbidding directory listings (for security/privacy)
<IfModule mod_dir.c>
Options -Indexes
DirectoryIndex index.html index.htm index.php
</IfModule>

### If no DirectoryIndex found then try with .html suffix (with mod_rewrite)
# e.g. "/hello" shall serve "/hello.html" if present
<IfModule mod_rewrite.c>
RewriteRule ^([^\.]+)$ $1.html [NC,L]
</IfModule>

## Everything else goes to Wordpress index.php and its standard htaccess configuration like this:
# BEGIN WordPress
# The directives (lines) between "BEGIN WordPress" and "END WordPress" are
# dynamically generated, and should only be modified via WordPress filters.
# Any changes to the directives between these markers will be overwritten.
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
RewriteBase /
RewriteRule ^index\.php$ - [L]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>
# END WordPress

Problems

1. mod_dir 's DirectoryIndex alone works:

• ✅ Requesting /a01 serves /a01 being a file without suffix.
• ✅ Requesting /a02 serves /a02/index.html which is the DirectoryIndex.

2. mod_rewrite's RewriteRule which tries with an added .html suffix alone works:

• ✅ Requesting /a03 serves /a03.html.
• ❌ But now requesting /a02 returns Apache error page 403 Access forbidden.
• ❌ And now requesting /a01 returns Wordpress error page 404 Not found.
  • Strange because that file exists and hence in the htaccess Wordpress section RewriteCond %{REQUEST_FILENAME} !-f is not even met, so how can that even land in Wordpress routing.

3. So mod_dir and mod_rewrite rules being active together seemingly result in a conflict.

• Is a central ruleset of my webhost interfering with this?
• Or is this a general interplay issue of the two modules? How do I get them to work together as intended?

Answer 1

If there is no matching #3 DirectoryIndex then continue to #4 CMS

You can't fail "gracefully" with mod_dir's DirectoryIndex to then do something else with the request using mod_rewrite (ie. route the request to #4 the CMS). mod_dir is processed too late. So, instead of using DirectoryIndex we would need to simulate this with mod_rewrite.

However, another (minor) issue here is that the WordPress code block (that, as the comment states, should not be edited manually) needs to be edited to allow requests for filesystem directories to be passed to the CMS.

I'm assuming that any direct requests for a directory should include a trailing slash. For example, if /hello is a physical directory then you should be requesting /hello/ (with a trailing slash). We will append the trailing slash if omitted (which is what mod_dir will do by default anyway, but we need to do this manually if overriding Directoryindex.) We could disable the trailing slash (and make the canonical URL the one without a trailing slash) but this requires additional rewriting.

So, to satisfy your requirements, you could do it like this in the root .htaccess file:

Options -Indexes

# Required for the root directory (eg. the homepage of the CMS)
DirectoryIndex index.html index.htm index.php

RewriteEngine On

# Initially part of the WordPress/CMS block
# (This is just an optimisation)
RewriteRule ^index\.php$ - [L]

# Abort early if a file is requested directly
# (Regardless of whether that file includes a file extension.)
RewriteCond %{REQUEST_FILENAME} -f
RewriteRule . - [L]

# If a directory is requested, which is missing the trailing slash then append it
RewriteCond %{DOCUMENT_ROOT}/$1 -d
RewriteRule ^(.*[^/])$ /$1/ [R=301,L]

# Test if "<url>.html" exists and rewrite if so
RewriteCond %{DOCUMENT_ROOT}/$1.html -f
RewriteRule ^([^.]*[^/])$ $1.html [L]

# Optimisation: If a directory is not requested then skip the next 3 rules
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . - [S=3]

# Check for "DirectoryIndex" documents in order: index.html, index.htm and  index.php
# NB: Directories end in a trailing slash (enforced above)
RewriteCond %{DOCUMENT_ROOT}/$1/index.html -f
RewriteRule ^(.+)/$ $1/index.html [L]
RewriteCond %{DOCUMENT_ROOT}/$1/index.htm -f
RewriteRule ^(.+)/$ $1/index.htm [L]
RewriteCond %{DOCUMENT_ROOT}/$1/index.php -f
RewriteRule ^(.+)/$ $1/index.php [L]

# CMS Fallback...
# But note that the two conditions (filesystem checks) are removed.
# The first one that checks for a "file" is simply not required.
# However, the second check MUST be removed otherwise directories that do not contain a "DirectoryIndex" are not routed to the CMS.

# WordPress...

RewriteRule ^ - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]

# CMS / Front-Controller
RewriteRule . /index.php [L]

Additional notes:

• As an optimisation, I'm assuming your URLs that map to .html files do not contain dots. This is what you had done, so I assume that's OK. (There is no need to backslash-escape a literal dot when used inside a regex character class.)

• I've removed the WordPress comment markers and reduced the WordPress code block to all that's required. One of the RewriteRule directives is moved to the top of the .htaccess file (since this is an optimisation, it doesn't make much sense to have it at the end anymore). You would need to configure WordPress (or your file perms) to prevent WordPress from trying to maintain the .htaccess file (although this could cause issues with plugins).

• Passing filesystem directories to the CMS is certainly non-standard. And boilerplate code (front-controller pattern) for most CMSs will explicitly exclude physical directories. However, the added complication here is that you only want directories where the DirectoryIndex document is not present in that directory, to be passed to the CMS.

I like to preserve the possibility for files without a suffix.

The "problem" with not having file extensions on the underlying file is that Apache does not necessarily know how to handle the request and what "Content-Type" header to send (so the browser does not know how to handle the response).

A workaround in this case is to have all extensionless "files" of a specific type in a known subdirectory and force all those requests with the same Content-Type.

Note that files and URLs are very different in this respect. URLs without extensions is not a problem.

---

Aside:

RewriteRule ^([^\.]+)$ $1.html [NC,L]

The problem with this rule is that you are unconditionally applying the .html extension to any URL that does not contain a dot. /a01 is rewritten to /a01.html, which is not a file (so the condition is successful) and /a01 (the URL that WP sees) is not a registered WP URL so results in a 404 generated by the CMS/WordPress.