When a request gets denied due to require all denied, it first gets routed to the ErrorDocument 403.shtml but then that gets rewritten rather than stopping there and the site content actually gets served up (albeit with a 403 status code).
We've boiled it down to a very simple test case
In .htaccess in the root of the site put the following.
Require all denied
RewriteRule .* test.txt [L]
In test.txt in the root of the site put the following:
You shouldn't see this text here but if you are it's because of the RewriteRule running after Require all denied gets processed
I confirmed commenting out the RewriteRule causes the Require Directive to behave as expected.
Here's some relevant LogLevel trace8 output:
authorization result of Require all denied: denied
authorization result of <RequireAny>: denied
AH01630: client denied by server configuration
auth phase 'check access' gave status 403
mod_rewrite.c: strip per-dir prefix: /home/path-to-site/403.shtml -> 403.shtml
mod_rewrite.c: applying pattern '.*' to uri '403.shtml'
mod_rewrite.c: rewrite '403.shtml' -> 'test.txt'
EDIT:
One thing that's odd about this issue is that it started occurring all of a sudden. It's almost like something changed on the server like something that would affect the ErrorDocument directives or their behavior, but I can't think of anything we might have changed that could cause that.
