Adding header to response for specific URLs with HAproxy

Question

I have a simple condition in my HAproxy config (I tried this for frontend and backend):

acl no_index_url path_end .pdf .doc .xls .docx .xlsx
rspadd X-Robots-Tag:\ noindex if no_index_url

It should add the no-robots header to content that should not be indexed. However it gives me this WARNING when parsing the config:

acl 'no_index_url' will never match because it only involves keywords
    that are incompatible with 'backend http-response header rule'

and

acl 'no_index_url' will never match because it only involves keywords
    that are incompatible with 'frontend http-response header rule'

According to documentation, rspadd can be used in both frontend and backend. The path_end is used in examples within frontend. Why am I getting this error and what does it mean?

score 22 · Accepted Answer · answered Feb 15 '16 at 19:11

22

Starting in HaProxy 1.6 you won't be able to just ignore the error message. To get this working use the temporary variable feature:

frontend main
   http-request set-var(txn.path) path

backend local
   http-response set-header X-Robots-Tag noindex if { var(txn.path) -m end .pdf .doc }

answered Feb 15 '16 at 19:11

Adam Albright

5,917
1
21
14

1

You saved my day :) – Yajo Jan 16 '17 at 14:05
This might be useful as a solution, although it is not useful (I think) in my situation, where I use HAProxy via OpnSense. Any feedback on the actual question of `Why am I getting this error and what does it mean`? – user2173353 May 02 '20 at 13:13
OK. I found the answer here: https://discourse.haproxy.org/t/http-response-set-header-with-condition-not-working/3108/2 . Not sure why this works like that, by anyway... – user2173353 May 02 '20 at 13:19

score 2 · Answer 2 · answered Apr 19 '15 at 15:13

2

Apparently, even with the warning, having the acl within the frontend works perfectly fine. All the resources with .pdf, .doc, etc are getting the correct X-Robots-Tag added to them.

In other words, this WARNING is misleading and in reality the acl does match.

answered Apr 19 '15 at 15:13

Xeos

5,975
11
50
79

Just a note for anyone coming across this, in 1.6.2 it does add the header despite the warning but also adds it to any response not matching the rule so I believe it is actually being ignored. That was the case for me anyway, I was using an 'unless' if that makes a difference. The accepted answer worked for me so try that. – James Jun 01 '16 at 10:50

score 0 · Answer 3 · answered Aug 30 '18 at 14:05

if using haproxy below v1.6, create a new backend block (could be a duplicate of the default backend) and add the special headers in there. then in frontend use that backend conditionally. i.e.

use_backend alt_backend if { some_condition }

admittedly not an ideal solution but it does the job.

Adding header to response for specific URLs with HAproxy

3 Answers3