http_sub_module / sub_filter of nginx and reverse proxy not working

Question

I am trying to reverse proxy my website and modify the content. To do so, I compiled nginx with sub_filter. It now accepts the sub_filter directive, but it does not work somehow.

server {
    listen       8080;
    server_name  www.xxx.com;

    access_log  /var/log/nginx/www.goparts.access.log  main;
    error_log  /var/log/nginx/www.goparts.error.log;
    root   /usr/share/nginx/html;
    index  index.html index.htm;

    ## send request back to apache1 ##
    location / {
       sub_filter <title> '<title>test</title>';
 sub_filter_once on;


     proxy_pass  http://www.google.fr;
     proxy_next_upstream error timeout invalid_header http_500 http_502 http_503 http_504;
     proxy_redirect off;
     proxy_buffering off;
     proxy_set_header        Host            $host;
     proxy_set_header        X-Real-IP       $remote_addr;
     proxy_set_header        X-Forwarded-For $proxy_add_x_forwarded_for;

   }
}

Please help me

Why do you intend to modify content within Nginx configuration? Do you want to use Nginx in forward-proxy mode? — Anatoly, Aug 08 '15 at 12:55
Because I intend to add a header and footer to the website this way. I want to use it in reverse proxy mode. — thms0, Aug 09 '15 at 03:24
I even tried Httpsubs and subs_filter, it doesn't work either. Anyone has an idea ? — thms0, Aug 10 '15 at 11:28
Based on proxy_pass *http://www.google.fr* it seems you try to build forward proxy but not reverse one? — Anatoly, Aug 10 '15 at 18:38
Yes, google just to try it out. I now tried to reverse proxy an apache directory listing default page and it works. I read somewhere that httpsubs had problems with gzip-served pages so I added proxy_set_header Accept-Encoding ""; to try to disable gzip from server but no way :/ — thms0, Aug 11 '15 at 18:29
sub_filter doesn't have a problem with gzip content, it just doesn't designed to work with that, the empty header *Accept-Encoding* helps to let backend send non-gzipped content to Nginx. — Anatoly, Aug 12 '15 at 06:58

score 86 · Accepted Answer · edited Jun 17 '17 at 02:13

86

Check if the upstream source has gzip turned on, if so you need

proxy_set_header Accept-Encoding "";

so the whole thing would be something like

location / {
    proxy_set_header Accept-Encoding "";
    proxy_pass http://upstream.site/;
    sub_filter_types text/css;
    sub_filter_once off;
    sub_filter .upstream.site special.our.domain;
}

Check these links

edited Jun 17 '17 at 02:13

M. Schmidt

35
7

answered Mar 29 '16 at 02:18

Mike Lee

2,440
26
12

1

I thought this was an ugly fix since it would disable gzip between origin server and public server (more back-end traffic), but it is Nginx official solution for this: https://www.nginx.com/resources/wiki/modules/substitutions/ – adrianTNT Feb 12 '20 at 16:23
1

I would also add that in my case, it only worked once I set the `proxy_set_header Accept-Encoding "";` inside `location` directive, it would not work under `server`. – adrianTNT Feb 12 '20 at 16:24
1

@adrianTNT - Most likely that is because *any* usage of `proxy_set_header` in a block requires repeating any desired usage from parent blocks. So if your location block had already used `proxy_set_header` and you tried to set it globally in the `server` block, it would have no effect within your `location` block unless you repeated it. This is just an nginx-ism and not specific to the features used here. – huntharo May 03 '22 at 17:39
When I add `Accept-Encoding "";` it gives me a gateway timeout, but when I do the request to the upstream server with that header it works fine, any idea how it could be? – Jonathan Daniel Aug 03 '22 at 13:50

http_sub_module / sub_filter of nginx and reverse proxy not working

1 Answers1

Linked