How to resolve IPs of AWS ELB in Openresty for every upstream

Question

The problem we're having is that openresty nginx only resolves the IP of AWS ELBs at startup and then caches the IP forever (until it reloads). Since the AWS loadbalancer IP can change anytime, I am looking for a solution that can resolve the ELB IPs for every time its being used in upstream. Looking for something similar to upstream "resolve" option in nginx+, but in Openresty. Or some other method to invalidate upstream DNS cache

user454322 · Accepted Answer · 2018-07-06T05:07:47.350

2

What about using a DNS server as your resolver and a variable for the value of proxy_pass?

Look at the example below from this post "Nginx with dynamic upstreams" by Jeppe Fihl-Pearson

resolver 172.16.0.23;
set $upstream_endpoint http://service-1234567890.us-east-1.elb.amazonaws.com;
location /foo/ {
    rewrite ^/foo/(.*) /$1 break;
    proxy_pass $upstream_endpoint;
}

edited Jul 06 '18 at 05:07

answered Jul 03 '18 at 06:36

user454322

7,300
5
41
52

ya this is good one also but it doesnt support variable e.g. `resolver $resolver_value;` – Bilal Ahmad Jul 05 '18 at 11:07
the targets are resolved dynamically by the resolver, why would a variable for the resolver be useful? – user454322 Jul 06 '18 at 05:10
This solution cannot work with `proxy_protocol on;` – Tonghua Nov 17 '20 at 15:50

score 1 · Answer 2 · answered May 29 '18 at 07:00

There is balancer module which you can use from within balancer_by_lua_block directive. You would be able to set any upstream IP.

This Lua code execution context does not support yielding, so Lua APIs that may yield (like cosockets and "light threads") are disabled in this context. One can usually work around this limitation by doing such operations in an earlier phase handler (like access_by_lua*) and passing along the result into this context via the ngx.ctx table.

You may use lua-resty-dns and resolve your ELB IP for every request (is it performant?!) in access_by_lua_*, save obtained IP(s) into ngx.ctx and use it within balancer_by_lua_block.

Thanks for the answer. It did help me in resolving the load balancer. But I ended up doing something very easy, which was to load dynamic module in openresty which did the same thing exactly — Bilal Ahmad, Jun 07 '18 at 12:26

score 0 · Answer 3 · answered Sep 03 '20 at 01:35

0

Add this to the server directive

resolver local=on valid=5s;

You can then use the upstream block as usual.

answered Sep 03 '20 at 01:35

basex

490
4
11

How to resolve IPs of AWS ELB in Openresty for every upstream

3 Answers3