0

I have an EC2 instance behind a load balancer. The security group attached to it allows for inbound connections (both ipv4 and ipv6 on port 6379). I am able to connect to my redis client:

redis-cli -h ec2-**-**-**-*.us-west-1.compute.amazonaws.com -p 6379

However, when I try to connect with nodeJS and express-session I get a ConnectionTimeoutError on EC2, but locally it works fine:

const redisClient = createClient() // uses default port localhost:6379 
redisClient.connect().catch(console.error)

If there is a race condition here, like others mentioned, why does this race condition happen on EC2 and not locally? Is the default localhost incorrect since there is a load balancer in front of the instance?

AlexDev
  • 35
  • 6
  • So you can connect using your redis client but not your code? Where is the EC2 address in your JS code? – Ermiya Eskandary Dec 17 '22 at 16:01
  • Yes, that is correct. In my code there is no EC2 address, because the EC2 hosts both the server and Redis. So this uses the default `localhost:6379`. This all works as expected on localhost, but not on EC2, and I am wondering if the load balancer plays a role here – AlexDev Dec 17 '22 at 16:29
  • Are you running the Node JS code on the instance itself? – Ermiya Eskandary Dec 17 '22 at 16:47
  • Yes, but even if I include the Redis `host:port` when I connect to redis, I still can't get the EC2 instance to work. Localhost connects fine to the Redis client that is hosted on that remote EC2 instance. It looks like there is a curious race condition when connecting to Redis – AlexDev Dec 17 '22 at 16:48
  • Some more details on the AWS setup would help to understand the problem better. What type of load balancer are you using? Is the EC2 on the same public subnet as the load balancer, or a private subnet? – chamal Dec 17 '22 at 16:50
  • It is an `Application` load-balancer, and includes two subnets, one of them is shared with the instance, no problems there – AlexDev Dec 17 '22 at 17:03
  • So you're running Redis locally self-managed on an EC2 instance? – Ermiya Eskandary Dec 17 '22 at 18:40
  • Yes, that is correct. I am running Redis on an EC2 instance – AlexDev Dec 17 '22 at 19:30

2 Answers2

0

Based on your comments, I'd say the problem is the load balancer. Redis communicates on a protocol based on TCP. An ALB is only for HTTP/HTTPS traffic, so it cannot handle this protocol. Use a Network Load Balancer instead, with a TCP listener. Also make sure your security group rule also allows TCP traffic for port 6379.

chamal
  • 858
  • 6
  • 13
  • Thanks for your reply. If this is the case, why can I connect to my redis server by using `redis-cli -h ec2-**-**-**-*.us-west-1.compute.amazonaws.com -p 6379`? – AlexDev Dec 17 '22 at 18:16
  • Are you sure that's your ALB endpoint you're using with redis-cli? ALB DNS are usually like ```my-alb-1234567890abcdef.elb.us-west-1.amazonaws.com```. The DNS in your example is in the pattern of what EC2 instances would have. If you're connecting directly to a public EC2 and your security group allows it, that will work. Are you using this same endpoint with your application as well? – chamal Dec 17 '22 at 19:05
  • That is a good point, I think that's where the problem is. I use the following in NodeJS: `createClient({ socket: { host: '172.**.**.**', port: 6379 }, legacyMode: true })`, which tries to connect to that host with 6379. What is my redis url, though? – AlexDev Dec 17 '22 at 19:15
0

Redis client should be instantiated explicitly in a setup like this one (covers both ipv4 and ipv6 inbound traffic):

createClient({ socket: { host: '127.0.0.1', port: 6379 }, legacyMode: true })

As redis is self-hosted on EC2 with a load balancer in front of the instance, localhost may not be mapped to 127.0.0.1 as a loopback address. This means that the default createClient() without a host or port specified, might try to establish a connection to a different internal, loopback address.

(Make sure to all inbound traffic to tcp 6379, or the port you are using)

AlexDev
  • 35
  • 6