Load Balancing Config with Yarp not working?

Question

I am working on porting over from nginx to Microsoft's YARP for my reverse proxy needs. Its going well enough. However, I am having my doubts about load balancing. Here is the config example from the yarp docs

"ReverseProxy": {
"Clusters": {
    "cluster1": {
        "LoadBalancingPolicy": "RoundRobin",
        "Destinations": {
            "cluster1/destination1": {
                "Address": "https://localhost:10000/"
            },
            "cluster1/destination2": {
                "Address": "https://localhost:10010/"
            }
        }
    }
}

I am testing in a docker swarm environment and deploying 2 replicas of mytestwebapp. Here is my config:

"ReverseProxy": {
"Routes": {
  "route1": {
    "ClusterId": "mytestwebapp",
    "Match": {
      "Path": "{**catch-all}"
    }
  }

},
"Clusters": {
  "mytestwebapp": {
    "LoadBalancingPolicy": "RoundRobin",
    "Destinations": {
      "destination1": {
        "Address": "http://mytestwebapp:5023/"
      }
    }
  }
}

}

My test app displays the unique id of each containerized deployment. This works....however, it is not, as far as I can tell, doing any RoundRobin action. It hits the same container most of the time.

The other container console logs look like this.

So I got creative and put delays in the repsonse. And simulated multiple concurrent requests. And still, almost always hits the same container.

I say "almost" always because it will go to the next container...after a while (maybe after 5 minutes of doing no requests). I was expecting it to hit one container, then the next, then back to the first, and so on, like, in a Round Robin. What am I missing here?

UPDATE---- Here is a look at my docker compose file:

version: '3.8'

services:
  tempwebapp:
    image: localstore/tempwebapp:1.4
    environment:
      - ASPNETCORE_URLS=http://*:5023
      - ASPNETCORE_ENVIRONMENT=Production
    ports:
      - 5026:5023
    deploy:
        replicas: 5
    networks:
      - localnet
  yarp:
    image: localstore/localclusteryarptest:1.6
    ports:
      - 6929:6929
    environment:
      - ASPNETCORE_ENVIRONMENT=Production
    deploy:
      replicas: 1
      placement:
        constraints: [node.role == manager]
      restart_policy:
        condition: on-failure
    networks:
      - localnet
networks:
  localnet:
    driver: overlay

brando · Answer 1 · 2023-06-01T16:09:28.727

So, it turns out that I was missing an understanding of a few key things.

First, I was helpfully informed on github by Sam Spencer from Microsoft of the following:

You need a mechanism to resolve the destinations by talking to whatever is doing the dynamic clustering - such as kubernetes, which there is a YARP ingress controller for k8s. One of the reasons we have the extensibility in YARP is to enable customers to write configuration management that will pull the data from their backend systems. Source

So, basically, you do not rely at all on Yarp do to your load balancing in a clustered environment. In my testing, I have been using Docker Swarm. So how do you do load balancing in a docker swarm environment? It is already baked in the swarm mode routing mesh.

Second, what I didnt understand is that Kestral server will keep alive connections by default for 120 seconds. Had I originally waited 120 or more seconds in my testing, it would have worked as expected. However, for testing purposes, I changed the keep alive settings in the yarp proxy server and for the testwepaps as follows:

Host.CreateDefaultBuilder(args)
 .ConfigureWebHostDefaults(webBuilder =>
   {
      webBuilder.ConfigureKestrel((context, serverOptions) =>
        {
           //config stuff etc
           serverOptions.Limits.KeepAliveTimeout = TimeSpan.FromSeconds(5);
         });
      webBuilder.UseStartup<Startup>();
  });

So, now if I wait 5 or more seconds after each request, then the docker ingress controller (or routing mesh) will handle the round robin action. Bing, bang, boom. Round Robin load balancing works as I was expecting.

You can get more sofisticated with load balancing in docker swarm and configure an external load balancer such as HAProxy. I did in fact test this and it worked. I might do this if I need to load balance based on the container with the least connections.

Kind of a "dumb" question in retrospect, but I now understand what is going on underneath the hood with yarp and load balancing. Hope this helps someone else.

Load Balancing Config with Yarp not working?

1 Answers1