0

In our application, Server A establishes a TCP connection with Server B, then it sends a large amount of requests to Server B over the TCP connection. The request message is XML-based. Server B needs to respond within a very short period, and it takes time to process the requests. So we hope a load balancer can be introduced and we can expedite the processing by using multiple Server B's. This is not a web application. I did some research but failed to find a similar application of load balancer.

Can anyone tell me if there's a load balancer can help in our application?

A diagram illustrating our application:

enter image description here

Hyppy
  • 15,608
  • 1
  • 38
  • 59
Shawn
  • 1
  • 1
  • Hmmm, I think a diagram of how your apps communicate might help us here. Even something crappy thrown together in paint would help. – Mark Henderson Mar 08 '11 at 23:26
  • Are you sure that all the data is transferred over the same TCP connection? You say there's a large amount of requests. Does this all definitely happen over a single connection attempt? – Lewis Mar 10 '11 at 17:03
  • The TCP connection is disconnected when there is no activity for some time. But if 20000 requests are received by Server A in 1 sec, all of them will be sent through the same connection. – Shawn Mar 10 '11 at 18:15

3 Answers3

5

Ok, correct me if I'm wrong, but this is the situation you have right now:

enter image description here

And this is what you're proposing:

enter image description here

This will only help if the bottleneck is inside B. If the bottleneck is between B and A, then this will only make things worse.

If the bottleneck is inside B, then adding LB will increase overheads in term of switching back and forth, but it will distribute your load fairly evenly over the two servers.

Mark Henderson
  • 68,823
  • 31
  • 180
  • 259
  • 1
    Plus one for the hand drawn diagram :) – Jacob Mar 08 '11 at 23:41
  • @Jacob - I was going to use Visio but I was able to draw that in the time it takes Visio to start up and create a blank document. – Mark Henderson Mar 08 '11 at 23:44
  • No this is *much* better :) – Jacob Mar 09 '11 at 00:16
  • Thanks, Mark! I added a link to a visio diagram. Cannot post image directly since I don't have enough credits. – Shawn Mar 10 '11 at 15:51
  • I believe the bottleneck is inside B. However, seems there's no easy way to distribute the requests from the same direct source (server A) and on the same connection. – Shawn Mar 10 '11 at 15:55
  • haven't read the answer, but +1 for the diagram! – Coops Mar 10 '11 at 16:08
  • @shawn - in light if your visio diagram and if the bottleneck is inside B then a load balancer should be great. If you are using TCP then haproxy should be a nice L7 load balancer to try out. – Mark Henderson Mar 10 '11 at 20:37
  • The L7 switching provided by Haproxy only works with HTTP header. For example, something like this:
    POST / HTTP/1.1 Connection: keep-alive Content-Length: 1377 Content-Type: text/xml; charset=UTF-8 Date: Tue, 30 Nov 2010 20:09:50 GMT Host: 164.2.75.63
    In our application, the requests are all XML messages.     – Shawn Mar 10 '11 at 22:28
  • @Shawn - What's the protocol you're using for communications? Just raw TCP to a listener? I believe that HAProxy can LB non-http protocols, but perhaps not at L7: http://www.linickx.com/645/load-balance-anything-with-haproxy – Mark Henderson Mar 11 '11 at 00:15
  • I feel the pain of the artist creating the diagram. Consider yEd, I can't live without it. It's available on all 3 major OSes. – Sridhar Sarnobat Aug 14 '19 at 00:25
2

No. Load balancers work by moving connections from one server to another. The connection would need to disconnect then send the next request, that or just make 100 connections and then send one request over each connection. As everything is happening over a single connection you are pretty much hosed without doing some redesign.

mrdenny
  • 27,174
  • 4
  • 41
  • 69
  • Thanks. I'm now trying to find out if L7 switching could help. – Shawn Mar 10 '11 at 16:28
  • This is unfortunate for my case where I'm using a Log4j `` appender to a load balanced log server. All log statements for one app get routed to just one server. – Sridhar Sarnobat Aug 14 '19 at 00:29
0

Provided that each request is a new TCP connection(and the data being requested is relatively static) this could be load balanced using something like Linux Virtual Server. However if you are able to modify the application why not build the load balancing in there instead?

Will
  • 226
  • 1
  • 3
  • I can only modify Server B. It seems we can implement our own layer-7 load balancing, which involves parsing the XML requests and forward requests to a Server B. Not sure if Linux Virtual Server can help and if the performance will be satisfying though. Don't want to implement from 0. – Shawn Mar 10 '11 at 16:07