1

Requirement:
I am load testing Rabbit MQ by generating a large number of requests/sec to the server (Server config: 8 core, 16GB).

To achieve the above, I am generating load from a client machine at maximum rate as possible (Client config: 4core, 4GB).

Problem:
Even after running the client to generate maximum req/sec, I am able to achieve only 10,000req/sec from 1 client. Only 20% of the CPU and 2% of memory is utilized. Hence I assume there are no limitations from the client side.

On monitoring the bandwidth usage per connection on the server, I notice that my client connection is allocated a maximum on 12MB only. Increasing the number of connections from client does not solve the problem either.

Query:
Is the bandwidth per connection limiting it? Is there something else which i am missing which can increase my overall connection rate from client?

minion
  • 133
  • 5
  • 4
    Don't suppose there might be a 100Mb/s connection anywhere in the pipeline? That would work out at almost exactly 12MB/s. – MadHatter Jun 18 '14 at 07:45
  • That makes sense. The bandwidth on my server side shows 21.4Gb/sec (2.6GB/sec). How does this lead only to 12MB/sec? – minion Jun 18 '14 at 11:58
  • Beats me. You said that client-server connections maxed out at 12MB/s; I pointed out that if anywhere in the network path between them there was a 100Mb/s link or interface, that would explain the figure perfectly. I said nothing about any multi-gigabit-bandwidth figures, and nor did you; where do they now come into the picture? – MadHatter Jun 18 '14 at 14:38
  • Sorry if i was unclear. This is what i meant. Using iftop monitored the bandwidth per connection and it showed my client connection using 12MB (even though my client is loading at the maximum possible extent). Whereas the available bandwidth on the server shows 21.4Gb/sec (measured using iperf). – minion Jun 19 '14 at 06:14
  • I'm still completely unsure what "*available bandwidth on the server*" is supposed to mean. `iperf` may have told you that the server, given a 21Gb/s link, would be capable of saturating it, **but it is extremely unlikely that the server actually has such a link to anywhere**. Have you been able to verify what sort of network path is available between client and server? – MadHatter Jun 19 '14 at 06:46

1 Answers1

2

I think you should clarify the environment you are using (s.o., hw, network) and what you mean with "server load". Bandwidth? Cpu? Memory? No. of connections? Sausages? :-p

For bandwidth, I think that MadHatter's comment is quite right. Go gigabit, it's quite cheap nowadays. Keep also in mind that often overheads prevents you to achieve the expected speed: I remember, when I was writing my MSc thesis, that I couldn't achieve the speed I expected because the kernel was capping me.

If you want to check if the cpu can handle the load, why don't you run your client software on the server? You could use the loopback address (127.0.0.1) to connect.

Connections: you could maybe try to reduce the size of the data you're sending and use a 3rd pc as a second client, if you just want to check the number of maximum connections. Or you can do some syn flood if you use tcp.

Sausages: help yourself with a beer :-D

rookie coder
  • 121
  • 1
  • Thanks for the answer. I tried your suggested method. Ran the test code on the server. The performance was almost the same. And the size of the data being sent is minimal. Hence I do not think that would cause any overhead. – minion Jun 18 '14 at 12:05