How do I do capacity test a websocket server?

Question

I am looking to capacity test my websocket server but don't really know where to start?

I am able to write a AI that will send messages to test the usage, but how would I simulate/make 100, 500, 1000 connections etc...?

Answer 1

I had a similar problem a little while ago when I had to load test thousands of connections against a server using the socket.io library. I was not able to find any off-the-shelf-solutions to do this so in the end I ended up building my own test using Node.js and a few for loops.

The advantage of Node is you can pretty much copy and paste the client side javascript into your server code so it's pretty simple to simulate the client and then you only need to make multiple connections to generate load. It's a quick and easy way to run the required javascript to establish the socket connection (assuming this is how you connect to your socket).

The gotcha I hit was running more than 600 listeners tended to max out the CPU on my node box but a little bit of AWS magic solved that.

Another issue is reporting results. There's not really any concept of response time with a socket connection, at least not in the classic sense, so it's hard to know when things are going wrong - at least from the client side perspective. But from monitoring the server we were able to see when connections failed and when resources started to get scarce and this was enough for us to benchmark how many connections it could support.

Answer 2

Autobahn Testsuite was designed to meet that need but the performance section of the tool still says "Under Development".

Answer 3

You could use JMeter for this purpose and get the WebSocket sampler plug-in from here: http://github.com/maciejzaleski/JMeter

For that many connections 1000 you might need to get more than one agent machine to achieve your task. This doesn't necessarily have to be dedicated server as you could deploy agents on few workstations (developers/testers machines) and used them for your test purpose. You could limit the impact by scheduling test execution to run out-of-hours.

Answer 4

Jmeter plugin is having severe limitations with number of concurrent users. It was working well only till ~450 users. Then I tried with artillery library(https://artillery.io/docs/testing_websockets.html) but this library also has restrictions with loops with their web socket package.