First of all, I'm very new to JMeter. I'm trying to do load testing of websocket service but I haven't been able to figure out how to send multiple messages within one connection sequentially (and not parallel with other threads) after response to previous message has been received, in limited number of concurrent connections.
My connection also does not get closed as close pattern is not recognized if response pattern is found.
Update
After modification of WebSocketSampler code both message-specific matching patterns and match & close do work; code changes in the end.
Goal for this example
- Open total 12 connections (3 concurrent, open new connection after previous has been closed)
- send 5 messages to each connection
- Each message contains key-value pair in format "messageIx":n (where n is integer from 1 to 5)
- Wait until first response containing same key is received
- Send next message
- When response containing "messageIx":5 is received, close connection
Actual application will send up to 4 responses for each message. But I haven't been able to find out a way to do this even with WS echo service. I'm using https://github.com/maciejzaleski/JMeter-WebSocketSampler as plugin for JMeter.
Current setup
- Thread group
- 3 threads
- ramp-up 1 second
- loop count 5
- WebSocket Sampler
- Connection timeout 5000ms
- Response timeout 5000ms
- Request data ${request}
- Response pattern ${response}
- Close connection pattern \"messageIx\":5
- CSV Data Set Config
- Variable names request,response
- Sharing mode current thread
- delimiter ;
- Recycle on EOF False
- Stop thread on EOF: True
My CSV file content:
{"messageType":0,"messageIx":1,"action":"None"};"messageIx":1
{"messageType":0,"messageIx":2,"action":"None"};"messageIx":2
{"messageType":0,"messageIx":3,"action":"None"};"messageIx":3
{"messageType":0,"messageIx":4,"action":"None"};"messageIx":4
{"messageType":0,"messageIx":5,"action":"Close"};"messageIx":5
Running this configuration against SimpleWebSocketServer (python) implementation output is
52682 : connected
52682 <-- {"messageType":0,"messageIx":1,"action":"None"}
52691 : connected
52691 <-- {"messageType":0,"messageIx":1,"action":"None"}
52700 : connected
52700 <-- {"messageType":0,"messageIx":1,"action":"None"}
52682 <-- {"messageType":0,"messageIx":2,"action":"None"}
52691 <-- {"messageType":0,"messageIx":2,"action":"None"}
52700 <-- {"messageType":0,"messageIx":2,"action":"None"}
52682 <-- {"messageType":0,"messageIx":3,"action":"None"}
52691 <-- {"messageType":0,"messageIx":3,"action":"None"}
52700 <-- {"messageType":0,"messageIx":3,"action":"None"}
52682 <-- {"messageType":0,"messageIx":4,"action":"None"}
52691 <-- {"messageType":0,"messageIx":4,"action":"None"}
52700 <-- {"messageType":0,"messageIx":4,"action":"None"}
52682 <-- {"messageType":0,"messageIx":5,"action":"Close"}
52691 <-- {"messageType":0,"messageIx":5,"action":"Close"}
52700 <-- {"messageType":0,"messageIx":5,"action":"Close"}
52682 <-- <EOF>
52691 <-- <EOF>
52700 <-- <EOF>
52682 <-- <EOF>
52691 <-- <EOF>
52700 <-- <EOF>
52691 : closed
52682 : closed
52700 : closed
From View Result Tree Listener for first message for each thread I get
[Execution Flow]
- Opening new connection
- Using response message pattern ""messageIx":1"
- Using disconnect pattern "Close"
- Waiting for the server connection for 5000 MILLISECONDS
- WebSocket conection has been opened
- Connection established
- Waiting for messages for 5000 MILLISECONDS
- Received message #1 (47 bytes); matched response pattern
- Leaving streaming connection open
but for following messages there is no match
[Execution Flow]
- Reusing exising connection
- Waiting for messages for 5000 MILLISECONDS
- Received message #2 (47 bytes); didn't match any pattern
- Leaving streaming connection open
For the last message for a thread data is the same except client closing the connection (streaming connection is left open though)
- WebSocket conection has been successfully closed by the server
- WebSocket session closed by the client
- Leaving streaming connection open
However if I modify response pattern to be generic \"messageIx\": all messages are matched but Close Connection is not matched with either \"messageIx\":5 or Close pattern;
Response data:
[Message 5]
{"messageType":0,"messageIx":5,"action":"Close"}
Sampler request:
[Execution Flow]
- Reusing exising connection
- Waiting for messages for 5000 MILLISECONDS
- Received message #5 (48 bytes); matched response pattern
- Leaving streaming connection open
Last, threading. If I set up 12 threads they will all get started with ramp-up delay in the beginning. Adding loop controller will just multiply messages for the same connection and increasing loops in thread group (and using recycle in csv reader) will fail as connection has already been closed (if close connection pattern matches).
Help I'm looking for
- How can I match each messages with unique matching pattern?
- How I can close message while matching pattern is found as well?
- How should I create amount of threads, each with new connection, but restricting concurrent amount to lower number?
Please excuse me if I'm asking something very silly here as I'm very new to JMeter :)
Update: Modified WebSocketSampler code
From Sampler's GitHub page I found issue similar to my message matching problem: https://github.com/maciejzaleski/JMeter-WebSocketSampler/issues/19
I downloaded 1.0.2-SNAPSHOT sources, added modifications from 2nd reply for issue above and also on ServiceSocket.java file changed code (from line 68) as follows:
boolean matchFound = false;
if (responseExpression == null || responseExpression.matcher(msg).find()) {
logMessage.append("; matched response pattern");
closeLatch.countDown();
matchFound = true;
}
if (!disconnectPattern.isEmpty() && disconnectExpression.matcher(msg).find()) {
logMessage.append("; matched connection close pattern").append("\n");
closeLatch.countDown();
matchFound = true;
close(StatusCode.NORMAL, "JMeter closed session.");
}
if (!matchFound)
{
logMessage.append("; didn't match any pattern").append("\n");
} else {
logMessage.append("\n");
}
I haven't ever written Java previously - I would highly appreciate someone more experienced to comment if changes I applied are sound.
Now I'm able to match messages individually and also close connection. Still looking for help on limiting amount of concurrent threads without reusing same connection for messages.
Update 2: Reconnecting
One way to keep concurrent number of users (connections) limited would be to use CSV file for source and set recycle on EOF to true, set threads to concurrent target and loop count to number of connections per thread multiplied by number of messages in CSV file.
On 1.0.2 this won't work directly as connectionId doesn't change and plugin tries to use old, closed connection. On WebSocketSamples.java, line 69:
if (isStreamingConnection()) {
if (connectionList.containsKey(connectionId)) {
socket = connectionList.get(connectionId);
if (socket.isConnected()){
socket.initialize(this);
return socket;
}
connectionList.remove(connectionId, socket);
}
modification seems to do the trick. Messages still seem to be sent parallel and performance compared to for example Node.JS based Thor seem pretty poor :(
