This article will hopefully save you some of the trouble I had, when I tried to use Gatling for Socket.IO sockets. Gatling has support for web sockets but one has to do a lot of research and workarounds in order to make it work with Socket.IO sockets.

What are Gatling and Socket.IO?

Gatling is a powerful load-testing solution for applications, APIs, and microservices (https://gatling.io/).

Socket.IO is a library that enables low-latency, bidirectional and event-based communication between a client and a server (https://socket.io/docs/v4/). The Socket.IO connection can be established with different low-level transports and one of them are web sockets. Socket.IO uses the web socket protocol as one of the underlying protocols for bidirectional communication.

Load testing Socket.IO

Back to our problem…

The Socket.IO server

I prefer to always work with minimal examples when I test something new. I want to avoid the complexity of a large problem getting into the way of understanding how to solve the technical issues. That’s why the working example is very simple:

• The Socket.IO server responds to the special connection event.
• On a successful connection:
  • the socket accepts messages with the message event and the message content. On the reception of the message it logs the message and then emits the message using a broadcast event.
  • Finally, the socket accepts the special disconnect event with a log message.

This is the implementation of the server in JavaScript:

const { Server } = require("socket.io");
const cors = require("cors");

const io = new Server({
  transports: ["websocket", "polling"],
  cors: {
    origin: "*",
    methods: ["GET", "POST"],
  },
});

io.on("connection", (socket) => {
  console.log("a user connected to the socket.io server: " + socket.id);

  socket.on("message", (msg) => {
    console.log("message: " + msg);
    io.emit("broadcast", "they say: " + msg);
  });

  socket.on("disconnect", () => {
    console.log("user disconnected: " + socket.id);
  });
});

io.listen(3000);

The only required dependencies are:

npm i socket.io cors

Back to Gatling:

Gatling websocket simulation

Following the Gatling documentation for Web Sockets: https://gatling.io/docs/gatling/reference/current/http/websocket/ I created a simple simulation:

package computerdatabase;

import static io.gatling.javaapi.core.CoreDsl.*;
import static io.gatling.javaapi.http.HttpDsl.*;

import io.gatling.javaapi.core.*;
import io.gatling.javaapi.http.*;

public class WebsocketExampleSimulation extends Simulation {

    HttpProtocolBuilder httpProtocol = http
            .wsBaseUrl("ws://localhost:3000");

    ScenarioBuilder scene = scenario("WebSocket")
            .exec(ws("Connect WS").connect("/"))
            .pause(1)
            .exec(ws("Say hi")
                    .sendText("message Hi")
                    .await(30).on(
                            ws.checkTextMessage("checkMessage")
                                    .check(regex(".*they say*"))))
            .pause(1)
            .exec(ws("Close WS").close());

    {
        setUp(scene.injectOpen(
                atOnceUsers(1))
                .protocols(httpProtocol));
    }
}

… but nothing works the first time:

Well in this particular case, nothing worked after several hours of all kinds of attempts (which I will not present) until I managed to figure out all the pieces of the puzzle. And this is the reason I want to share with you my findings.

Testing with Postman

To make sure that my server is working properly I used Postman and I created a new Socket.IO connection.

Note

Well, in reality I did many other things before trying out Postman and this is one of the major mistakes that I have done. I should have used a trusted tool to make sure that my server is working instead of trying things around.

When I created a new Socket.IO client in Postman I could successfully connect to the Socket.IO server, send and receive messages. But the Gatling simulation could not even connect to the server.

Gatling URL for Socket.IO connections

So my first investigation involved finding out how to connect to the Socket.IO server. Searching for examples, I discovered a question in a forum (https://community.gatling.io/t/checking-socket-io-messages/7350) using the syntax:

.exec(ws("Connect WS").connect("/socket.io/?EIO=4&transport=websocket"))

This time the Gatling output was:

Gatling reported that Connect WS received an OK response. Even the Say hi received and OK response. But on the server side there was no log message for the connection or the reception of the message.

Logging the Socket.IO server connection with Postman as a client

This is the part that took me the longest to figure out and I managed to solve it only after turning on the debug messages on the server side and trying the successful connection and transfer of messages with Postman.

On a terminal, I started the server with:

DEBUG="*" node server.js

And then I connected with Postman. The debug messages are:

At the bottom I could see the log message from my server, indicating a successful connection. The interesting messages are those starting with engine:ws:

Gatling: connecting to a Socket.IO server

So my guess was that the first interaction from Postman received a web-socket response with the number 0 and an object with the connection info. Even more interestingly, the websocket then received a 40 from Postman. And the answer was 40{"sid":"JnYOfDWvpdm1zrCEAAAB"}.

At that moment I tested sending the message 40 in Gatling after the connect.

    .exec(ws("Connect WS").connect("/socket.io/?EIO=4&transport=websocket"))
    .exec(ws("Connect to SocketIO").sendText("40"))

This time I also received the log message from the server on connection:

That was progress!

Logging the Socket.IO messages with Postman as a client

I then had to see what happens with the other messages. So back to Postman, I sent a message Hi (with the default event name message). The output of the debug was:

The last line was from the log from the server. Observing:

Gatling: sending Socket.IO messages

I imitated the format in my message in Gatling:

        .exec(ws("Say hi")
                .sendText("42[\"message\",\"Hi\"]")

Finally! I could see the server receiving my message and additionally no errors in the simulation:

So what are these numbers? According to the Socket.IO documentation:

So, in 40, 4 is for EIO=4 and 0 for connect. In 42, 4 is for EIO=4 and 2 for an event.

The next thing I wanted to solve was to receive the broadcast message sent by:

socket.on("message", (msg) => {
  console.log("message: " + msg);
  io.emit("broadcast", "they say: " + msg);
});

Back to the debug log:

So the message is emitted as:

Gatling: receiving server messages

Actually it is emitted as 42["broadcast","they say: Hi"]. I found this by adding in Gatling:

            .exec(ws("Say hi")
                    .sendText("42[\"message\",\"Hi\"]")
                    .await(30).on(
                            ws.checkTextMessage("checkMessage")
                                    .check(regex(".*")
                                            .saveAs("response"))))
            .exec(session -> {
                System.out.println("response: " + session.get("response"));
                return session;
            })

So, I could extract with a regular expression the exact response after the broadcast event.

            .exec(ws("Say hi")
                    .sendText("42[\"message\",\"Hi\"]")
                    .await(30).on(
                            ws.checkTextMessage("checkMessage")
                                    .check(regex(".*broadcast...([^\"]*)")
                                            .saveAs("response"))))

Closing the Socket.IO connection

A minor detail, but I noticed that closing the Socket.IO connection (with Postman) involved yet another text message:

So to properly disconnect, I have to send the text 41:

            .exec(ws("Disconnect from Socket.IO").sendText("41"))
            .exec(ws("Close WS").close())

Complete simulation scenario

With this addition, my debug log when executing with Gatling was identical with the debug log when executing with Postman. Mystery revealed. My Gatling scenario looks like this:

    ScenarioBuilder scene = scenario("WebSocket")
            .exec(ws("Connect WS").connect("/socket.io/?EIO=4&transport=websocket"))
            .exec(ws("Connect to Socket.IO").sendText("40"))
            .pause(1)
            .exec(ws("Say hi")
                    .sendText("42[\"message\",\"Hi\"]")
                    .await(30).on(
                            ws.checkTextMessage("checkMessage")
                                    .check(regex(".*broadcast...([^\"]*)")
                                            .saveAs("response"))))
            .exec(session -> {
                System.out.println("response: " + session.get("response"));
                return session;
            })
            .exec(ws("Disconnect from Socket.IO").sendText("41"))
            .exec(ws("Close WS").close());

The Socket.IO server sends ping text messages to check if the client is still alive. Thankfully, this is already implemented in Gatling with wsAutoReplySocketIo4 which automatically replies to a ping TEXT frame with the corresponding pong TEXT frame:

    HttpProtocolBuilder httpProtocol = http
            .wsBaseUrl("ws://localhost:3000")
            .wsAutoReplySocketIo4();

Hopefully, this article will save some time for some of you.

Edit:

Read my second article about Gatling and Socket.IO, this time dealing with a server using a Message Pack parser:

• Load testing Socket.IO servers using msgpack-parser with Gatling

Some useful links

• https://socket.io/docs/v4/socket-io-protocol/
• https://socket.io/docs/v4/troubleshooting-connection-issues/