Open Source RDBMS - Seamless, Scalable, Stable and Free

한국어 | Login |Register

Using nGrinder to perform load test for a socket.io app


nGrinder can not only be used to test the normal web application, but also can be used for JDBC, web service and realtime apps like socket.io provides.

Socket.io aims to make realtime apps possible in every browser and mobile device. With the extensibility of nGrinder, we can test a socket.io based app with the help of extended library. We can use this socket.io-java-client library to do it.

But this library use an async way to send request and receive response. So we need some modification based on the SocketIO class to do it in a synchronized way.

If you are not familiar with the  socket.io-java-client , please go to get the code and check the example in it.

The main idea is, using the SocketIO object to create a connection to the app server, and provide a function to send request to server, and get the response and return. And in this example, I use Java Lock and Condition to archive this.

And below is the source of BlockingSocketIO class:

package my;

import io.socket.IOAcknowledge;
import io.socket.IOCallback;
import io.socket.SocketIO;
import io.socket.SocketIOException;

import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.ReentrantLock;

import org.json.JSONObject;

/**
 * Class description.
 *
 * @author Mavlarn
 * @since
 */
public class BlockingSocketIO implements IOCallback {
    
    private SocketIO socketIO;
    private ReentrantLock transportLock;
    private Condition responseCondition;
    private String respMsg;
    
    public BlockingSocketIO (String url) {
        try {
            transportLock = new ReentrantLock();
            responseCondition = transportLock.newCondition();
            socketIO = new SocketIO(url, this);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
    
    public String sendAndRcv (final String message) {
        try {
            transportLock.lock();
            socketIO.send(message);
            respMsg = null;
            responseCondition.await();
            return respMsg;
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            transportLock.unlock();
        }
        return respMsg;
    }
    
    public String sendAndRcv(final JSONObject json) {
        try {
            transportLock.lock();
            socketIO.send(json);
            respMsg = null;
            responseCondition.await();
            return respMsg;
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            transportLock.unlock();
        }
        return respMsg;
    }

    public String emitAndRcv(String event, final Object args) {
        try {
            transportLock.lock();
            socketIO.emit(event, args);
            respMsg = null;
            responseCondition.await();
            return respMsg;
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            transportLock.unlock();
        }
        return respMsg;
    }

    @Override
    public void onMessage(JSONObject json, IOAcknowledge ack) {
        setResponse(json.toString());
    }

    @Override
    public void onMessage(String data, IOAcknowledge ack) {
        setResponse(data);
    }

    private void setResponse(String data) {
        try {
            transportLock.lock();
            respMsg = data;
            responseCondition.signal();
            System.out.println("Server said:" + data);
        } finally {
            transportLock.unlock();
        }
    }

    @Override
    public void onError(SocketIOException socketIOException) {
        System.out.println("an Error occured");
        socketIOException.printStackTrace();
    }

    @Override
    public void onDisconnect() {
        System.out.println("Connection terminated.");
    }

    @Override
    public void onConnect() {
        System.out.println("Connection established");
    }

    @Override
    public void on(String event, IOAcknowledge ack, Object... args) {
        System.out.println("Server triggered event '" + event + "'");
        setResponse(args[0.toString());
    }

}

 

In this class, I use socket.io-java-client library to send message to socket.io server, and get response with the help of Lock and Condition.

And we need to package this class in jar and upload it into lib folder in nGrinder. You should also upload socketio.jar and its dependency libs WebSocket.jar and json-org.jar.

And next, we need  the python script for test in nGrinder.  It will be as below:

from net.grinder.script.Grinder import grinder
from net.grinder.script import Test

from org.json import JSONObject
from my import BlockingSocketIO

test1 = Test(1, "Test1")

class TestRunner:

    def testSocketIO(self):
        json = JSONObject()
        user = "Thread-%s" % grinder.threadNumber
        json.putOpt("user", user)
        msg = "test message<%s>." % user
        json.putOpt("message", msg)
        grinder.logger.info("msg:" + json.toString())
        respMsg = self.socketIO.emitAndRcv("user message", json)
        return respMsg
    
    def __init__(self):
        grinder.statistics.delayReports=True
        #init socket io
        #create socket io object in thread init function. Then every thread will use its own socket.io connection.
        self.socketIO = BlockingSocketIO("http://127.0.0.1:3000")
        
        #send socket.io server to init user
        json = JSONObject()
        user = "Thread-%s" % grinder.threadNumber
        json.putOpt("username", user)
        self.socketIO.emitAndRcv("user", json)

    # test method        
    def __call__(self):
        resp = self.testSocketIO()

        if "test message" in resp :
            grinder.statistics.forLastTest.success = 1
        else :
            grinder.statistics.forLastTest.success = 0

test1.record(TestRunner.testSocketIO)

 

In this script, in the init function of test runner, we created a socket.op connection object, then all the tests of this thread will use the same connection. It is very important for the long pooling app based on socket.io. And in thie init function, a message with “user” event in sent to server to init the user in server side. And in every test function, we will send a message with “user message” event.

Before we can test this script, we need a server side app. We need node.js installed with the module “socket.io”.

And write a script as below, which is named server.js:

var http = require('http'), io = require('socket.io');

var app = http.createServer();
app.listen(3000);

console.log('Server running at http://127.0.0.1:3000/');

// Socket.IO server
var io = io.listen(app);

io.sockets.on('connection', function (socket) {
  console.log("new connection from" + socket); get and log connection
  socket.on('user message', function (msg) {  //accept a request with “user message” event
    socket.emit('user message processed', {user: msg.user, message: msg.message});
  });

  socket.on('user', function (userMsg) { //accept a request with “user” event, like user login.
    socket.user = userMsg.username;
    socket.emit('user processed', {user: userMsg.user, message: "New user come in."});
  });

  socket.on('disconnect', function () {
    if (!socket.user) return;
    socket.emit('announcement', {user: socket.user, action: 'disconected'});
  });
});

 

And run this simulated server with :

node server.js

You should see a log said the server is running at http://127.0.0.1:3000/.

 

Then, validate this script to make sure it can work properly. The validation result should be as below:

2013-03-11 13:13:08,844 INFO  elapsed time is 17 ms
2013-03-11 13:13:08,844 INFO  Final statistics for this process:
2013-03-11 13:13:08,854 INFO  
             Tests        Errors       Mean Test    Test Time    TPS          
                                       Time (ms)    Standard                  
                                                    Deviation                 
                                                    (ms)                      

Test 1       1            0            3.00         0.00         58.82         "Test1"

Totals       1            0            3.00         0.00         58.82        

  Tests resulting in error only contribute to the Errors column.          
  Statistics for individual tests can be found in the data file, including
  (possibly incomplete) statistics for erroneous tests. Composite tests   
  are marked with () and not included in the totals.                      


……
2013-03-11 13:13:08,750 INFO  validation-0: starting threads
Mar 11, 2013 1:13:08 PM io.socket.IOConnection sendPlain
INFO: > 5:::{"args":{"username":"Thread-0"},"name":"user"}
Mar 11, 2013 1:13:08 PM io.socket.IOConnection transportMessage
INFO: < 1::
Connection established
Mar 11, 2013 1:13:08 PM io.socket.IOConnection transportMessage
INFO: < 5:::{"name":"user processed","args":{"message":"New user come in."}}
Server triggered event 'user processed'
Server said:{"message":"New user come in."}
Mar 11, 2013 1:13:08 PM io.socket.IOConnection sendPlain
INFO: > 5:::{"args":{"message":"test message<Thread-0>.","user":"Thread-0"},"name":"user message"}
Mar 11, 2013 1:13:08 PM io.socket.IOConnection transportMessage
INFO: < 5:::{"name":"user message processed","args":{"user":"Thread-0","message":"test message<Thread-0>."}&a93;}
Server triggered event 'user message processed'
Server said:{"message":"test message<Thread-0>.","user":"Thread-0"}
2013-03-11 13:13:08,855 INFO  validation-0: finished

 

From the result message, we can see that the test is successful, and the server processed 2 request, one is “user”, another is “user message”. For the user name, I use the “Thread-[thread number”we should see the different name if we test with multiple vuser.

The server side log should be similar as this:

debug - client authorized
info - handshake authorized gr0AYzAn7sAKTE_XsORt
debug - setting request GET /socket.io/1/websocket/gr0AYzAn7sAKTE_XsORt
debug - set heartbeat interval for client gr0AYzAn7sAKTE_XsORt
debug - client authorized for 
debug - websocket writing 1::
new connection fromobject Object
debug - websocket writing 5:::{"name":"user processed","args":{"message":"New user come in."}}
debug - websocket writing 5:::{"name":"user message processed","args":{"user":"Thread-0","message":"test message<Thread-0>."}}
info - transport end (socket end)
debug - set close timeout for client gr0AYzAn7sAKTE_XsORt
debug - cleared close timeout for client gr0AYzAn7sAKTE_XsORt
debug - cleared heartbeat interval for client gr0AYzAn7sAKTE_XsORt
debug - discarding transport

The server log said, it got a client connection, and succeed to handshake, and processed 2 request. And at last, the client is disconnected.

 

Then we can create a test with this script to test with nGrinder.

Below is the final report:

NewImage 

At last, don’t forget to check the server log:

   ......
   info  - transport end (socket end)
   debug - set close timeout for client JFrRHYoO3__jN4pdsOSi
   debug - cleared close timeout for client JFrRHYoO3__jN4pdsOSi
   debug - cleared heartbeat interval for client JFrRHYoO3__jN4pdsOSi
   debug - discarding transport
   info  - transport end (socket end)
   debug - set close timeout for client pDHSiLJhTXaqVR5osOSk
   debug - cleared close timeout for client pDHSiLJhTXaqVR5osOSk
   debug - cleared heartbeat interval for client pDHSiLJhTXaqVR5osOSk
   debug - discarding transport
   info  - transport end (socket end)
   debug - set close timeout for client 7u_rypQFSZ2vcTGKsOSj
   debug - cleared close timeout for client 7u_rypQFSZ2vcTGKsOSj
   debug - cleared heartbeat interval for client 7u_rypQFSZ2vcTGKsOSj
   debug - discarding transport
   info  - transport end (socket end)
   debug - set close timeout for client fmxnHFQ_U-wsCmdMsOSg
   debug - cleared close timeout for client fmxnHFQ_U-wsCmdMsOSg
   debug - cleared heartbeat interval for client fmxnHFQ_U-wsCmdMsOSg
   debug - discarding transport

In the server log, there should be some log to say that the connection is discarded. In this test, the vuser is 10, so in the log, there should be 10 times of “discarding transport”. It means that, one vuser is simulated as one actual user with one connection. I you don’t want to use one connection for one thread, you can move the code :

socketIO = BlockingSocketIO("http://127.0.0.1:3000")
to above of TestRunner, then all thread in this process will use the same connection.

 

By the way, the node server is running on my note book. From the TPS and mean time, we can see that the performance of socket.io server is really impressive. Please check the attachment for the java libs used by this test.

 

comments powered by Disqus
Page info
viewed 3655 times
translations en
Author
posted last year by
mavlarn
Contributors
updated 6 months ago by
View revisions
Share this article