ngrinder

Groovy Maven Structure

2013-05-06T18:49:19-07:00

If you choose to use Groovy Script when you create a script, it works simular to Jython script except the JUnit styled test cases.
However if you want not only to use Groovy but also to execute JUnit runner in your IDE or easily configure dependencies, you can choose to make Groovy Maven Project first.

If you create Groovy Maven Project, you can have the following directories containing three files under the specified project folder.

Location	Description
${name}/pom.xml	Maven project file. You can add the library dependencies in this file.
${name}/src/main/java/Test1.groovy	Default groovy script file
${name}/src/main/java/resources1.txt	Default resource file
${name}/lib	Not automatically generated. but this folder can be used to distribute private libraries if available.

You can import this project into you IDE using SVN checkout. You may need to check the below links.

[Install Groovy IDE]
[Import Groovy Maven Project in IDE]

In pom.xml, You can specify the additional library dependencies.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>ngrinder</groupId>
    <artifactId>${name}</artifactId>
    <version>0.0.1</version>

    <properties>
        <jdk-version>1.6</jdk-version>
        <source-encoding>UTF-8</source-encoding>
    </properties>

    <repositories>
        <repository>
            <id>nhnopensource.maver.repo</id>
            <url>https://github.com/nhnopensource/nhnopensource.maven.repo/raw/master/releases</url>
        </repository>
    </repositories>

    <dependencies>
        <!-- Following dependency is mandatory -->
        <dependency>
            <groupId>org.ngrinder</groupId>
            <artifactId>ngrinder-groovy</artifactId>
            <version>3.2</version>
            <scope>provided</scope>
        </dependency>

        <!-- Add the dependencies -->
        <!--  <dependency>
            <groupId>commons-io</groupId>
            <artifactId>commons-io</artifactId>
            <version>2.0.1</version>
        </dependency>
         -->
        <!-- Put your private library like this -->
        <!--  
        <dependency>
            <groupId>your_lib</groupId>
            <artifactId>your_lib</artifactId>
            <version>your_lib_version</version>
            <scope>system</scope>
            <systemPath>${project.basedir}/lib/hello.jar</systemPath>
        </dependency>
        -->
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>${jdk-verion}</source>
                    <target>${jdk-version}</target>
                    <encoding>${source-encoding}</encoding>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-eclipse-plugin</artifactId>
                <version>2.9</version>
                <configuration>
                    <additionalProjectnatures>
                        <projectnature>
                            org.eclipse.jdt.groovy.core.groovyNature
                        </projectnature>
                        <projectnature>
                            org.eclipse.m2e.core.maven2Nature
                        </projectnature>
                    </additionalProjectnatures>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

If you like to add the private libraries which do not exist in the public maven repository, you can put them in the lib folder directly and specify the systemPath option like the above sample. lib folder will be automatically detected and copied into agents as well when controller prepare the script distribution.
When a script is selected in the performance test configuration, controller automatically detects whether the script is under the folder ${name}/src/main/java and there is pom.xml in the base folder. Then controller will copy the files in the ${name}/src/main/resources and ${name}/src/main/java folder recursively into controller’s distribution folder before distributing them to agents. Below shows how each folders in svn are copied to the distribution folder.

- from svn foler + pom.xml
                 + src/main/java/Test1.groovy
                                /package_names/the_other_groovyscripts
                 + src/main/resources/resources1.txt
                                     /subfolder_names/the_other_resources
                 + lib (private libraris)


- to dist folder + Test1.groovy 
                 + resources1.txt 
                 + package_names/the_other_groovy_scripts 
                 + subfolder_names/the_other_resources 
                 + lib (copied from maven dependencies and files in lib folder existing subversion)

In generic Jython script and Groovy script, the resources are loaded using command "open(“./resources/resource1.txt”)". However this won’t work in Groovy Maven project. We had to change this path based resource discovery into the classpath based resource discovery to make Groovy JUnit test workable in IDE as well,
All resources in ${project_name}/src/main/resources/.. are copied into the same folder where the test script is copied but keeping the sub directory hierachy. You can load the resources using code below.

import org.codehaus.groovy.reflection.ReflectionUtils;
....


class YourTest {
    String text;

    @BeforeThread
    public void beforeThread() {
       // In groovy, InputStream contains text field.
       text = loadResourceFromClassPath("/resource1.txt").text;
    }

    @Test
    public void doTest() {
       ....
    }

    // This is groovy way to load resource from classpath
    public loadResourceFromClassPath(String resourcePath) {
        return ReflectionUtils.getCallingClass(0).getResourceAsStream(resourcePath);
    }
}

How to use library

2012-11-20T21:40:22-08:00

Jython and Groovy

Some guy may want to use java libraries or python py modules in the scripts. For this case, nGrinder supports upload of these files and a user can use external libraries without any system configuration. Just upload the jar or py files in the lib folder on the same location of the test script file. The files in the lib folder will be automatically passed to agents and be located in the agent classpath when the test is executed.

Then You can just refer the uploaded jar or py like followings..

from net.grinder.script import Test
from net.grinder.script.Grinder import grinder
from org.json.simple import JSONValue
from java.util import Random
from java.lang import String
# This library is located in the lib folder.
from mpcCaller import NpcCaller

random = Random()Z

caller = NpcCaller("xx.xx.xx.15", 7090);
module = "tchat"
procedure = "chatBO/RelayCommand"…

Groovy Maven Project

If you're using Groovy Maven Project you can easily add the external libraries in the pom.xml. Please refer [Groovy Maven Structure]

Jython Script Structure

2013-05-06T18:26:57-07:00

nGrinder uses The Grinder Jython Script with no modification. Following is the jython template nGrinder uses.

# -*- coding:utf-8 -*-

# A simple example using the HTTP plugin that shows the retrieval of a
# single page via HTTP. 
#
# This script is auto generated by ngrinder.
#
# @author ${userName}
from net.grinder.script.Grinder import grinder
from net.grinder.script import Test
from net.grinder.plugin.http import HTTPRequest
from net.grinder.plugin.http import HTTPPluginControl


control = HTTPPluginControl.getConnectionDefaults()
# if you don't want that HTTPRequest follows the redirection, please modify the following option 0.
# control.followRedirects = 1
# if you want to increase the timeout, please modify the following option.
control.timeout = 6000

test1 = Test(1, "mytest")
request1 = HTTPRequest()

# Make any method call on request1 increase TPS
test1.record(request1)

class TestRunner:
    # initlialize a thread 
    def __init__(self):
        grinder.statistics.delayReports=True
        pass

    # test method        
    def __call__(self):
        result = request1.GET("http://www.google.com”)
        
        # You get the message body using the getText() method.
        # if result.getText().find("HELLO WORLD") != -1 :
        #    grinder.statistics.forLastTest.success = 1
        # else :
        #     grinder.statistics.forLastTest.success = 0
            
        # if you want to print out log.. 
        # Don't use print keyword. 
        # instead use following.
        # grinder.logger.info("Hello World")
        

        if result.getStatusCode() == 200 :
            grinder.statistics.forLastTest.success = 1
        elif result.getStatusCode() in (301, 302) :
            grinder.logger.warn("Warning. The response may not be correct. The response code was %d." %  result.getStatusCode()) 
            grinder.statistics.forLastTest.success = 1
        else :
            grinder.statistics.forLastTest.success = 0

The each process in the Grinder jython scriptengine reads the whole script and each thread executes the TestRunner’s __call__ method.
This execution can be illustrated by the following figure.

When JythonScript engine creates processes, each process run the global scope script first. In above template, the following is the gloabl scope script which is executed once per process.

control = HTTPPluginControl.getConnectionDefaults()
# if you don't want that HTTPRequest follows the redirection, please modify the following option 0.
# control.followRedirects = 1
# if you want to increase the timeout, please modify the following option.
control.timeout = 6000

test1 = Test(1, "mytest”)
request1 = HTTPRequest()

# Make any method call on request1 increase TPS
test1.record(request1)

In the above part, the test statistics are prepared and request1 object is instrumented to record any methods call on request1 object.
Then the multiple threads are created and each thread creates a TestRunner object by running the __init__ method.
__init__ method is subject to be invoked once per thread. Therefore, this method is the perfect place to login if your target system needs the login.

class TestRunner:
    # initlialize a thread 
    def __init__(self):
        grinder.statistics.delayReports=True
        pass

After creating TestRunner object, each thread continuously invokes __call__() method on the created TestRunner object until the test is finished.

    # test method        
    def __call__(self):
        result = request1.GET(http://www.google.com)
        
        # You get the message body using the getText() method.
        # if result.getText().find("HELLO WORLD") != -1 :
        #    grinder.statistics.forLastTest.success = 1
        # else :
        #     grinder.statistics.forLastTest.success = 0
            
        # if you want to print out log.. 
        # Don't use print keyword. 
        # instead use following.
        # grinder.logger.info("Hello World")
        

        if result.getStatusCode() == 200 :
            grinder.statistics.forLastTest.success = 1
        elif result.getStatusCode() in (301, 302) :
            grinder.logger.warn("Warning. The response may not be correct. The response code was %d." %  result.getStatusCode()) 
            grinder.statistics.forLastTest.success = 1
        else :
            grinder.statistics.forLastTest.success = 0

When the agent gets the stop message from controller or specified run count is reached, each thread is terminated. Before thread terminations, thread invokes __del__(self) method to clean up the test. If you don’t provide this method in the test script, it’s just ignored.
When starting a test in the controller, not only script but also following folders which exists at the same folder as the script is located are transfered to the agents.

All transfered library files like .class, .py, .jar located in the lib folder are automatically are set a library path. Therefore you can extend the test script to use 3rd party library. In addition, you can locate any resources in the resources folder. The resources can be opened by calling

open("./resources/resource_file_name.txt")

Please check out [How to use library] and [How to use resources] for detailed instruction.

However, from nGrinder 3.2, we start to support Groovy as a scripting language. We believe this is much powerful way. Check out [Groovy Script Structure] as well for the detailed instruction.

Groovy Script

2013-05-05T20:26:38-07:00

nGrinder 3.2 supports Groovy as scripting language. It’s not a just replacement of jython but also new test script development model. When you decide to write scripts with groovy in maven structure, you can reuse the exiting full development experiences from JUnit and Eclipse development enviornments.

[Groovy Script Structure]
[Groovy Maven Structure]
[Install Groovy IDE]
[Import Groovy Maven Project in IDE]

Groovy is not only convenient to write but also efficient to execute. Our benchmark result shows that the Groovy script which does same job as Jython script can have 2 times more virtual user count(especially thread count) per agent.

Using nGrinder to perform load test for a socket.io app

2013-03-10T21:31:59-07:00

nGrinder can not only be used to test the normal web application, but also can be used for JDBC, web service and realtime apps like socket.io provides.

Socket.io aims to make realtime apps possible in every browser and mobile device. With the extensibility of nGrinder, we can test a socket.io based app with the help of extended library. We can use this socket.io-java-client library to do it.

But this library use an async way to send request and receive response. So we need some modification based on the SocketIO class to do it in a synchronized way.

If you are not familiar with the socket.io-java-client , please go to get the code and check the example in it.

The main idea is, using the SocketIO object to create a connection to the app server, and provide a function to send request to server, and get the response and return. And in this example, I use Java Lock and Condition to archive this.

And below is the source of BlockingSocketIO class:

package my;

import io.socket.IOAcknowledge;
import io.socket.IOCallback;
import io.socket.SocketIO;
import io.socket.SocketIOException;

import java.util.concurrent.locks.Condition;
import java.util.concurrent.locks.ReentrantLock;

import org.json.JSONObject;

/**
 * Class description.
 *
 * @author Mavlarn
 * @since
 */
public class BlockingSocketIO implements IOCallback {
    
    private SocketIO socketIO;
    private ReentrantLock transportLock;
    private Condition responseCondition;
    private String respMsg;
    
    public BlockingSocketIO (String url) {
        try {
            transportLock = new ReentrantLock();
            responseCondition = transportLock.newCondition();
            socketIO = new SocketIO(url, this);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
    
    public String sendAndRcv (final String message) {
        try {
            transportLock.lock();
            socketIO.send(message);
            respMsg = null;
            responseCondition.await();
            return respMsg;
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            transportLock.unlock();
        }
        return respMsg;
    }
    
    public String sendAndRcv(final JSONObject json) {
        try {
            transportLock.lock();
            socketIO.send(json);
            respMsg = null;
            responseCondition.await();
            return respMsg;
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            transportLock.unlock();
        }
        return respMsg;
    }

    public String emitAndRcv(String event, final Object args) {
        try {
            transportLock.lock();
            socketIO.emit(event, args);
            respMsg = null;
            responseCondition.await();
            return respMsg;
        } catch (Exception e) {
            e.printStackTrace();
        } finally {
            transportLock.unlock();
        }
        return respMsg;
    }

    @Override
    public void onMessage(JSONObject json, IOAcknowledge ack) {
        setResponse(json.toString());
    }

    @Override
    public void onMessage(String data, IOAcknowledge ack) {
        setResponse(data);
    }

    private void setResponse(String data) {
        try {
            transportLock.lock();
            respMsg = data;
            responseCondition.signal();
            System.out.println("Server said:" + data);
        } finally {
            transportLock.unlock();
        }
    }

    @Override
    public void onError(SocketIOException socketIOException) {
        System.out.println("an Error occured");
        socketIOException.printStackTrace();
    }

    @Override
    public void onDisconnect() {
        System.out.println("Connection terminated.");
    }

    @Override
    public void onConnect() {
        System.out.println("Connection established");
    }

    @Override
    public void on(String event, IOAcknowledge ack, Object... args) {
        System.out.println("Server triggered event '" + event + "'");
        setResponse(args[[0].toString());
    }

}

In this class, I use socket.io-java-client library to send message to socket.io server, and get response with the help of Lock and Condition.

And we need to package this class in jar and upload it into lib folder in nGrinder. You should also upload socketio.jar and its dependency libs WebSocket.jar and json-org.jar.

And next, we need the python script for test in nGrinder. It will be as below:

from net.grinder.script.Grinder import grinder
from net.grinder.script import Test

from org.json import JSONObject
from my import BlockingSocketIO

test1 = Test(1, "Test1")

class TestRunner:

    def testSocketIO(self):
        json = JSONObject()
        user = "Thread-%s" % grinder.threadNumber
        json.putOpt("user", user)
        msg = "test message<%s>." % user
        json.putOpt("message", msg)
        grinder.logger.info("msg:" + json.toString())
        respMsg = self.socketIO.emitAndRcv("user message", json)
        return respMsg
    
    def __init__(self):
        grinder.statistics.delayReports=True
        #init socket io
        #create socket io object in thread init function. Then every thread will use its own socket.io connection.
        self.socketIO = BlockingSocketIO("http://127.0.0.1:3000")
        
        #send socket.io server to init user
        json = JSONObject()
        user = "Thread-%s" % grinder.threadNumber
        json.putOpt("username", user)
        self.socketIO.emitAndRcv("user", json)

    # test method        
    def __call__(self):
        resp = self.testSocketIO()

        if "test message" in resp :
            grinder.statistics.forLastTest.success = 1
        else :
            grinder.statistics.forLastTest.success = 0

test1.record(TestRunner.testSocketIO)

In this script, in the init function of test runner, we created a socket.op connection object, then all the tests of this thread will use the same connection. It is very important for the long pooling app based on socket.io. And in thie init function, a message with “user” event in sent to server to init the user in server side. And in every test function, we will send a message with “user message” event.

Before we can test this script, we need a server side app. We need node.js installed with the module “socket.io”.

And write a script as below, which is named server.js:

var http = require('http'), io = require('socket.io');

var app = http.createServer();
app.listen(3000);

console.log('Server running at http://127.0.0.1:3000/');

// Socket.IO server
var io = io.listen(app);

io.sockets.on('connection', function (socket) {
  console.log("new connection from" + socket); get and log connection
  socket.on('user message', function (msg) {  //accept a request with “user message” event
    socket.emit('user message processed', {user: msg.user, message: msg.message});
  });

  socket.on('user', function (userMsg) { //accept a request with “user” event, like user login.
    socket.user = userMsg.username;
    socket.emit('user processed', {user: userMsg.user, message: "New user come in."});
  });

  socket.on('disconnect', function () {
    if (!socket.user) return;
    socket.emit('announcement', {user: socket.user, action: 'disconected'});
  });
});

And run this simulated server with :

node server.js

You should see a log said the server is running at http://127.0.0.1:3000/.

Then, validate this script to make sure it can work properly. The validation result should be as below:

2013-03-11 13:13:08,844 INFO  elapsed time is 17 ms
2013-03-11 13:13:08,844 INFO  Final statistics for this process:
2013-03-11 13:13:08,854 INFO  
             Tests        Errors       Mean Test    Test Time    TPS          
                                       Time (ms)    Standard                  
                                                    Deviation                 
                                                    (ms)                      

Test 1       1            0            3.00         0.00         58.82         "Test1"

Totals       1            0            3.00         0.00         58.82        

  Tests resulting in error only contribute to the Errors column.          
  Statistics for individual tests can be found in the data file, including
  (possibly incomplete) statistics for erroneous tests. Composite tests   
  are marked with () and not included in the totals.                      


……
2013-03-11 13:13:08,750 INFO  validation-0: starting threads
Mar 11, 2013 1:13:08 PM io.socket.IOConnection sendPlain
INFO: > 5:::{"args":[{"username":"Thread-0"}],"name":"user"}
Mar 11, 2013 1:13:08 PM io.socket.IOConnection transportMessage
INFO: < 1::
Connection established
Mar 11, 2013 1:13:08 PM io.socket.IOConnection transportMessage
INFO: < 5:::{"name":"user processed","args":[{"message":"New user come in."}]}
Server triggered event 'user processed'
Server said:{"message":"New user come in."}
Mar 11, 2013 1:13:08 PM io.socket.IOConnection sendPlain
INFO: > 5:::{"args":[{"message":"test message<Thread-0>.","user":"Thread-0"}],"name":"user message"}
Mar 11, 2013 1:13:08 PM io.socket.IOConnection transportMessage
INFO: < 5:::{"name":"user message processed","args":[{"user":"Thread-0","message":"test message<Thread-0>."}&a93;}
Server triggered event 'user message processed'
Server said:{"message":"test message<Thread-0>.","user":"Thread-0"}
2013-03-11 13:13:08,855 INFO  validation-0: finished

From the result message, we can see that the test is successful, and the server processed 2 request, one is “user”, another is “user message”. For the user name, I use the “Thread-[thread number]”we should see the different name if we test with multiple vuser.

The server side log should be similar as this:

debug - client authorized
info - handshake authorized gr0AYzAn7sAKTE_XsORt
debug - setting request GET /socket.io/1/websocket/gr0AYzAn7sAKTE_XsORt
debug - set heartbeat interval for client gr0AYzAn7sAKTE_XsORt
debug - client authorized for 
debug - websocket writing 1::
new connection from[object Object]
debug - websocket writing 5:::{"name":"user processed","args":[{"message":"New user come in."}]}
debug - websocket writing 5:::{"name":"user message processed","args":[{"user":"Thread-0","message":"test message<Thread-0>."}]}
info - transport end (socket end)
debug - set close timeout for client gr0AYzAn7sAKTE_XsORt
debug - cleared close timeout for client gr0AYzAn7sAKTE_XsORt
debug - cleared heartbeat interval for client gr0AYzAn7sAKTE_XsORt
debug - discarding transport

The server log said, it got a client connection, and succeed to handshake, and processed 2 request. And at last, the client is disconnected.

Then we can create a test with this script to test with nGrinder.

Below is the final report:

At last, don’t forget to check the server log:

   ......
   info  - transport end (socket end)
   debug - set close timeout for client JFrRHYoO3__jN4pdsOSi
   debug - cleared close timeout for client JFrRHYoO3__jN4pdsOSi
   debug - cleared heartbeat interval for client JFrRHYoO3__jN4pdsOSi
   debug - discarding transport
   info  - transport end (socket end)
   debug - set close timeout for client pDHSiLJhTXaqVR5osOSk
   debug - cleared close timeout for client pDHSiLJhTXaqVR5osOSk
   debug - cleared heartbeat interval for client pDHSiLJhTXaqVR5osOSk
   debug - discarding transport
   info  - transport end (socket end)
   debug - set close timeout for client 7u_rypQFSZ2vcTGKsOSj
   debug - cleared close timeout for client 7u_rypQFSZ2vcTGKsOSj
   debug - cleared heartbeat interval for client 7u_rypQFSZ2vcTGKsOSj
   debug - discarding transport
   info  - transport end (socket end)
   debug - set close timeout for client fmxnHFQ_U-wsCmdMsOSg
   debug - cleared close timeout for client fmxnHFQ_U-wsCmdMsOSg
   debug - cleared heartbeat interval for client fmxnHFQ_U-wsCmdMsOSg
   debug - discarding transport

In the server log, there should be some log to say that the connection is discarded. In this test, the vuser is 10, so in the log, there should be 10 times of “discarding transport”. It means that, one vuser is simulated as one actual user with one connection. I you don’t want to use one connection for one thread, you can move the code :

socketIO = BlockingSocketIO("http://127.0.0.1:3000")

to above of TestRunner, then all thread in this process will use the same connection.

By the way, the node server is running on my note book. From the TPS and mean time, we can see that the performance of socket.io server is really impressive. Please check the attachment for the java libs used by this test.

How to run the multiple tests with different weight

2013-01-07T20:44:51-08:00

The real user is not simple. There might have multiple usage scenarios for the target system. If you want to simulate this on The Grinder script, you may need to run multiple tests with different weight.

With following The Grinder script technique, it’s possible to simulate this case. You can upload the each scenarios in the lib folder(refer [How to use library]) and create the test script which control each test scenario like following.

Jython

# Run test scripts in parallel
# Run TestScript1 in 50% of threads, TestScript2 in 30% of threads,
# and TestScript3 in 20% of threads.

from net.grinder.script.Grinder import grinder

 
# You can locate Scenario1,2,3.py files in the lib folder.
scripts = 〔"Scenario1", "Scenario2", "Scenario2"〕


# Ensure modules are initialised in the process thread.
for script in scripts: exec("import %s" % script)

def createTestRunner(script):
    exec("x = %s.TestRunner()" % script)
    return x

class TestRunner:
    def __init__(self):
        tid = grinder.threadNumber
        s = tid % 10
        if s in range(1,5):
            self.testRunner = createTestRunner(scripts〔0〕)
        elif s in range(6,8):
            self.testRunner = createTestRunner(scripts〔1〕)
        else:
            self.testRunner = createTestRunner(scripts〔2〕)

    # This method is called for every run.
    def __call__(self):
        self.testRunner()

Groovy

	@Test
	public void test(){
		switch (grinder.threadNumber % 10) {
			case 1..5:
				doTest1()
				break
			case 6..8:
				doTest2()
				break
			default :
				doTest3()
				break
		}
	}

	public void doTest1() {
		...
	}

	public void doTest2() {
		...
	}

	public void doTest3() {
		...
	}

User defined statistic in nGrinder

2013-02-21T22:57:06-08:00

Maybe you know that "The Grinder" supports user defined statistic. If you define and put some value in the script, then this filed can be saved in data log or summary data. You can check the example script for this from:

http://grinder.sourceforge.net/g3/script-gallery.html#jmsreceiver.py

But in ngrinder, in previous 3.1.2 version, you can not see that data in test report. But from version 3.1.2, you can see that data in report chart.

The sample script is below:

Jython

# -*- coding:utf-8 -*-

# A simple example using the HTTP plugin that shows the retrieval of a
# single page via HTTP. 
#
# This script is auto generated by ngrinder.
#
# @author admin
from net.grinder.script.Grinder import grinder
from net.grinder.script import Test
from net.grinder.plugin.http import HTTPRequest
from net.grinder.plugin.http import HTTPPluginControl
from java.util import Random

control = HTTPPluginControl.getConnectionDefaults()
# if you don't want that HTTPRequest follows the redirection, please modify the following option 0.
# control.followRedirects = 1
# if you want to increase the timeout, please modify the following option.
control.timeout = 6000

test1 = Test(1, "Test1")
request1 = HTTPRequest()

#grinder.statistics.registerDataLogExpression("Delivery time", "userLong0")
grinder.statistics.registerSummaryExpression("User_defined", "(/ userLong0(+ (count timedTests)))")
                        
# Make any method call on request1 increase TPS
test1.record(request1)
random = Random()

class TestRunner:
	# initlialize a thread 
	def __init__(self):
		grinder.statistics.delayReports=True
		pass

	# test method		
	def __call__(self):
		result = request1.GET("http://www.naver.com")
		
		# do something
		deliveryTime = random.nextInt(1000)
		grinder.sleep(deliveryTime)
		grinder.logger.info("deliveryTime: %d" % deliveryTime)
		grinder.statistics.forLastTest.setLong("userLong0", deliveryTime)
		
		if result.getStatusCode() == 200 :
			grinder.statistics.forLastTest.success = 1
		elif result.getStatusCode() in (301, 302) :
			grinder.logger.warn("Warning. The response may not be correct. The response code was %d." %  result.getStatusCode()) 
			grinder.statistics.forLastTest.success = 1
		else :
			grinder.statistics.forLastTest.success = 0

Groovy

@RunWith(GrinderRunner)
class Test1 {

	public static GTest test;
	public static HTTPRequest request;

	@BeforeProcess
	public static void beforeClass() {
		test = new GTest(1, "aa000000");
		// Register a custom statistics
		grinder.statistics.registerSummaryExpression("User_defined", "(/ userLong0(+ (count timedTests)))")
		request = new HTTPRequest();
		test.record(request);
		grinder.logger.info("before process.");
	}
	
	Random random = new Random();
	
	@BeforeThread
	public void beforeThread() {
		grinder.statistics.delayReports=true;
		grinder.logger.info("before thread.");
	}


	@Test
	public void test(){
		HTTPResponse result = request.GET("http://www.google.com");
		def deliveryTime = random.nextInt(1000)
		grinder.sleep(deliveryTime)
		grinder.logger.info("deliveryTime: ${deliveryTime}")
		// Update statistics
		grinder.statistics.forLastTest.setLong("userLong0", deliveryTime)
		
		if (result.statusCode == 301 || result.statusCode == 302) {
			grinder.logger.warn("Warning. The response may not be correct. The response code was {}.", result.statusCode);
		} else {
			assertThat(result.statusCode, is(200));
		}
	}

Currently, only one chart can be displayed for the user defined statistic, and the name of this statistic should be “User_defined”. For more detailed description of usage, you can refer the document of grinder: registerSummaryExpression. After running a test with this script, you can see the chart for user defined statistics in the detailed report.

How to handle the big response

2013-02-24T18:11:23-08:00

Some performance testing script may return a significant amount of response. This situation occurs when you try to download the file and test video in the script especially. Because nGrinder is configured to use at maximum 1GB memory per one process, the each thread can use 100MB if 10 threads per a process are invoked. Considering the variety of other overhead, the maximum response nGrinder can handle might be about 50 mega per request. if the each response is over this amount, nGrinder agent might cause OOM error.

The easiest way to solve this problem is to reduce the number of threads and to increase the number of processes. Instead of this solution, you can make script not to read whole response to save memory if the whole response read is not necessary. Please refer to the following code.

Jython

from jarray import zeros
 
 
request1 = HTTPRequest()
# set this not to read whole response when the GET/POST is called.
request1.setReadResponseBody(False)
# Make any method call on request1 increase TPS
test1.record(request1)
 
 
class TestRunner:
    # initlialize a thread 
    def __init__(self):
        grinder.statistics.delayReports=True
        pass
 
    # test method        
    def __call__(self):
        result = request1.GET("http://www.naver.com")
        # create a buffer.. 
        buf = zeros(1024, 'b')
        # read only 1024 byte from input stream 
        stream = result.getInputStream();
        stream.read(buf)
        stream.close()
 
......

Groovy

@RunWith(GrinderRunner)
class Test1 {

	public static GTest test;
	public static HTTPRequest request;

	@BeforeProcess
	public static void beforeClass() {
		test = new GTest(1, "aa000000");
		request = new HTTPRequest();
		// Make not to read whole response.
		request.setReadResponseBody(false);
		test.record(request);
		grinder.logger.info("before process.");
	}
	
	// Prepare buffer
	byte[] buffer = new byte[1000];

	@BeforeThread
	public void beforeThread() {
		grinder.statistics.delayReports=true;
		grinder.logger.info("before thread.");
	}


	@Test
	public void test(){
		HTTPResponse result = request.GET("http://www.google.com");

		// Read only 1000 byte and close the stream
		def stream = result.getInputStream();
		stream.read(buffer);
		stream.close();

		if (result.statusCode == 301 || result.statusCode == 302) {
			grinder.logger.warn("Warning. The response may not be correct. The response code was {}.", result.statusCode);
		} else {
			assertThat(result.statusCode, is(200));
		}
	}
}

How to use resources

2013-01-07T19:37:43-08:00

Jython and generic Groovy

Some users want to use txt or xml files to externalize the resources from scripts. For example, if you have a bunch of user ids and need to test some system with those ids, keeping all ids in the script is not realistic. Instead, you can use resources files which nGrinder supports. Only following file type are distributable

json, csv, txt, properties

To use resources files, you can create resources folder at the same location with the script.

The resources folder containing multiple resource files will be passed to the agents during the test execution. In the script you can refer the these resources like following code.

Jython

from net.grinder.script import Test
from net.grinder.script.Grinder import grinder
from net.grinder.plugin.http import HTTPRequest                                                             

test1 = Test(1, "Test1")
test_host = "http://www.sample.com"
request1 = test1.wrap(HTTPRequest(url=test_host))

test_query_file = "./resources/rows_10.txt"


class TestRunner:
    def __init__(self):
        self.urlCount = 0
        self.urlList = []
        
        try:
            fpQuery = open(test_query_file, "r") 
            ....

Generic Groovy

Generic groovy without maven project structure does exactly same resource action as Jython. You can get the resource using new File("./resources/resource_name"). Please notice that File object in Groovy contains text field for easy file manipulation.

String text
	@BeforeThread
	public void beforeThread() {
		grinder.statistics.delayReports=true;
		grinder.logger.info("before thread.");
		text = new File("./resources/resource1.txt").text;
	}
	
	
	@Test
	public void test(){
		println(text)
		HTTPResponse result = request.GET("http://www.google.com");
		if (result.statusCode == 301 || result.statusCode == 302) {
			grinder.logger.warn("Warning. The response may not be correct. The response code was {}.", result.statusCode);
		} else {
			assertThat(result.statusCode, is(200));
		}
	}

Groovy Maven Project

However the above method does not work if you're using Groovy Maven Project.

It's because the resource discovery is changed to classpath based in Groovy Maven Project.

Please refer [Groovy Maven Structure] to know how to open resources.

import org.codehaus.groovy.reflection.ReflectionUtils;
....
 
 
class YourTest {
    String text;
 
    @BeforeThread
    public void beforeThread() {
       // In groovy, InputStream contains text field.
       text = loadResourceFromClassPath("/resource1.txt").text;
    }
 
    @Test
    public void doTest() {
       ....
    }
 
    // This is groovy way to load resource from classpath
    public loadResourceFromClassPath(String resourcePath) {
        return ReflectionUtils.getCallingClass(0).getResourceAsStream(resourcePath);
    }
}

How to parse XML

2013-02-05T17:18:33-08:00

Sometimes it’s necessary to analyze data returned from server and put the value to another request. Actually, we don’t recommend to use “Parsing” because it’s very expensive processing and eventually decrease the whole agent performance. It’s always better to use regular expression to extract some data from the returned data. However if you really need to parse XML, you can do it easily.

Jython

You can use ElementTree parser which nGrinder 3.1.1 and above has.

from xml.etree import ElementTree
xmlstring = """
<hello>
    <world>
        WOW
    </world>
    <my>
        <ngrinder>
            GOOD!!
        </ngrinder>
    </my>
</hello>"""
mydom = ElementTree.fromstring(xmlstring)
# XPATH
grinder.logger.info(mydom.findall([0].text)

The result will be “GOOD"!!”. Please refer http://docs.python.org/2/library/xml.etree.elementtree.html for the detailed API usage.

Groovy

def message = """ 
    <hello>
    <world>
        WOW
    </world>
    <world a="wow_attr">
        WOW2
    </world>
    <my>
        <ngrinder>
            GOOD!!
        </ngrinder>
    </my>
</hello>"""
def hello = new XmlParser().parseText(message);
grinder.logger.info(hello.world[1].text())
grinder.logger.info(hello.world[1].'@a')

This will print "WOW2" and "wow_attr". Please refer the detailed API usage

How to parse JSON

2013-02-03T17:55:22-08:00

nGrinder 3.1.1 contains json.jar, one of most famous Json parser(provided from www.json.org). You can easily parse json string with this library.
You can use this like the following.

JYTHON

from org.json import JSONObject 

message = """ 
{ 
  "glossary": { 
     "title": "example glossary", 
        "GlossDiv": { 
           "title": "S", 
           "GlossList": { 
              "GlossEntry": { 
                 "ID": "SGML", 
                 "SortAs": "SGML", 
                 "GlossTerm": "Standard Generalized Markup Language", 
                 "Acronym": "SGML", 
                 "Abbrev": "ISO 8879:1986", 
                 "GlossDef": { 
                    "para": "A meta-markup language, used to create markup languages such as DocBook."
                 }, 
                 "GlossSee": "markup" 
            } 
         } 
      } 
  } 
} 
""" 

json = JSONObject(message) 
grinder.logger.info(json.get("glossary").getString("title"))

GROOVY

For groovy you can use JsonSlurper instead.

import groovy.json.JsonSlurper


....
def message = """ 
{ 
  "glossary": { 
     "title": "example glossary", 
        "GlossDiv": { 
           "title": "S", 
           "GlossList": { 
              "GlossEntry": { 
                 "ID": "SGML", 
                 "SortAs": "SGML", 
                 "GlossTerm": "Standard Generalized Markup Language", 
                 "Acronym": "SGML", 
                 "Abbrev": "ISO 8879:1986", 
                 "GlossDef": { 
                    "para": "A meta-markup language, used to create markup languages such as DocBook."
                 }, 
                 "GlossSee": "markup" 
            } 
         } 
      } 
  } 
}"""
def jsonMsg = new JsonSlurper().parseText(message)
grinder.logger.info(jsonMsg.glossary.title)

How to adjust logger level

2013-01-07T18:14:22-08:00

Whenever a script calls the HTTP GET/POST method, each HTTP call is recorded at the log files in the agent's side. A log files are created per process and threads belonging to a process use a single log file. Logs can be huge(Giga byte size) depending on how many times the tests are executed and how many errors are occurred. However, nGrinder users need to check these logs to resolve their script problem which occurs in the agents. nGrinder solves this by limiting the maximum size of log file and passing only one process log to the controller. If the log file goes up to 1MB, nGrinder overwrites the existing log to reduce the logger overhead.

However, It might result the problem when a user tries to find out the error causes by checking agent logs. However, what if a user ran a test more than 1,000,000 times and 5 errors were occurred? Logs for 5 errors might have been overwritten by log rotation so the user can not find the clue for their problem,

For this case, you need to adjust log level to ERROR to focus on real error.

Jython

# -*- coding:utf-8 -*-
 
# A simple example using the HTTP plugin that shows the retrieval of a
# single page via HTTP. 
#
# @author admin
from net.grinder.script.Grinder import grinder
from net.grinder.script import Test
from net.grinder.plugin.http import HTTPRequest
from org.slf4j import LoggerFactory
from ch.qos.logback.classic import Level
from ch.qos.logback.classic import Logger 
 
test1 = Test(1, "Test1")
request1 = test1.wrap(HTTPRequest())
 
class TestRunner:
    def __init__(self) :
        logger = LoggerFactory.getLogger("worker");
        logger.setLevel(Level.ERROR);
        
    def __call__(self):
        grinder.statistics.delayReports=True
        result = request1.GET("http://sample.com")
 
       
        if result.getStatusCode() == 200 :
            grinder.statistics.forLastTest.success = 1
        else :
            grinder.logger.error(result.getStatusCode())
            grinder.statistics.forLastTest.success = 0

Groovy

import static net.grinder.script.Grinder.grinder
import static org.junit.Assert.*
import static org.hamcrest.Matchers.*
import net.grinder.plugin.http.HTTPRequest
import net.grinder.script.GTest
import net.grinder.script.Grinder
import net.grinder.scriptengine.groovy.junit.GrinderRunner
import net.grinder.scriptengine.groovy.junit.annotation.BeforeThread
import net.grinder.scriptengine.groovy.junit.annotation.BeforeProcess

import org.apache.commons.io.IOUtils;
import org.codehaus.groovy.reflection.ReflectionUtils;
import org.junit.BeforeClass
import org.junit.Test
import org.junit.runner.RunWith
import org.slf4j.LoggerFactory;

import ch.qos.logback.classic.Level;

import HTTPClient.HTTPResponse

/**
 * A simple example using the HTTP plugin that shows the retrieval of a
 * single page via HTTP.
 *
 * This script is auto generated by ngrinder.
 *
 * @author ${userName}
 */
@RunWith(GrinderRunner)
class Test1 {

	public static GTest test;
	public static HTTPRequest request;

	@BeforeProcess
	public static void beforeClass() {
		test = new GTest(1, "test");
		request = new HTTPRequest();
		test.record(request);
		grinder.logger.info("before process.");
	}

	
	@BeforeThread
	public void beforeThread() {
                // Adjust logger to error level
		LoggerFactory.getLogger("worker").setLevel(Level.ERROR)
		grinder.statistics.delayReports=true;
		grinder.logger.info("before thread.");
	}
	
	
	@Test
	public void test(){
		HTTPResponse result = request.GET("http://www.google.com");
		if (result.statusCode == 301 || result.statusCode == 302) {
			grinder.logger.warn("Warning. The response may not be correct. The response code was {}.", result.statusCode);
		} else {
			assertThat(result.statusCode, is(200));
		}
	}

How to ramp up by threads

2013-01-07T20:30:05-08:00

nGrinder supports the process rampup as a default feature. If a user want to increase the loads step by steps, User can set a number of process and how to increase them at the ramp-up panel in the test configuration page.

It’s the process ramp-up. If you like to 10 steps in ramp-up, you should set the process count at least 10. If you want more, you should set the process count more.
However, the processes take a lot of resources to be invoked. 100 processes in a agent is not realistic. It will cause the agent machine out of memory error.

Let’s assume that you want to know from which TPS level the system starts to be satuated.

You can use the thread level ramp up for this case. All you need is to add following code in your script.

Jython

# -*- coding:utf-8 -*-

# A simple example using the HTTP plugin that shows the retrieval of a
# single page via HTTP. 
#
# This script is auto generated by ngrinder.
#

from net.grinder.script.Grinder import grinder
from net.grinder.script import Test
from net.grinder.plugin.http import HTTPRequest
from net.grinder.plugin.http import HTTPPluginControl
from HTTPClient import NVPair

control = HTTPPluginControl.getConnectionDefaults()
control.setTimeout(30000)

test1 = Test(1, "Test1")
request1 = HTTPRequest();
test1.record(request1)

class TestRunner:
  def initialSleep( self ):
      sleepTime = grinder.threadNumber * 1000  # 1 seconds per thread
      grinder.sleep(sleepTime, 0)

  def __call__( self ):
      if grinder.runNumber == 0: self.initialSleep()

      grinder.statistics.delayReports=True
      result = request1.GET("http://www.google.com")
      
      if result.getText().find("Google") != -1 :
          grinder.statistics.forLastTest.success = 1
      else :
          grinder.statistics.forLastTest.success = 0

Groovy

/**
 * A simple example using the HTTP plugin that shows the retrieval of a
 * single page via HTTP.
 *
 * This script is auto generated by ngrinder.
 *
 * @author ${userName}
 */
@RunWith(GrinderRunner)
class Test1 {

	public static GTest test;
	public static HTTPRequest request;

	@BeforeProcess
	public static void beforeClass() {
		test = new GTest(1, "aa000000");
		request = new HTTPRequest();
		test.record(request);
		grinder.logger.info("before process.");
	}


	@BeforeThread
	public void beforeThread() {
		grinder.statistics.delayReports=true;
		grinder.logger.info("before thread.");
		
	}
	
	public void initialSleep() {
		grinder.sleep(grinder.threadNumber * 1000, 0)
	}

	@Test
	public void test(){
		if (grinder.runNumber == 0) {
		   initialSleep()
                }
		HTTPResponse result = request.GET("http://www.google.com");
		if (result.statusCode == 301 || result.statusCode == 302) {
			grinder.logger.warn("Warning. The response may not be correct. The response code was {}.", result.statusCode);
		} else {
			assertThat(result.statusCode, is(200));
		}
	}

Import Groovy Maven Project in IDE

2013-05-07T06:30:25-07:00

Groovy Maven Project is structured for users to easily import the project into IDE. If you already have an IDE which supports Groovy and Maven, you can easily import it. However if you don’t have it yet, please follow the instruction in [Install Groovy IDE] first.

If you have installed the Eclipse based Groovy IDE and created Groovy Maven Project in nGrinder subversion, you can import the Groovy Maven Project using the subversion client in Eclipse.

Notice that there is the subversion URL field in the script management menu. The above figure shows the our test instance. In this instance, the created Groovy Maven project can be accessed through http://ngrinder-staging.nhncorp.com:8080/svn/admin/project URL.
OK. Let’s import the project.

Open your Eclipse and Choose File ==> Others ==> Maven ==> Check out Maven Projects from SCM
Select svn in the SVN URL and type the SVN URL provided in nGrinder. Then click Finish.
Then the project will be imported.
Enable Groovy on this project.
Remove groovy libraries from classpath by selecting "Groovy ==> Remove Groovy libraries from classpath".
Because there is a conflict between libraries provided by Maven and Eclipse.
Select the Groovy test case and Select Run As ==> JUnit Test.
Then you might see the following error. It’s because the specialized nGrinder JUnit Runner(named GrinderRunner) needs a special JVM argument for instrumentation
Check the error message and add the JVM argument following the instruction. Select Run As ==> Run Configurations ==> Select Test1 ==> Select Arguments panel.

Please be aware that the argument is varied depending on users. So provide the exact argument which is shown in the JUnit error message. This setting should be provided for every JUnit test running set. If you like to make it as a default vm option for every test run. You can set it up in the global configuration. Please check [Install Groovy IDE] for details.
Then run as JUnit test again.
In my case, the access to please_modify_this.com causes the error. Ater changing this to http://www.google.com . It shows the green bar now.
You can add the break point in the test case. Then select Debug As ==> JUnit Test.
Then you can debug the test case using the generic Eclipse debugging feature.
When you think the test case is almost completed, you may need to see what happens in the repeated run. For this you can provide the @Repeat annotation on the test case class. Then run as JUnit test again.
Then you can see that test case is executed multiple times you specified. Please note that it’s 100 repetition by single thread.
If you think the test case is ready, commit it into subversion selecting Team ==> Commit. Finally run this test in nGrinder.

User Guide

2012-10-08T20:41:15-07:00

This page provide the collection of user guides when you access nGrinder with user permission.

[Quick Start]
[Test Configuration]

[Virtual User]
[Process and Thread]
[How to add custom monitor data]

Scripting

[Scripting Guide]

[Script Security]

[Script Management]

User

[User Permission]

[User Account Share]
[User Agent]

[Recorder]
[Frequently Asked Question]

If you're Korean speaker(한국인), please check you can check the following resources.

[User Guide in Korean]

Install Groovy IDE

2013-05-06T00:47:00-07:00

When you create Groovy Maven project, you can use IDE to write and debug test cases. Groovy maven project use only maven’s pom.xml to structure src folders and specify dependencies. Therefore if your IDE supports Groovy and Maven project, you can use it. For who don’t have IDE yet. This explains how to install Eclipse based Groovy Maven IDE.

Download and unzip the lastest “Eclipse IDE for Java Developers” which fits to your OS.

http://www.eclipse.org/downloads/packages/eclipse-ide-java-developers/junosr2
The above version already includes Maven Eclipse plugin but Groovy and Subversion. You need to install them. Firstly, run the downloaded eclipse by double-clicking eclipse.exe.

Install Groovy

Go to “Help” ==> “Eclipse Market Place”
Type groovy in the search field. Then click the install button in Groovy Eclipse for Juno.
Then install all.
Then click “Next”s and “Finish”s and restart Eclipse until the installation is completed.

Install Subversive

Go to “Help” ==> “Eclipse Market Place”
Type subversive in the search field. Then click install below.
Select Subveresive SVN JDK Ignore Extensions and Team Provider. Then select “Next” buttons until the installation is started.
Restart Eclipse and and select SVN Kit 1.7.8. Subversion repo provided by nGrinder is compatible with above SVN Kit 1.7.8.
Then click “Next”s and “Finish”s and restart Eclipse until the installation is completed.
Now it’s time to connect Maven and Subversion by installing m2e subversive connector. Click File menu ==> New ==> Checkout Maven Projects from SCM.
Then click m2e Marketplace in the next dialog.
Check the m2e subversive and click Finish button.

Then click “Next”s and “Finish”s and restart Eclipse until the installation is completed.
Groovy JUnit Running requires the special JVM argument for every test run. It can be very annoying. If you plan to use this Eclipse instance only for nGrinder Test Case, it might be a good choice to provide the default JVM options for every JVM execution in this Eclipse.
Go To Window ==> Preference ==> Java ==> Installed JRE. Then click Edit on the default JRE.

Finally you can provide Default VM arguments here. Detailed option you should provide depends on the user configuration. Please check [Import Groovy Maven Project in IDE] to know the option.

Now your IDE is ready to run and debug your nGrinder groovy test cases. [Import Groovy Maven Project in IDE]

Scripting Guide

2012-11-18T18:05:40-08:00

nGrinder 3.2 or later support not only Jython but also Groovy as scripting language. If you want to know about Groovy scripting, please refer [Groovy Script]

You can see various The Grinder script samples.

You can check The Grinder APIs.

More practical ways for the advanced nGrinder user are...

Do you want to know what script types nGrinder supports?

[Jython Script Structure]
[Groovy Script Structure]

Do you want to ramp-up more precisely?

[How to ramp up by threads]

Do you need a custom dns entry? Need to change the /etc/hosts file in the agents? No. You can configure target host fields for this. You don't need root permission.

[How to change DNS]

Do you need to use your own jar or py? Read followings.

[How to use library]

Do you need to load your own xml or txt file? Read followings.

[How to use resources]

Do you need to simulate the multiple user scenario in the different run weight?

[How to run the multiple tests with different weight]

Do you need to adjust the logger level to only see the errors? Read followings.

[How to adjust logger level]

Do you need to test DB performance? Doesn't nGrinder seems to support this? No.. You can still test it with nGrinder

[Using nGrinder to perform db load test]

Do you need to parse data retrieved from server?

[How to parse JSON]
[How to parse XML]

Do you need to define your own statistic?

[User defined statistic in ngrinder]

Do you cope with the big response like movie or file download?

[How to handle the big response]

Do you need you test a socket.io app?

Groovy Script Structure

2013-05-06T06:29:47-07:00

Not like [Jython Script Structure], Groovy Script Structure is based on JUnit. It’s because we want to reuse existing JUnit experiences as much as possible. Several IDEs already well integrate JUnit and let a user to run the test case directly. If you created Goorvy Maven Project option in script creation dialog box in nGrinder, You can enjoy the this excellant approach in your current IDE.

For now, I’d like to explain see the generic Groovy script stucture whatever you choose Groovy Script or Groovy Maven Project.
Following is a base template for groovy script.

import static net.grinder.script.Grinder.grinder
import static org.junit.Assert.*
import static org.hamcrest.Matchers.*
import net.grinder.plugin.http.HTTPRequest
import net.grinder.script.GTest
import net.grinder.script.Grinder
import net.grinder.scriptengine.groovy.junit.GrinderRunner
import net.grinder.scriptengine.groovy.junit.annotation.BeforeThread
import net.grinder.scriptengine.groovy.junit.annotation.BeforeProcess

import org.junit.BeforeClass
import org.junit.Test
import org.junit.runner.RunWith

import HTTPClient.HTTPResponse

/**
 * A simple example using the HTTP plugin that shows the retrieval of a
 * single page via HTTP.
 *
 * This script is auto generated by ngrinder.
 *
 * @author ${userName}
 */
@RunWith(GrinderRunner)
class Test1 {

    public static GTest test;
    public static HTTPRequest request;

    // This method is executed once per a process. 
    @BeforeProcess
    public static void beforeClass() {
        test = new GTest(1, "${name}");
        request = new HTTPRequest();
        test.record(request);
        grinder.logger.info("before process.");
    }

    // This method is executed once per a thread. 
    @BeforeThread
    public void beforeThread() {
        grinder.statistics.delayReports=true;
        grinder.logger.info("before thread.");
    }

    // This method is continously executed until you stop the test 
    @Test
    public void test(){
        HTTPResponse result = request.GET("${url}");
        if (result.statusCode == 301 || result.statusCode == 302) {
            grinder.logger.warn("Warning. The response may not be correct. The response code was {}.", result.statusCode);
        } else {
            assertThat(result.statusCode, is(200));
        }
    }
}

The Groovy test case in nGrinder should be annotated with @RunWith(GroovyRunner). This will invoke GroovyRunner to control the JUnit running behavior and mount the grinder context on JUnit.
@Test annotated methods are subject to be executed repeatedly. As you can see above, you can assert the test result using JUnit assertion way. If the assertion is failed, the last test bounded this thread will be evaluated as a error.

Instead of @BeforeClass @Before @AfterClass @After whch are used the generic JUnit tests, Groovy test case in nGrinder uses its own annotation to control script behaviors.

	Description	Applied to	Usage
@BeforeProcess	Define behaviors which should be executed before the process is invoked	static method	Load resource files which are shared by threads Define GTest and instrument the test target using record method.
@AfterProcess	Define behaviors which should be executed after the process is terminated	static method	Close resource files.
@BeforeThread	Define behaviors which should be executed before the each thread is executed.	member method	Login the target system. Set up the thread bounded values. (For example, Cookie Handler)
@AfterThread	Define behaviors which should be exeucted after the each thread is exeucted	member method	Logout the target system
@Before	Define behaviors which should be executed before every @Test annotated method are executed.	member method	Common logic which is shared by the multiple @Test annotated methods. Variable set up
@After	Define behaviors which should be executed after every @Test annotated methods are executed.	member method	Not used much..
@Test	Define the test behavior. Excuted muliple times	member method	Test body

The exeution flow can be illustrated by following.

In the test case sample, Please look this part carefully.

    public static GTest test;
    public static HTTPRequest request;

    // This method is executed once per a process. 
    @BeforeProcess
    public static void beforeProcess() {
        // Instead of Test in Jython, GTest is used here
        // It's because the identifier "Test" is alredy used by the @Test 
        // GTest is the replacement to avoid the naming confliction.
        test = new GTest(1, "${name}");
        request = new HTTPRequest();
        test.record(request);
        grinder.logger.info("before process.");
    }

In @BeforeProcess annotated method, it defines GTest instance and record the HTTPRequest instance and save them in the static member variable.
It’s the first step of test statistic preparation. Any method calls on request variable will increase the TPS. If you want to record multiple HTTPRequest instances using multiple GTest named diiferently, You can just add another member variables and invoke record method on different HTTPRequest instance. Furthermore, you can define another the test methods in a single test case class as well. Just add @Test annotation on those like the following.

    @Test
    public void testGoogle(){
        HTTPResponse result = request.GET("http://www.google.com");
        if (result.statusCode == 301 || result.statusCode == 302) {
            grinder.logger.warn("Warning. The response may not be correct. The response code was {}.", result.statusCode);
        } else {
            assertThat(result.statusCode, is(200));
        }
    }

    @Test
    public void testYahoo(){
        HTTPResponse result = request.GET("http://www.yahoo.com");
        if (result.statusCode == 301 || result.statusCode == 302) {
            grinder.logger.warn("Warning. The response may not be correct. The response code was {}.", result.statusCode);
        } else {
            assertThat(result.statusCode, is(200));
        }
    }

However, you should aware nGrinder GroovyRunner creates only one test case object per a thread not like generic JUnit test case which creates a separate test case object for each @Test annotated method. So if each @Test annotated methods share member variables, they can affect each other. It’s intentional. In nGrinder test case, it’s very likely each @Test method relies on the previous @Test annotated method execution result. Therefore, if you save the previous test result in the member varible and refer it to decide to continue to next test method, you can easily implement the test dependency.

    private boolean googleResult;
    
    @Test
    public void testGoogle(){
        googleResult = false;
        HTTPResponse result = request.GET("http://www.google.com");
        if (result.statusCode == 301 || result.statusCode == 302) {
            grinder.logger.warn("Warning. The response may not be correct. The response code was {}.", result.statusCode);
        } else {
            assertThat(result.statusCode, is(200));
        }
        googleResult = true;
    }

    @Test
    public void testYahoo(){
        if (!googleResult) {
            grinder.logger.warn("Just return. Because prev google test is failed.");
            return;
        }
        HTTPResponse result = request.GET("http://www.yahoo.com");
        if (result.statusCode == 301 || result.statusCode == 302) {
            grinder.logger.warn("Warning. The response may not be correct. The response code was {}.", result.statusCode);
        } else {
            assertThat(result.statusCode, is(200));
        }
    }

How can you use library and resources? If you don’t use [Groovy Maven Structure] yet, you can refer [Jython Script Structure] to how to include them in your script. It’s exactly same as Jython way.
However if you use Maven structure plus Groovy, you can enjoy the easier library dependency setting and run the JUnit test in your IDE. Interested in this?
Then continue to see the [Groovy Maven Structure] to know how to enable this for you.

Recorder Limitation

2013-04-12T07:42:51-07:00

Features per OS

Due to the embedded browser component limitation, nGrinder Recorder has a different feature support depending on OS.

Features	Windows	MAC	Linux
Proxy	O	O	O
Embedded Browser	O	△	△ (Crashed Frequently)
HTTPS support	O	X	O
Flash support	O	X	X

We're highly recommending you to use Windows when recording user interaction. If you like to use MAC or Linux, use the separate browser with proxy settings. Refer [Recorder-Recording Guide For Non Window User].

Missing Recorded Element

nGrinder Recorder does not guarantee to generate the perfectly executable scripts. It's a just helper application with which a user can start to write suitable scripts. Even though nGrinder Recorder contains embedded browsers nsides, it doesn't take much information from these to generate scripts. As you can see the nGrinder Recorder Architecture, the actual recording is performed by intercepting HTTP/HTTPS messages not by intercepting the browser events. After recording HTTP/HTTPS messages by a proxy, nGrinder Recorder tries to generate the script which simulates user behavior. It might be not perfect due to information loss. For example, The web pages in browsers can create cookies in javascript. nGrinder Recorder can understand cookies generated only by server but browser. Therefore this cookie processing code won't show up in the finally generated script. The users should add the client side cookie generation code by themselves.

# User need to write his own cookie handling code to complement the missing part.
threadContext = HTTPPluginControl.getThreadHTTPClientContext()
expiryDate = Date()
expiryDate.year += 10
cookie = Cookie("key", "value","localhost", "/", expiryDate, 0)
CookieModule.addCookie(cookie, threadContext)

Only information nGrinder Recorder takes from the embedded browser is click events to separate messages into the pages. Following is the code generated by nGrinder Recorder.

class TestRunner:
    """A TestRunner instance is created for each worker thread."""
    def __init__(self) :
        grinder.statistics.delayReports=True
        pass
    
    def __call__(self) :
        try : 
            # Following two pages are created because a user click the mouse.
            self.page1()
            self.page2()
            grinder.statistics.forLastTest.success = 1
        except Exception, e:
            err(e.message)
            grinder.statistics.forLastTest.success = 0

    def page1(self) :
        ##########################################################################################
        # http://img.naver.net/static/newsstand/up/2012/1119/nsd102245777.gif
        ##########################################################################################
        grinder.sleep(54)
        result = request_img_naver_net.GET(
                "/static/newsstand/up/2012/1119/nsd102245777.gif",
                None,    
                headers0
            ) 
        self.checkResponse(result, 200,  "http://img.naver.net/static/newsstand/up/2012/1119/nsd102245777.gif")

        ##########################################################################################
        # http://img.naver.net/thumb.opencast/opencast01/n/a/naver_story/20130411/17442431363707.jpg
        ##########################################################################################
        self.token_type = 'f10060'  
        result = request_img_naver_net.GET(
                "/thumb.opencast/opencast01/n/a/naver_story/20130411/17442431363707.jpg?type=%s"
                    % (self.token_type),
                None,    
                headers0
            ) 
        self.checkResponse(result, 200,  "http://img.naver.net/thumb.opencast/opencast01/n/a/naver_story/20130411/17442431363707.jpg")

Not recordable HTTPS messages

When users try to records HTTPS traffics, The browsers may shows the following message box.

This alert is shown because the proxy should use a self-signed private certificate between the browser and proxy HTTPS communication to capture HTTPS messages. The certificate issued by the webserver is used only between the proxy and target web server communication. The proxy decodes the messages from the target server and decode them with this valiid certificate and record the messages. Then the proxy encode the message with the grinder self-signed certificate and send it back to the browsers. From the perspective of browsers, they don’t aware of the proxy behavior. They just show warning mesage because they think the messages are encoded by something wrong certificiate. Currently only IE engine among our embedded browser engines allows users to accepts the invalid certificate while browsing HTTPS sites. Therefore the embedded browsers only works with the proxy well in Windows and IE env. In OS X, users should configure separate browsers to connect the proxy instead of using embedded browser. The general web browser has a way to accept the self-signed certificate. See [Recorder-How to import self signed certificates].

The messages used in the mobile devices can also be captured by setting the mobile device’s network configuration. However the most of all mobile browsers may reject the messages encoded with such a private certificates. There might be no way to show this HTTPS pages in the mobile devices. If someone knows how to configure it, please let us know.

Only way to browse these is to let the proxy use the valid certificate when it returns back the HTTPS messages to the browsers. See Configuration section in [Recorder-Installation].

Recorder Installation

2013-04-12T00:34:55-07:00

Prerequsite

nGrinder Recorder is cross os application becauses it is implemented in Java. At least you requires followings to run nGrinder Recorder.
- Java Runtime (over 1.6)
- GUI enabled env (not supporting headless OS)
However, Some major features of nGrinder Recorder can work in Windows due to some bugs in internal components.
We’ll continue to work to make the disabled feature runnable in the other OS.

Check out [Recorder-Limitation] first.

Installation

Currently nGrinder Recorder is provided in the form of java webstart. You can install it just by clicking the following URL in the browser.
- TBD
Then nGrinder Recorder will ask you to install the Therefore the following warning message will show up.
Check “Always trust content from the publisher” and click Run. nGrinder Recorder requires full permission in your PC because the underlying JXBrowser requires it.
Then.. If your PC’s firewall monitor port openning, the port openning warning message may come up. You should allow nGrinder Recorder to open the port appropriately.
If you can see the following window, You’re ready to use nGrinder Recorder.
Copy the downloaded jnlp file into the desktop to run nGrinder without network access.

Configuration

Just like nGrinder Controller and Agent, nGrinder Recorder uses the configuration in the ${user.home}/.ngrinder_recorder folder. You can open recorder.conf file to configure followings.

key	value
recorder.additional.headers	The recorder only records the limited set of headers. If you want to record more, you can specify this value with header names separated by comma. Check the detailed info here.
keystore.password	A user can provide the same certificate which is used already for the webserver HTTPS communication. A user should locates the keystore in the ${user.home}/.ngrinder_recorder/keystore file. Then specify the password of this keystore here.
keystore.type	A user can specify the keystore type. The default value is jks.

Recorder

2013-04-09T22:46:01-07:00

Grinder Recorder will be releasde at the end of April.

nGrinder Recorder is a helper application to create the basic nGrinder script. It records HTTP/HTTPS traffics sent from the embedded browser and creates the The Grinder compatible script. It’s based on TCPProxy which The Grinder project provides. We found out that TCPProxy is very powerful but it’s little bit difficult to understand and run because it’s command line based. In addition, the generated script is more complex than we expected. Therefore, we built new The Grinder script recording tool named "nGrinder Recorder" with the built-in web browser and java webstart technology. You can just start the script recording just by just few clicks.
We, nGrinder Development Team, don’t put much energy on this product. We believe the manual script writing is the best way to achieve efficient peformance testing not by this kind of recoding. So there might be some crash due to lack of its functionality testing.

[Recorder-Architecture]
[Recorder-Limitation]
[Recorder-ScreenShot]
[Recorder-Installation]
[Recorder-Quick-Start]
[Recorder-Recording-Guide]
[Recorder-Recording-Guide-For-Non-Window-User]
[Recorder-Generated Test Script Structure]

Recorder-Recording Guide For Non Window user

2013-04-16T18:46:52-07:00

Due to serious bugs existing internal component, nGrinder Recorder only provides its full funtionalities in Windows. For Linux and Mac users may not enjoy its simple use case but need some configuration to make nGrinder Recorder do its job.

In Mac and Linux, Embedded browsers are invoked with to the initiated proxy connection. However, it's not so well tested and we found there are some frequent crash especially in OS X. Furthermore, HTTPS is not supported in OS X. Therefore, we highly recommend you to set up separated browsers connecting to the proxy.

As you can see below, the proxy is initiated using 10288 port.

If you like to record HTTP/HTTPS with Firefox, Open Firefox and go to Preferences and Select Advanced ==> Settings in the popup window.

Set the proxy in HTTPS and HTTPS according the proxy info provided by nGrinder Recorder.

Then, navigate where you want in Firefox. Don’t forget that you should click Start Record button in nGrinder Recorder.

Then click Stop Recording in nGrinder Recorder. Then you can see the recorded script.

Please don’t forget to disable proxy settings when you finish the recording job. Otherwise you can not browse the Internet afterwards.

When you navigates to HTTPS sites, you will see the following page. It’s because the proxy send messages encrypted by the self-signed certificate.
To ignore this, you should click “I Understand the Risks” and add the exception.

How to build nGrinder Recorder using maven

2013-04-16T21:19:41-07:00

This article is only for nGrinder Recorder commiters. The source code of nGrinder Recorder exists in the private repo in github. Due to the licensing problem of 3rd party library, we can not publicize the source yet.

create ngrinder recorder folder and move in

mkdir ngrinder-recorder
cd ngrinder-recorder

clone the repo.

git clone https://github.com/nhnopensource/ngrinder-recorder.git

Move to cloned repo and install custom libraris into local maven repo.
```
cd ngrinder-recorder
install_maven_lib.bat
```
clone the build script repo at the same folder where ngrinder-recorder is cloned. This repo is located in NHN internal repo due to the certificate. If you have no permission yet. Please ask JunHo Yoon a permission.
```
cd ..
git clone http://devcode.nhncorp.com/git/recorderpack.git ngrinder_recorder_package
```
Move to ngrinder_recorder_package directory
```
cd ngrinder_recorder_package
```
create keystore
```
create_keystore.bat
```
package nGrinder Recorder with code signing.
```
deploy.bat
```

Dev Document

2012-06-21T20:59:50-07:00

This page contains nGrinder 3.0 related developer documents.

[How to build nGrinder from the scratch using Maven]
[How to build nGrinder Recorder using Maven]
[Java Implementation for Google Analytics Measurement Protocol]
[How to setup nGrinder Project in Eclipse]
[Coding Convention]
[Feature Completion Rate]
[How to refer nGrinder Home]
[Setup Mylyn task reposiroty for JIRA]
[Spring in nGrinder]
[How to create dynamic queries in SpringData]
[How to develop Plugin]
[v3.1 development features]
[nGrinder test execution performance test]
[Git Branching Policy]
[Contributors]

Recorder-Generated Test Script Structure

2013-03-05T18:03:25-08:00

The structure of a recorded test-script looks like this:

(
...
import statements
...
)

(
...
header definitions
...
)

(
...
url definitions
...
)

(
...
request and test definitions
...
)
class TestRunner:
     (
     ...
     method definitions - a method is defined for each recorded page
     ...
     )

     def __call__(self):
     (
     ...
     calls to defined methods, which actually runs the requests
     ...
     )

     ( utility function (I'll go over this later) )

     (
     ...
     calls to utility function to wrap/instrument tests (I'll go over this later)
     ...
     )

Import statements

you will import classes that make the tests work.By default,the following are indispensable.

from net.grinder.script import Test
from net.grinder.script.Grinder import grinder
from net.grinder.plugin.http import HTTPPluginControl, HTTPRequest
from HTTPClient import NVPair, Cookie, CookieModule, CookiePolicyHandler

The first two import statements import the Test and grinder objects, which give you access to grinder framework and its environment. Next two import statements import utility classes which perform HTTP Requests. The NVPair class lets you organize parameters and values for POST requests nto name-value pairs.

Header definitions

These headers are used to create the request objects .The generated code looks like this:

connectionDefaults = HTTPPluginControl.getConnectionDefaults()
httpUtilities = HTTPPluginControl.getHTTPUtilities()

log = grinder.logger.info
err = grinder.logger.error

# Set up a cookie handler to log all cookies that are sent and received.
class MyCookiePolicyHandler(CookiePolicyHandler):
	def acceptCookie(self, cookie, request, response):
		# log("accept cookie: %s" % cookie)
		return 1

	def sendCookie(self, cookie, request):
		# log("send cookie: %s" % cookie)
		return 1
 
def createRequest(url, headers=None):
	"""Create an instrumented HTTPRequest."""
	request = HTTPRequest(url=url)
	if headers: request.headers=headers
	return request

CookieModule.setCookiePolicyHandler(MyCookiePolicyHandler())
 
# These definitions at the top level of the file are evaluated once,
# when the worker process is started.
connectionDefaults.defaultHeaders = [
	NVPair('Accept-Encoding', 'gzip, deflate'),
	NVPair('Referer', 'https://accounts.google.com/ServiceLogin?service=mail&passive=true&continue=http://mail.google.com/mail/?ui%3Dmobile%26zyp%3Dl&scc=1<mpl=ecobh&nui=5&btmpl=mobile'),
	NVPair('Accept-Language', 'ko,Korean;q=0.7,en-US;q=0.3'),
	NVPair('User-Agent', 'Mozilla/5.0 (iPhone; U; CPU iPhone OS 1_2_3 like Mac OS X; en-us) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8J2 Safari/6533.18.5')
]


headers0 = [
	NVPair('Accept', '*/*')
]

The first statement creates an HTTPPluginConnection object, which lets you control the behavior of the connection. The next statement creates an instance of an HTTPUtilities object which provides access to various utility methods.

In the next few lines, the script actually defines the headers. First, it sets up the default headers that are used by each request.

Request and test definitions

This is the actual requests that we are going to use for our tests:

request_https_ssl_google_analytics_com = createRequest("https://ssl.google-analytics.com");
request_https_accounts_youtube_com = createRequest("https://accounts.youtube.com");
request_https_mail_google_com = createRequest("https://mail.google.com");

First,we create an instance of an HTTPRequest object using the previously defined URL’s and Headers.

Page definitions

After the request objects have been defined, we come to the actual TestRunner class.

# A method for each recorded page.
	def page1(self) :
		##########################################################################################
		# https://accounts.youtube.com/accounts/CheckConnection
		##########################################################################################
		grinder.sleep(196)
		self.token_pmpo = 'https://accounts.google.com'  
		self.token_v = '180753535'  
		self.token_timestamp = '1366165396372'  
		result = request_https_accounts_youtube_com.GET(
				"/accounts/CheckConnection?pmpo=%s&v=%s×tamp=%s"
					% (self.token_pmpo, self.token_v, self.token_timestamp),
				None,	
				[
					NVPair('Accept', 'application/x-ms-application, image/jpeg, application/xaml+xml, image/gif, image/pjpeg, application/x-ms-xbap, application/msword, application/vnd.ms-excel, application/vnd.ms-powerpoint, */*')
				]
			) 
		self.checkResponse(result, 200,  "https://accounts.youtube.com/accounts/CheckConnection")

		##########################################################################################
		# https://ssl.google-analytics.com/__utm.gif
		##########################################################################################
		self.token_utmwv = '5.4.1'  
		self.token_utms = '1'  
		self.token_utmn = '1757647728'  
		self.token_utmhn = 'accounts.google.com'  
		self.token_utmcs = 'utf-8'  
		self.token_utmsr = '1920x1080'  
		self.token_utmvp = '839x617'  
		self.token_utmsc = '24-bit'  
		self.token_utmul = 'en-us'  
		self.token_utmje = '1'  
		self.token_utmfl = '11.6 r602'  
		self.token_utmdt = 'Gmail: Google 이메일'  
		self.token_utmhid = '1841284240'  
		self.token_utmr = '-'  
		self.token_utmp = '/mail/homepage'  
		self.token_utmht = '1366165396400'  
		self.token_utmac = 'UA-992684-1'  
		self.token_utmcc = '__utma=72592003.2014635375.1366162754.1366162754.1366165396.2;+__utmz=72592003.1366162754.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none);'  
		self.token_utmu = 'qjCAAC~'  
		result = request_https_ssl_google_analytics_com.GET(
				"/__utm.gif?utmwv=%s&utms=%s&utmn=%s&utmhn=%s&utmcs=%s&utmsr=%s&utmvp=%s&utmsc=%s&utmul=%s&utmje=%s&utmfl=%s&utmdt=%s&utmhid=%s&utmr=%s&utmp=%s&utmht=%s&utmac=%s&utmcc=%s&utmu=%s"
					% (self.token_utmwv, self.token_utms, self.token_utmn, self.token_utmhn, self.token_utmcs, self.token_utmsr, self.token_utmvp, self.token_utmsc, self.token_utmul, self.token_utmje, self.token_utmfl, self.token_utmdt, self.token_utmhid, self.token_utmr, self.token_utmp, self.token_utmht, self.token_utmac, self.token_utmcc, self.token_utmu),
				None,	
				headers0
			) 
		self.checkResponse(result, 200,  "https://ssl.google-analytics.com/__utm.gif")

		##########################################################################################
		# https://mail.google.com/mail/images/c.gif
		##########################################################################################
		grinder.sleep(94)
		self.token_t = '1366165396404'  
		result = request_https_mail_google_com.GET(
				"/mail/images/c.gif?t=%s"
					% (self.token_t),
				None,	
				headers0
			) 
		self.checkResponse(result, 200,  "https://mail.google.com/mail/images/c.gif")

In the script, you have a method for each recorded page, which are called page1, page2.

Within each page, there are one or more GET and/or POST requests made via the request objects, which simulate the transactions made by a single page.

call method

In Jython any object implementing __call__ is callable, which basically means that this function is called when you call the class instance as a function. In the context of the this script, it just means that this is where all the testing starts off. In the __call__ method you can see that there are calls to all the recorded page methods,and this essentially runs the recorded test.

	def __call__(self) :
		try :
			self.page1()
			grinder.statistics.forLastTest.success = 1
		except Exception, e:
			err(e.message)
			grinder.statistics.forLastTest.success = 0

Recorder Recording Guide

2013-04-16T00:00:43-07:00

Filtering

If you ran the example following Quick Start, you may have been suprised by overwhelmly long generated scripts. It is because there are actually many HTTP/HTTPS messages even in a single browsing which you didn’t notice before. Do we have to include all these in the script? Sometimes it is useless to include all these in the performance test.
High load services usually use CDN(Content Delivery Network)s for static resources like images or javascripts. CDNs are located in multiple datacenters and the most speedy datacenter is automatically selected depending from where the calls are made. CDNs usually have the very efficient content delivery mechanism enough to handle high loads. Furthermore, they charges the fee depending on the how many contents are delivered. So it’s really necessary to simulate the call to here? You can not even control the content delivery logics if there are some performance issues.

Usually developers might want to run the performance test on the web servers under their controls. nGrinder Recorder includes several filtering features.

Host name filter

nGrinder Recorder recognizes the host names to which the proxy is connecting and how many calls are made. On the right panel there is a table which lists up the connected hosts and connection counts. When you don’t start Recording yet, this panel shows connected hosts but call count.

If you start Recording, it shows the recorded call count in “R” column. You can examine the host names and uncheck all hosts which you don’t interested in.

The easist way to achieve this is to click “Unselect All” button downside and check the interested host.

Resource type filter

There is one more way to filter out unnecessary static resources. At the downside of the right panel, there is a filter based on file types. “Recorded Type” panel lists the general static resource content type.

If you uncheck Record Type like above, nGrinder Recorder doesn’t record the HTTP messages which URL paths end with well known resource type names (like .png, .css, .js) or response content type has the well known static resource content type string (like text/javascript). If you uncheck some of these while you are recoding, the previously recorded messages are not deleted. However eventually it will be not included in the final script. If you check them again and generate a script, the script will contains the previously recorded messages.

Reset Recording

You can completly remove all recording by selecting “Clear all recoding” submenu which is popped up by clicking “Reset” button.

There are several sub menus in Reset button. Each has the different action like below.

Submenu name	Action
Clear all recording	Clear all recorded messages. However do not change the filter.
Clear all filters	Clear all filters. However do not chnage the recorded messages.
Clear connection count	Make the all connection count 0 in the filter panel.
Reset browser cache	Delete all cached content from the embedded browser so that the browser can actually make a call to the given web pages.
Reset cookies	Delete all saved cookes from the embedded browser so that the browser is not interfared by the previosly issued cookies.

Script Generation

When a user clicks “Stop Recordng” button, the recorded HTTP/HTTPS messages are transformed to The Grinder script. The script generated by nGrinder Recorder is little bit different from TCPProxy provides. We made the script template little bit compact than what TCPProxy generates.

Be aware that “Stop Recording” doesn’t mean “Reset Recording”. If you click Start Recording again, it continues to record the messages appending to existing recording. It’s because there are many reasons to pause the recording (A user might fall down by too much overwork)

Add sleep time

There are one option for script generation. If you check “Add sleep time” in Script Generation Options option groups, the actual wating time during recoding are included in the unit of milliseconds at the final script like following.

If you went to the toilet after starting recoding, Recorded sleep time might be very long. Please check the how long the script sleep between calls before running a script in nGrinder Controller.

You can also find very detailed generated script structure explaination here.

Mobile Web Page Recording

nGrinder Recorder supports the user agent string change in Windows. which means you navigate the mobile site which detects the browser and send the appropriate contents. You can select the which user agent will be used during navigation. Then even if you navigate the www.amazon.com, Then the mobile page is sent from the server like below

If you don’t want to use this fake browser technique but the real mobile browser from your devices, you can record the messages by configuring your mobile devices proxy settings.

IOS	Android
Go to the Settings Application Go to General < Network < Wi-Fi Select Wi-Fi connection you want to use with HTTP Proxy and connect. Edit Wi-Fi connection, scroll down and find "Proxy HTTP" and press on "Manual"", here we need to fill all proxy connection settings. Set server as 10.66.9.95 Set port as 10288 Disable authentication	Open the Applications tab and tap Settings. Tap Wireless & networks. Make sure the wireless network is enabled, and connected to the network for which you want to set the proxy server. Tap Wi-Fi settings. In the list of available networks, tap and hold the network to which you are connected, until a menu shows. Tap Modify Network. Tap Proxy Settings, and then Manual. Set server as 10.66.9.95 Set port as 10288 Tab Save

IOS

Android

Go to the Settings Application
Go to General < Network < Wi-Fi
Select Wi-Fi connection you want to use with HTTP Proxy and connect.
Edit Wi-Fi connection, scroll down and find "Proxy HTTP" and press on "Manual"", here we need to fill all proxy connection settings.
Set server as 10.66.9.95
Set port as 10288
Disable authentication

Open the Applications tab and tap Settings.
Tap Wireless & networks.
Make sure the wireless network is enabled, and connected to the network for which you want to set the proxy server.
Tap Wi-Fi settings.
In the list of available networks, tap and hold the network to which you are connected, until a menu shows.
Tap Modify Network.
Tap Proxy Settings, and then Manual.
Set server as 10.66.9.95
Set port as 10288
Tab Save

If you set up above, you can record the messages from your devices, just as you did in the embedded browser.

HTTPS Handling

nGrinder Recorder supports HTTPS as well but very limited way. As described in Recorder Limitation, the private self-signed ceritificate is used to intercept the HTTPS messages. So you may see following alerts whenever you access HTTPS sites.

To avoid this alert, you have to import the self signed certificate in the trusted certificates. Check out [Recorder-How to import self-signed certificates]

nGrinder Recorder is to provide another way to work around this issue. Actually this method is introduced by TCPProxy and nGrinder uses it with little modification. If you are going to test your HTTPS server, you might have already the valid certificate. You can provide this instead of embedded certificate for the proxy to use. You can refer how to provide your own certificate in Recorder Installation guide.

nGrinder DevZone

2012-06-20T18:23:28-07:00

nGrinder (pronounced as en-g-ra-in-der) is a stress tests platform that enables you to create test scripts, execute tests, monitor target servers and generate results simultaneously. The open-source nGrinder offers easy ways to conduct stress tests by eliminating inconveniences and providing integrated environments. It uses the renowned performance test tool, The Grinder, as its internal engine.

This is the tool developed by the service provider who really runs frequent load testing not by tool vendor. A breakthrough feature of nGrinder 3.X is support for concurrent tests which means that multiple users can run different tests at the same time! And it's cloud friendly and has attractive interface !

nGrinder is being developed by NHN Korea and NHN China together.

Official Home Page
[Architecture]
[Screen Shot]
[Guide]

[Installation Guide]
[Admin Guide]
[User Guide]
[Scripting Guide]
[Recorder Guide]

[Presentations]
[Professional Service]

[nGrinder Release Notes|Release Notes]
Downloads
[Road Map]
[Dev Document]
Mailing List
Issue Tracker
[Frequently Asked Question]
[How to Use Windows LiveWriter in nGrinder wiki|Wiki Guide]
[Who uses nGrinder]
[Help nGrinder]

Recorder Guide

2013-04-16T18:45:05-07:00

Please follow [Recorder].

Guide

2012-09-27T07:22:21-07:00

This page provides the collection of nGrinder guides.

[Installation Guide] / [Upgrade Guide]
[Quick Start]
[User Guide]
[Admin Guide]
[Recorder Guide]

How to add custom monitor data

2013-04-01T22:31:18-07:00

From nGrinder 3.1.3, we provided support for the custom monitor data for target monitoring. It can be used if you want to check some monitor data of target server in the test report. All you need to do, is to use any tools you like to generate the monitor data with some internal time. (The best practice is to set the internal time as same as the sampling interval of nGrinder test.) And the data should be saved in:

${ngrinder_agent}/monitor/custom.data

And the content of this file should be the several values of monitoring, seporated with “,”. And it only contains the current value with only one line. As below:

315630613,1123285602,1106612131

And then, you need to make sure the file is being updated during the test running. nGrinder monitor will get its content and send to nGrinder controller. After the test finished, you can see the target monitoring chart page in test report:

Be noticed that the chart name is “CUSTOM MONITOR DATA 1”, “CUSTOM MONITOR DATA 2” .., till “CUSTOM MONITOR DATA 5”. And there will be at most 5 custom data. You can save the meaning of this fileds as the test comment.

So, the next, we need to use some tool to get the monitor data and save into that file with some interval.

To do that, the first thing on Linux you may think about is “cron”. We can create a script and run it with “cron”. But with “cron”, the minimal interval is minute. But for nGrinder monitoring, at the most of time the interval is one second.

So in this guide, I will use java to get the java VM running information with JMX. If the target server is a java application like tomcat, we can use the way in this guide to get the VM running information of tomcat, and display the data in chart in the final report.

As I said above, I will use JMX to get the VM data. But in the normal we will not enable the JMX server of tomcat. So we will use attach API to attach to the target process, and enable the “management-agent”, and then we can connect with JMX to the target process.

You can find the VMMonitor.java from gist. Be aware about the code that, in java VM, for different java version and different GC setting, the GC names will be different. So in this code, I get the GC names of current Java VM, and using these names to get target process GC information with JMX. So the Java environment you use to run this code should be same as that you run the target tomcat. And the VM option like “-server” or GC setting should also be same. Otherwise, you can not get the GC names of target tomcat process.

And below, we will use this code to get the GC information of target process and save into custom.data file. The new code is as below:

import java.io.BufferedWriter;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.lang.management.GarbageCollectorMXBean;
import java.lang.management.ManagementFactory;
import java.util.HashSet;
import java.util.List;
import java.util.Set;

import javax.management.MBeanServerConnection;
import javax.management.ObjectName;
import javax.management.remote.JMXConnector;
import javax.management.remote.JMXConnectorFactory;
import javax.management.remote.JMXServiceURL;

import com.sun.tools.attach.AttachNotSupportedException;
import com.sun.tools.attach.VirtualMachine;

/**
 * Class description.
 * 
 * @author Mavlarn
 */
public class GCMonitor {

    public static Set<String> youngGCNames = new HashSet<String>();
    public static Set<String> oldGCNames = new HashSet<String>();

    static {
        // Oracle (Sun) HotSpot
        youngGCNames.add("Copy"); // -XX:+UseSerialGC
        youngGCNames.add("ParNew"); // -XX:+UseParNewGC
        youngGCNames.add("PS Scavenge"); // -XX:+UseParallelGC

        // Oracle (BEA) JRockit
        youngGCNames.add("Garbage collection optimized for short pausetimes Young Collector"); // -XgcPrio:pausetime
        youngGCNames.add("Garbage collection optimized for throughput Young Collector"); // -XgcPrio:throughput
        youngGCNames.add("Garbage collection optimized for deterministic pausetimes Young Collector"); // -XgcPrio:deterministic

        // Oracle (Sun) HotSpot
        oldGCNames.add("MarkSweepCompact"); // -XX:+UseSerialGC
        oldGCNames.add("PS MarkSweep"); // -XX:+UseParallelGC and
                                        // (-XX:+UseParallelOldGC or -XX:+UseParallelOldGCCompacting)
        oldGCNames.add("ConcurrentMarkSweep"); // -XX:+UseConcMarkSweepGC

        // Oracle (BEA) JRockit
        oldGCNames.add("Garbage collection optimized for short pausetimes Old Collector"); // -XgcPrio:pausetime
        oldGCNames.add("Garbage collection optimized for throughput Old Collector"); // -XgcPrio:throughput
        oldGCNames.add("Garbage collection optimized for deterministic pausetimes Old Collector"); // -XgcPrio:deterministic
    }

    static final String CONNECTOR_ADDRESS = "com.sun.management.jmxremote.localConnectorAddress";

    public static void main(String[] args) throws InterruptedException {
        if (args == null || args.length == 0) {
            System.err.println("Please specify the target PID to attach.");
            return;
        }

        // attach to the target application
        VirtualMachine vm;
        try {
            vm = VirtualMachine.attach(args[0]]);
        } catch (AttachNotSupportedException e) {
            System.err.println("Target application doesn't support attach API.");
            e.printStackTrace();
            return;
        } catch (IOException e) {
            System.err.println("Error during attaching to target application.");
            e.printStackTrace();
            return;
        }

        try {
            // get the connector address
            String connectorAddress = vm.getAgentProperties().getProperty(CONNECTOR_ADDRESS);
            MBeanServerConnection serverConn;
            // no connector address, so we start the JMX agent
            if (connectorAddress == null) {
                String agent = vm.getSystemProperties().getProperty("java.home") + File.separator + "lib"
                        + File.separator + "management-agent.jar";
                vm.loadAgent(agent);
                // agent is started, get the connector address
                connectorAddress = vm.getAgentProperties().getProperty(CONNECTOR_ADDRESS);
            }

            // establish connection to connector server
            JMXServiceURL url = new JMXServiceURL(connectorAddress);
            JMXConnector connector = JMXConnectorFactory.connect(url);
            serverConn = connector.getMBeanServerConnection();
            ObjectName objName = new ObjectName(ManagementFactory.RUNTIME_MXBEAN_NAME);

            // Get standard attribute "VmVendor"
            String vendor = (String) serverConn.getAttribute(objName, "VmVendor");
            System.out.println("vendor:" + vendor);

            String[] gcNames = getGCNames();
            while(true) {
                long minorGCCount = 0;
                long minorGCTime = 0;
                long fullGCCount = 0;
                long fullGCTime = 0;
                
                for (String currName : gcNames) {
                    objName = new ObjectName("java.lang:type=GarbageCollector,name=" + currName);
                    Long collectionCount = (Long) serverConn.getAttribute(objName, "CollectionCount");
                    Long collectionTime = (Long) serverConn.getAttribute(objName, "CollectionTime");
                    if (youngGCNames.contains(currName)) {
                        minorGCCount = collectionCount;
                        minorGCTime = collectionTime;
                    } else if (oldGCNames.contains(currName)) {
                        fullGCCount = collectionCount;
                        fullGCTime = collectionTime;
                    }
                    StringBuilder sb = new StringBuilder("[");
                    sb.append(getGCType(currName)).append("	: ");
                    sb.append("Count=" + collectionCount);
                    sb.append(" 	GCTime=" + collectionTime);
                    sb.append("]");
                    System.out.println(sb.toString());
                }
                StringBuilder valueStr = new StringBuilder();
                //custom data format is:
                //minorGCCount,minorGCTime,fullGCCount,fullGCTime
                valueStr.append(minorGCCount);
                valueStr.append(",");
                valueStr.append(minorGCTime);
                valueStr.append(",");
                valueStr.append(fullGCCount);
                valueStr.append(",");
                valueStr.append(fullGCTime);
                writeToFile(valueStr.toString());
                Thread.sleep(1000); //sleep one second.
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    public static String getGCType(String name) {
        if (youngGCNames.contains(name)) {
            return "Minor GC";
        } else if (oldGCNames.contains(name)) {
            return "Full GC";
        } else {
            return name;
        }
    }

    public static String[] getGCNames() {
        List<GarbageCollectorMXBean> gcmbeans = ManagementFactory.getGarbageCollectorMXBeans();
        String[] rtnName = new String[gcmbeans.size()];
        int index = 0;
        for (GarbageCollectorMXBean gc : gcmbeans) {
            rtnName[index] = gc.getName();
            index++;
        }
        return rtnName;
    }
    
    public static void writeToFile(String gcData) {
        String currDir = System.getProperty("user.dir");
        BufferedWriter writer = null;
        
        try {
            File customFile = new File(currDir + File.separator + "custom.data");
            if (!customFile.exists()) {
                customFile.createNewFile();
            }
            writer = new BufferedWriter(new FileWriter(customFile));
            writer.write(gcData);
            writer.flush();
        } catch (IOException e) {
            System.err.println("Error to read custom monitor data:" + e.getMessage());
        } finally {
            if (writer != null) {
                try {
                    writer.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }

        }
    }
}

Some comments about the code are:

a) We need to pass the target process ID as arguments to run.

b) This code should be run under same java environment and VM option with the target process. Otherwise, the attaching will properly fail. And if the target tomcat process is running with “-server” option, also be aware to set “-server” option to run this java code.

Becasue under different java version and setting, the GC names will be different, I founded some names and their types(monior GC or full). They are not complete and maybe not very correct. If you use IBM JVM, you should modify the code by yourself.

c) The custom values format is “minorGCCount,minorGCTime,fullGCCount,fullGCTime”.

d) This java code should be run in “${ngrinder_agent}/monitor/” directory. Becasue in the code, it will create custom.data file in current directory.

e) To compile the code, it needs “tools.jar” of JDK. You need to compile it like this:

javac -cp /home/ngrinder/jdk1.6.0_38/lib/tools.jar GCMonitor.java

#get target tomcat process ID, it is 24003
java -cp /home/ngrinder/jdk1.6.0_38/lib/tools.jar: GCMonitor  24003

Then you should see the output in console as below:

current dir:/home/ngrinder/.ngrinder_agent/monitor
[Minor GC       : Count=3564    GCTime=27850]
[Full GC        : Count=166     GCTime=65525]
[Minor GC       : Count=3564    GCTime=27850]
[Full GC        : Count=166     GCTime=65525]

And in custom.data file, the content is:

3564,27850,166,65525

Then, create a new test on nGrinder and set the target host properly. You can see the chart in test report:

(Becasue the GC count is not changed very much, so we can not see much difference from the chart.)

In this way, you can add any kind of custom monitor data and check it from test report.