• If you want to see more generic product info, please visit http://nhnopensource.org/ngrinder  
  
  

1. Release Info

2. Changes

outline

nGrinder (pronounced as en-g-ra-in-der) is a stress tests platform that enables you to create test scripts, execute tests, monitor target servers and generate results simultaneously. The open-source nGrinder offers easy ways to conduct stress tests by eliminating inconveniences and providing integrated environments. It uses the renowned performance test tool, The Grinder, as its internal engine.

This is the tool developed by the service provider who really runs frequent load testing not by tool vendor. A breakthrough feature of nGrinder 3.X is support for concurrent tests which means that multiple users can run different tests at the same time! And it's cloud friendly and has attractive interface !

nGrinder is being developed by NHN Korea and NHN China together.

If you ran the example following [Recorder Quick Start], you may have been surprised by overwhelmingly long generated scripts. It is because there are actually many HTTP/HTTPS messages even in a single web browsing which you didn't notice before. Do we have to include all these in the script? Sometimes it is useless to include all these in the performance test.
High load services usually use CDN(Content Delivery Network)s for static resources like images or javascripts. CDNs are located in multiple sites and the most close site is automatically selected depending from where the calls are made. In addition, CDNs usually have the very efficient content delivery mechanism enough to handle high loads. Moreover, they charge the fee depending on the how many contents are delivered.  So it’s really necessary to simulate the call to here? You can not even control the content delivery logics if there are some performance issues.

Usually developers might want to run the performance test on the web servers under their controls. Therefore, nGrinder Recorder includes several filtering features.

Host name filter

nGrinder Recorder recognizes the host names to which the proxy is connecting and how many calls are made. On the right panel there is a table which lists up the connected hosts and connection counts. When you don’t start Recording yet, this panel shows connected hosts but call count.

If you start Recording, it shows the recorded call count in “R” column.  You can examine the host names and uncheck all hosts which you don’t interested in.

The easiest way to achieve this is to click “Unselect All” button downside and check the interested host one by one.

Resource type filter

There is one more method to filter out unnecessary static resources. At the downside of the right panel, there is a filter based on file types. “Recorded Type” panel lists the general static resource content type.

If you uncheck Record Type like above, nGrinder Recorder won't record the HTTP messages which URL paths end with well known static resource type names (like .png, .css, .js) or response content type has the well known static resource content type string (like text/javascript). Even if you uncheck some of these during recording session, the previously recorded messages are not removed in the memory. However eventually it will be not included in the final script. If you check them again and generate a script, the script will contain the previously recorded messages.

Reset Recording

You can completely remove all recording by selecting “Clear all recording” sub-menu which is popped up in “Reset” button.

There are several sub menus in Reset button. Each has the different action like below.

Script Generation

When a user clicks “Stop Recording” button, the recorded HTTP/HTTPS messages are transformed to The Grinder script. The script generated by nGrinder Recorder is little bit different from TCPProxy provides. We made the script template little bit more compact than what TCPProxy generates.

Be aware that “Stop Recording” doesn't mean “Reset Recording”. If you click Start Recording again, it continues to record the messages appending to existing recording. It’s because there are many reasons to pause the recording (A user might fall down by too much overwork)

Add sleep time

There are one option for script generation. If you check “Add sleep time” in Script Generation Options option groups, the actual waiting time during recording are included in the unit of milliseconds at the final script like following.

If you went to the toilet after starting recording, Recorded sleep time might be very long. Please check the how long the script sleep between calls before running a script in nGrinder Controller.

Mobile Web Page Recording

nGrinder Recorder supports the user agent string manipulation which means you can navigate the mobile site which detects the browser type and send the appropriate contents. You can select the mobile user agent. Then even if you navigate the www.amazon.com, Then the mobile page is sent from the server like below.

If you don’t want to use this fake browser technique but the real mobile browser from your devices, you can record the messages by configuring your mobile devices proxy settings.

If you set up above, you can record the messages from your devices, just as you did in the embedded browser.

HTTPS Handling

nGrinder Recorder supports HTTPS as well but very limited way. As described in Recorder Limitation, the private self-signed ceritificate is used to intercept the HTTPS messages. So you may see following alerts whenever you access HTTPS sites.

To avoid this alert, you have to import the self signed certificate in the trusted certificates. Check out [Recorder-How to import self-signed certificates]

nGrinder Recorder provides another way to work around this issue. Actually this method is introduced by TCPProxy and nGrinder uses it with little modification. If you are going to test your own HTTPS server, you might have already the valid certificate. You can provide this instead of embedded certificate for the proxy to use. You can refer how to provide your own certificate in [Recorder Installation].

Due to the embedded browser component's limitation, nGrinder Recorder has a different feature set depending on OS.

We're highly recommending you to use Windows when recording the user interaction. If you like to use MAC or Linux, use the separate browser with proxy settings. Refer [Recorder-Recording Guide For Non Window User].

Missing Recorded Element

nGrinder Recorder does not guarantee to generate the perfectly executable scripts. It's a just helper application with which a user can easily start to write scripts. Even though nGrinder Recorder contains embedded browser component, nGrinder Recorder doesn't use this component very much much information. As you can find in nGrinder Recorder Architecture, the actual recording is performed in HTTP/HTTPS messages level not the browser events level. This causes some information loss. For example, The web pages in browsers can create cookies in javascript. nGrinder Recorder can understand cookies generated only by server but browser. Therefore this cookie processing code won't show up in the finally generated script. The users should add the client side cookie generation code by themselves.

# User need to write his own cookie handling code to complement the missing part.
threadContext = HTTPPluginControl.getThreadHTTPClientContext()
expiryDate = Date()
expiryDate.year += 10
cookie = Cookie("key", "value","localhost", "/", expiryDate, 0)
CookieModule.addCookie(cookie, threadContext)

Only information which nGrinder Recorder currently takes from the embedded browser is the click events to separate messages into the multiple pages. Following is the code generated by nGrinder Recorder.

class TestRunner:
    """A TestRunner instance is created for each worker thread."""
    def __init__(self) :
        grinder.statistics.delayReports=True
        pass
    
    def __call__(self) :
        try : 
            # Following two pages are created because a user click the mouse.
            self.page1()
            self.page2()
            grinder.statistics.forLastTest.success = 1
        except Exception, e:
            err(e.message)
            grinder.statistics.forLastTest.success = 0

    def page1(self) :
        ##########################################################################################
        # http://img.naver.net/static/newsstand/up/2012/1119/nsd102245777.gif
        ##########################################################################################
        grinder.sleep(54)
        result = request_img_naver_net.GET(
                "/static/newsstand/up/2012/1119/nsd102245777.gif",
                None,    
                headers0
            ) 
        self.checkResponse(result, 200,  "http://img.naver.net/static/newsstand/up/2012/1119/nsd102245777.gif")

        ##########################################################################################
        # http://img.naver.net/thumb.opencast/opencast01/n/a/naver_story/20130411/17442431363707.jpg
        ##########################################################################################
        self.token_type = 'f10060'  
        result = request_img_naver_net.GET(
                "/thumb.opencast/opencast01/n/a/naver_story/20130411/17442431363707.jpg?type=%s"
                    % (self.token_type),
                None,    
                headers0
            ) 
        self.checkResponse(result, 200,  "http://img.naver.net/thumb.opencast/opencast01/n/a/naver_story/20130411/17442431363707.jpg")

HTTPS messages

When users try to record HTTPS messages, The browsers may show the following security alert.

This alert is popped up because the proxy uses a self-signed private certificate between the browser and proxy HTTPS communication to capture HTTPS messages. The certificate issued by the web server is used only in the communication between the proxy and target web server. The proxy decodes the messages from the target server  with the valid certificate and records the messages. Then the proxy encodes the message with the The Grinder embedding self-signed certificate and sends it back to the browsers. From the perspective of browsers, browsers don’t aware of this proxy behavior. They just show the alert messages because they think the messages are encoded by the wrong certificate. Currently only IE engine among our embedded browser engines allows users to accepts the invalid certificate while browsing HTTPS sites. Which means only windows supports both HTTP and HTTPS recording. In OS X, users should use the separate browsers not embedded browser if they want to record HTTPS as well. The general web browser has a way to accept the self-signed certificate. See [Recorder-How to import self signed certificates].

The messages used in the mobile devices can also be captured by setting the mobile device’s network configuration. However the most of all mobile browsers may reject the messages encoded with a private certificate. There might be no way to show these HTTPS pages in the mobile devices. If someone knows how to configure it, please let us know.

Only way to browse these is to let the proxy use the valid certificate when it returns back the HTTPS messages to the browsers. See Configuration section in [Recorder-Installation].

From nGrinder 3.1.3, nGrinder support the custom monitoring for targets. It can be used if you want to monitor target server statuses which are not provided as default by nGrinder monitor. All you need to do is to use any tools you like to collect the monitor data with some internal time. (It might be best if you set it as same as the sampling interval of nGrinder test.) and the data should be saved in:

• ${ngrinder_agent_home}/monitor/custom.data

The content of this file can have several monitoring values separated with “,” in the single line  like below:

315630613,1123285602,1106612131

Then you need to make sure the file is being updated during the test running. If you have a shell script named "update.sh" which updates custom.data file, you can run it regularly by

• watch -n 1 update.sh

Find there are charts named “CUSTOM MONITOR DATA 1”, “CUSTOM MONITOR DATA 2” .., till “CUSTOM MONITOR DATA 5”. nGrinder monitor can collect at most 5 custom data. 

How to collect JVM status

In the following guide, I will use java to get the java VM running information with JMX. If the target server is a java application like tomcat, we can use the way in this guide to get the VM running information of tomcat, and display the data in chart in the final report. As I said above, I will use JMX to get the VM data. But in the normal we will not enable the JMX server of tomcat. So we will use attach API to attach to the target process, and  enable the “management-agent”, and then we can connect with JMX to the target process.

You can find the VMMonitor.java from gist. Be aware about the code that, in java VM, for different java version and different GC setting, the GC names will be different. So in this code, I get the GC names of current Java VM, and using these names to get target process GC information with JMX. So the Java environment you use to run this code should be same as that you run the target tomcat. And the VM option like “-server” or GC setting should also be same. Otherwise, you can not get the GC names of target tomcat process.

And below, we will use this code to get the GC information of target process and save into custom.data file. The new code is as below:

import java.io.BufferedWriter;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.lang.management.GarbageCollectorMXBean;
import java.lang.management.ManagementFactory;
import java.util.HashSet;
import java.util.List;
import java.util.Set;

import javax.management.MBeanServerConnection;
import javax.management.ObjectName;
import javax.management.remote.JMXConnector;
import javax.management.remote.JMXConnectorFactory;
import javax.management.remote.JMXServiceURL;

import com.sun.tools.attach.AttachNotSupportedException;
import com.sun.tools.attach.VirtualMachine;

/**
 * Class description.
 * 
 * @author Mavlarn
 */
public class GCMonitor {

    public static Set<String> youngGCNames = new HashSet<String>();
    public static Set<String> oldGCNames = new HashSet<String>();

    static {
        // Oracle (Sun) HotSpot
        youngGCNames.add("Copy"); // -XX:+UseSerialGC
        youngGCNames.add("ParNew"); // -XX:+UseParNewGC
        youngGCNames.add("PS Scavenge"); // -XX:+UseParallelGC

        // Oracle (BEA) JRockit
        youngGCNames.add("Garbage collection optimized for short pausetimes Young Collector"); // -XgcPrio:pausetime
        youngGCNames.add("Garbage collection optimized for throughput Young Collector"); // -XgcPrio:throughput
        youngGCNames.add("Garbage collection optimized for deterministic pausetimes Young Collector"); // -XgcPrio:deterministic

        // Oracle (Sun) HotSpot
        oldGCNames.add("MarkSweepCompact"); // -XX:+UseSerialGC
        oldGCNames.add("PS MarkSweep"); // -XX:+UseParallelGC and
                                        // (-XX:+UseParallelOldGC or -XX:+UseParallelOldGCCompacting)
        oldGCNames.add("ConcurrentMarkSweep"); // -XX:+UseConcMarkSweepGC

        // Oracle (BEA) JRockit
        oldGCNames.add("Garbage collection optimized for short pausetimes Old Collector"); // -XgcPrio:pausetime
        oldGCNames.add("Garbage collection optimized for throughput Old Collector"); // -XgcPrio:throughput
        oldGCNames.add("Garbage collection optimized for deterministic pausetimes Old Collector"); // -XgcPrio:deterministic
    }

    static final String CONNECTOR_ADDRESS = "com.sun.management.jmxremote.localConnectorAddress";

    public static void main(String[] args) throws InterruptedException {
        if (args == null || args.length == 0) {
            System.err.println("Please specify the target PID to attach.");
            return;
        }

        // attach to the target application
        VirtualMachine vm;
        try {
            vm = VirtualMachine.attach(args[0]]);
        } catch (AttachNotSupportedException e) {
            System.err.println("Target application doesn't support attach API.");
            e.printStackTrace();
            return;
        } catch (IOException e) {
            System.err.println("Error during attaching to target application.");
            e.printStackTrace();
            return;
        }

        try {
            // get the connector address
            String connectorAddress = vm.getAgentProperties().getProperty(CONNECTOR_ADDRESS);
            MBeanServerConnection serverConn;
            // no connector address, so we start the JMX agent
            if (connectorAddress == null) {
                String agent = vm.getSystemProperties().getProperty("java.home") + File.separator + "lib"
                        + File.separator + "management-agent.jar";
                vm.loadAgent(agent);
                // agent is started, get the connector address
                connectorAddress = vm.getAgentProperties().getProperty(CONNECTOR_ADDRESS);
            }

            // establish connection to connector server
            JMXServiceURL url = new JMXServiceURL(connectorAddress);
            JMXConnector connector = JMXConnectorFactory.connect(url);
            serverConn = connector.getMBeanServerConnection();
            ObjectName objName = new ObjectName(ManagementFactory.RUNTIME_MXBEAN_NAME);

            // Get standard attribute "VmVendor"
            String vendor = (String) serverConn.getAttribute(objName, "VmVendor");
            System.out.println("vendor:" + vendor);

            String[] gcNames = getGCNames();
            while(true) {
                long minorGCCount = 0;
                long minorGCTime = 0;
                long fullGCCount = 0;
                long fullGCTime = 0;
                
                for (String currName : gcNames) {
                    objName = new ObjectName("java.lang:type=GarbageCollector,name=" + currName);
                    Long collectionCount = (Long) serverConn.getAttribute(objName, "CollectionCount");
                    Long collectionTime = (Long) serverConn.getAttribute(objName, "CollectionTime");
                    if (youngGCNames.contains(currName)) {
                        minorGCCount = collectionCount;
                        minorGCTime = collectionTime;
                    } else if (oldGCNames.contains(currName)) {
                        fullGCCount = collectionCount;
                        fullGCTime = collectionTime;
                    }
                    StringBuilder sb = new StringBuilder("[");
                    sb.append(getGCType(currName)).append("	: ");
                    sb.append("Count=" + collectionCount);
                    sb.append(" 	GCTime=" + collectionTime);
                    sb.append("]");
                    System.out.println(sb.toString());
                }
                StringBuilder valueStr = new StringBuilder();
                //custom data format is:
                //minorGCCount,minorGCTime,fullGCCount,fullGCTime
                valueStr.append(minorGCCount);
                valueStr.append(",");
                valueStr.append(minorGCTime);
                valueStr.append(",");
                valueStr.append(fullGCCount);
                valueStr.append(",");
                valueStr.append(fullGCTime);
                writeToFile(valueStr.toString());
                Thread.sleep(1000); //sleep one second.
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    public static String getGCType(String name) {
        if (youngGCNames.contains(name)) {
            return "Minor GC";
        } else if (oldGCNames.contains(name)) {
            return "Full GC";
        } else {
            return name;
        }
    }

    public static String[] getGCNames() {
        List<GarbageCollectorMXBean> gcmbeans = ManagementFactory.getGarbageCollectorMXBeans();
        String[] rtnName = new String[gcmbeans.size()];
        int index = 0;
        for (GarbageCollectorMXBean gc : gcmbeans) {
            rtnName[index] = gc.getName();
            index++;
        }
        return rtnName;
    }
    
    public static void writeToFile(String gcData) {
        String currDir = System.getProperty("user.dir");
        BufferedWriter writer = null;
        
        try {
            File customFile = new File(currDir + File.separator + "custom.data");
            if (!customFile.exists()) {
                customFile.createNewFile();
            }
            writer = new BufferedWriter(new FileWriter(customFile));
            writer.write(gcData);
            writer.flush();
        } catch (IOException e) {
            System.err.println("Error to read custom monitor data:" + e.getMessage());
        } finally {
            if (writer != null) {
                try {
                    writer.close();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }

        }
    }
}

Some comments about the code are:

a) We need to pass the target process ID as arguments to run.

b) This code should be run under same java environment and VM option with the target process. Otherwise, the attaching will properly fail. And if the target tomcat process is running with “-server” option, also be aware to set “-server” option to run this java code.

Because under different java version and setting, the GC names will be different, I founded some names and their types(minor or full GCl). They are not complete and maybe not very correct. If you use IBM JVM, you should modify the code by yourself.

c) The custom values format is “minorGCCount,minorGCTime,fullGCCount,fullGCTime”.

d) This java code should be run in “${ngrinder_agent}/monitor/” directory. Because in the code, it will create custom.data file in current directory.

e) To compile the code, it needs “tools.jar” of JDK.  You need to compile it like this:

javac -cp /home/ngrinder/jdk1.6.0_38/lib/tools.jar GCMonitor.java

#get target tomcat process ID, it is 24003
java -cp /home/ngrinder/jdk1.6.0_38/lib/tools.jar: GCMonitor  24003

Then you should see the output in console as below:

current dir:/home/ngrinder/.ngrinder_agent/monitor
[Minor GC       : Count=3564    GCTime=27850]
[Full GC        : Count=166     GCTime=65525]
[Minor GC       : Count=3564    GCTime=27850]
[Full GC        : Count=166     GCTime=65525]

And in custom.data file, the content is:

3564,27850,166,65525

Then, create a new test on nGrinder and set the target host properly. You can see the chart in test report:

(Because the GC count is not changed very much, so we can not see much difference from the chart.)

 In this way, you can add any kind of custom monitor data and check it from test report.

nGrinder 3.2 and later supports not only Jython but also Groovy as scripting language. If you want to know about Groovy scripting, please refer [Groovy Script]

• You can see various The Grinder script samples.
• You can check The Grinder APIs.

• Do you want to know what script types nGrinder supports?
• [Jython Script Structure]
• [Groovy Script Structure]
• Do you want to ramp-up more precisely? 
• [How to ramp up by threads]
• Do you need a custom dns entry? Need to change the /etc/hosts file in the agents? You can configure target host fields for this. You don't need root permission in agent machines.
• [How to change DNS]
• Do you need to use your own jar or py? Read followings. 
• [How to use library]
• Do you need to load your own xml or txt file? Read followings. 
• [How to use resources]
• Do you need to simulate the multiple user scenario in the different run weight?
• [How to run the multiple tests with different weight]
• Do you need to adjust the logger level to only see the errors? Read followings.
• [How to adjust logger level]
• Do you need to test DB performance? Doesn't nGrinder seems to support this? No.. You can still test it with nGrinder
• [Using nGrinder to perform db load test]
• Do you need to parse data retrieved from server? 
  
• [How to parse JSON]
• [How to parse XML]

• This feature is available from 3.1

Users can share their account with others by User Account Share feature. If a user share his permissions to others, the other users can switch to his account anytime the others want. This feature is extremely useful when the developers in a team should work on the same script.

If you want to share own permissions to other user,  press Profile and select whom you want to share your account.  In above case, Juno make his account shared by Mavlarn and Alex.

When Mavlarn and Alex login, they can see Juno in the Switch To Other User menu.

Note:

Admins can switch other users by default.
Admins / Users can not switch to admins.

User can create/stop/delete a test in this page. The ball color changes depending on the test status and it will refresh every 5 seconds. By hovering on it, you can see each test status.

Test configuration

Test configuration is a critical aspect of a test. This sets the test duration, agent count, number of processes and threads etc in the test before running the test. Of course there are also more specific configuration items to satisfy a range of users.
 

 If you check "Enable Ramp-up", you can specify a gradual initialization of the process.

Please remember this is process ramp-up not thread ramp-up!!.. So you must have enough process count to increase the count gradually. If you set up 1 agent, 10 processes and 2 threads and you set the 1 in Ramp-Up, the count of vuser will be increased like 2, 4, 6, 8.... If you set 2 agents, it will be 4, 8, 12, 16...

The count of processes should be less than 10. Too many processes requires much agent memory. So you may not configure very smooth ramp-up. If you need to have more smooth ramp-up, please consider the thread ramp-up.

Currently, there will initially be a maximum of 10 tests running simultaneously by default, and one user can only run one test at the same time. All the agents will be assigned automatically to run some test. If there are not enough agents to run one test, that test will be delayed for several minutes, and then stopped as an error.
When a user creates a test, the user can choose just to save the test or prepare it to run. If it is prepared to run, the test (including its configuration) cannot be modified. But the user can clone this test and create a new test with the same configuration.
The test can also be scheduled to run later, rather than starting immediately.

After the test is started, the test running tab will be enabled and you can check the running status via this page.
When the test is finished, you can then check the report and the detailed report.
 

At the bottom right, you can download the log file for each agent. Using this file, you can identify script execution problems.

How to upgrade controller

Nothing is really necessary to upgrade controller. Just replace ngrinder-controller-{VERSION}.war in the your tomcat webapps folder. When you restart the tomcat, nGrinder detects the current database schema version and try to upgrade it to the version you're installing. Just it works.

How to upgrade agent

If you're upgrading controller, mostly previous version agents are just working with later nGrinder controller version if we don't mention it. If you'd like to upgrade the agent, just replace the current agent binary with the newer version. And run run_agent.sh or run_agent.bat.

${NGRINDER_HOME} - nGrinder Controller Home

If you start tomcat with "catalina.sh start" or "startup.sh", nGrinder will create ${user.home}/.ngrinder directory into user’s home directory after it starts up successfully. this directory contains the configuration files and data.
Following is .ngrinder default location.

• Window7   : C:\Users\${user.home}\.ngrinder
• Unix/Linux :${user.home}/.ngrinder

But if you like to assign the other directory for this, please set Environment Variables “NGRINDER_HOME” before running tomcat.

Advanced Configuration

${NGRINER_HOME} contains the following files and directories.

• database.conf – This contains the database configuration. You can modify this file when you need to use the other DB. The default DB nGrinder uses is H2.

  database=H2
  database_username=admin
  database_password=admin

  NOTE: If you like to use Cubrid for database, you need to add following configuration more

  database_url={your_cubrid_host_ip_or_name}:{cubrid_port_maybe_33000}:{dbname}

  NOTE: If you want to duplicate Cubrid DB with HA. Please enables HA in cubrid following the link and add the alternatives db address in database.conf

  database_url_option=&althosts={you_cubrid_secondary_host_ip_or_name}:{cubrid_port_maybe_33000}

• NOTE: Currently, we just support 2 types of database, H2,and Cubrid. If you want to use other DB, please modify org.ngrinder.infra.config.Database class to add a new one like Mysql. Or make a official request to us.

• system.conf – This file contains the general policy and configuration settings for nGrinder controller. You can modify these settings to calibrate nGrinder controller behavior.                                                        

  Name	Since	Default Value	Description
  verbose	3.0	false	Set true to print the more detailed log.
  usage.report	3.2	true	Set false if you don't want to report ngrinder usage to google analytics.
  security	3.0.1	false	You can specify whether nGrinder controller uses security mode or not. When the security mode is activated, Each test script is limited to access limited range of targets. Please refer [Script Security]
  agent.max.size	3.0	10	The maximum number of agents attached into one test. This is useful when you like to make nGrinder shared by more user. With this option, each test can use only limited number of agents. For example, if your nGrinder instance has 15 agents in total and you set this field 5, you can guarantee 3 users can run performance tests concurrently.
  agent.max.vuser	3.0	1000	The maximum number of vuser which can be created per one agent. If your agent machine spec is good enough, you can increase this.
  agent.max.runcount	3.0	10000	The maximum runcount of test per ech agent.
  agent.max.hour	3.0	8	The maximum running hour of each test
  ngrinder.console.portbase	3.0	12000	The starting port number of console which will be mapped to each test. You need to restart nGrinder to apply the this configuration.
  ngrinder.max.concurrenttest	3.0	10	How many concurrent tests are allowed in nGrinder.
  ngrinder.max.waitingmilliseconds	3.0	5000	How many milliseconds console will wait until all agents are connected.
  ngrinder.frontpage.rss	3.0	http://www.cubrid.org/ wiki_ngrinder/rss	You can point your own rss page on the nGrinder front page.
  ngrinder.help.url	3.1	http://www.cubrid.org/ wiki_ngrinder/entry/user-guide	If you want to provide your own custom help page. please modify this.
  ngrinder.langauge.default	3.1	en	language default en/kr/cn is available. This is useful when you're installing custom SSO plugin.
  ngrinder.security.sha256	3.1	false	By default, nGrinder uses sha1 to encode passwords. If you nedd shar256, please set true. You need to reinstall nGrinder to apply the this configuration.
  ngrinder.dist.logback	3.1.1	true	To be compatible with old agents (before 3.1.1), please set this true.  If you use the latest version of agents, just set it false.
  ngrinder.dist.safe	3.1.1	false	From 3.1.1, nGrinder doesn't check the file distribution result to speed up the test execution. If your agent is located in the far places or you distribute big files everyday, you'd better to change this to true.
  ngrinder.dist.safe.region	3.1.1	false	If a some region has the slow connection speed, set true.
  ngrinder.dist.safe.threashhold	3.1.1	1000000	Set the safe distribution threshold to enable safe distribution for specific transfer size by force.
  The followings are related to the clustering. It should be very carefully set. # You can refer http://www.cubrid.org/wiki_ngrinder/entry/controller-clustering-guide
  ngrinder.cluster.mode	3.1	false	if you want to enable nGrinder controller clustering mode, Set true. You need to restart nGrinder to apply the this configuration.
  ngrinder.cluster.uris	3.1	none	All controller IPs which belong to the cluster. You need to restart nGrinder to apply this configuration.
  ngrinder.cluster.listener.port	3.1	40003	Communication port for Cache synchronization in the cluster. You need to restart nGrinder to apply the this configuration.

• process_and_thread_policy.js – This file defines the logic to determine the count of processes and threads for the given vuser count. This file provides the flexibility to configure default process and thread allocation scheme. User usually don’t know which process and thread combination provides the best performance. Therefore, nGrinder let a user just input expected vuser per agent and configure the process and thread count automatically.
  The default content is like following.
  NGrinder  3.0

• function getProcessCount(total) {
      if (total < 2) {
          return 1;
      }
      
      return 2;
  }
  
  function getThreadCount(total) {
      if (total < 2) {
          return 1;
      }
      return parseInt(total / 2 + 0.5);
  }

  nGrinder; 3.1

  function getProcessCount(total) {
      if (total < 2) {
          return 1;
      }
      if (total > 80) {
          return parseInt(total / 30);
      }
      return 2;
  }
  
  function getThreadCount(total) {
      if (total < 2) {
          return 1;
      }
          if (total > 80) {
          return parseInt(total / (parseInt(total / 30)));
          }
      return parseInt(total / 2 + 0.5);
  }

  nGrinder 3.1.1

  function getProcessCount(total) {
      if (total < 2) {
          return 1;
      }
      
      var processCount = 2;
  
      if (total > 80) {
          processCount = parseInt(total / 40) + 1;
      }
      
      if (processCount > 20) {
          processCount = 20;
      }
      return processCount;
  }
  
  function getThreadCount(total) {
      var processCount = getProcessCount(total);
      return parseInt(total / processCount);
  }

  nGrinder 3.2

  function getProcessCount(total) {
  	if (total < 2) {
  		return 1;
  	}
  	
  	var processCount = 2;
  
  	if (total > 80) {
  		processCount = parseInt(total / 40) + 1;
  	}
  	
  	if (processCount > 10) {
  		processCount = 10;
  	}
  	return processCount;
  }
  
  function getThreadCount(total) {
  	var processCount = getProcessCount(total);
  	return parseInt(total / processCount);
  }
  

  
  You can freely rewrite this methods to fit your needs.
• grinder.properties – This file defines the default underlying The Grinder behavior. You can configure The Grinder behavior with this file. Some are overloaded by nGrinder runtime but some are not. In most cases, the admin doesn't need to change this file.
  Please refer http://grinder.sourceforge.net/g3/properties.html for detail.
• plugins folder – In this folder, you can locate nGrinder plugin. Just drop the plugin into this folder. If you want to check the available plugin, refer [nGrinder Plugins]

Data Structure

In ${NGRINDER_HOME}, there are some folders to store data used in nGrinder. Followings are description for them.

Change list:

• This feature is available from 3.1

We enhanced nGrinder to support clustering from 3.1.

If you don’t know about nGrinder cluster, please refer to [Cluster Architecture]. Following shows how to set up nGrinder cluster mode. Assuming you have been familiar with nGrinder in single instance mode. All nGrinder controllers should use the same DB and file system in cluster mode. What we should do at first is to make a shared folder on the NFS and make all controllers point this as ${NGRINDER_HOME}. And edit ${NGRINDER_HOME}/system.conf like followings.  

# if you want to enable ngrinder controller clustering. please enable below. 
ngrinder.cluster.mode=true

# list of controller IPs
ngrinder.cluster.uris=xx.xx.xx.xx;xx.xx.xx.xx

# cache cluster listener port.
ngrinder.cluster.listener.port=40003

Next step is to create the each controller's specific information. For this ${NGRINDER_EX_HOME}/system-ex.conf file is used. Not like ${NGRINDER_HOME}, ${NGRINDER_EX_HOME} is not automatically created by nGrinder. Therefore you should make it in the local file system not to be shared by other controller. Create ${NGRINDER_EX_HOME}/system-ex.conf file and specify the region name here. The controller which reads this configuration will act as specified region controller.

# Region setting of this controller.
# When running nGrinder in cluster mode, the ${NGRINDER_HOME} should be shared by other controllers.
# We expect this to be done by running ${NGRINDER_HOME} on NFS. 
# However region setting can not be set on ${NGRINDER_HOME} because each controller in cluster must have its own configuration.
# Therefore, nGrinder supports one more configuration store folder ${NGRINDER_EX_HOME}
# Please locate following setting in the ${NGRINDER_EX_HOME}(by default ${user.home}/.ngrinder_ex)/system-ex.conf
ngrinder.cluster.region=Beijing

After finish this configuration, just restart all nGrinder controllers and login again. We will see this icon at headline!

If you want to put the L4 or load balancer in front of all controllers, you should set it up as sticky session so that each client connects only one controller. Otherwise, users may login again whenever they access difference controller.

After the controller configuration, you should also set the region name to all agents. As you should know, we should set the connecting controller IP  in each agents. Now there are several controllers. So you should choose one of them depending on what region you want this agent to serve for. You need to add the not only connecting controller IP but also region name this controller handles.

• agent.console.ip=controller ip
• agent.console.port= controller port
• agent.region=controller region name

If this agent is [User Agent], it should be like:

• agent.console.ip=controller ip
• agent.console.port= controller port
• agent.region={controller region name}_owned_{userID}

Below is the architecture without cluster supports.

1. nGrinder uses an embedded svn server(SVNKit DAV) to manage script files. The svn repositories are stored in ${NGRINDER_HOME} directory.
2. nGrinder uses EhCache to improve the data retrieval performance from DB and SVN repositories.
3. nGrinder uses Spring Security to protect the system and uses Atlassian Plugin Framework for extensibility.

If you enable clustering mode by configuring the system.conf (See link) and install nGrinder into multiple machine, the system architect is changed as below:

1. All the controllers in the cluster shares the same DB and file system. All ngrinder controller should point same ${NGRINDER_HOME} folder which should be shared by NFS.
2. Each controllers can have its own special properties and log output folder. This information will be saved in ${NGRINDER_EX_HOME} in the each controller.
3. All controllers replicates the EhCache to each other to make some data to be visible in all controllers in cluster.

Each controller can serve nGrinder web content equally but handles the different test set depending the region. We don't provide any session clustering by default. So you may suffer login problem because sessions stored in a controller are not replicated to the other controllers. You may need to configure this by referring Tomcat Session Clustering Guide or use the sticky session on L4. If you are not a expert on this, Just let users to connect only one controller. It's the easiest. :-)

How to configure the clustering? Check [Controller Clustering Guide]

• Bootstrap : Bootstrap has a lot of predefined web style(js, css etc ), it is sleek and intuitive.We uses it span all over the nGrinder web pages.
• Bootstrap-slider:Bootstrap-slider is a beauty of Bootstrap UI widgets. We use this for test duration setting in Test Configuration.
• Jquery :jQuery is a lightweight JavaScript library,it make much easier to use JS on nGrinder web pages.
• Jquery Validate : Jquery Validate is an excellent tool for web pages to validate data entries at the client side.
• Datatables : DataTables is a highly flexible JS tool which will add advanced interaction controls to any HTML table.
• Select2 : Select2 is jQuery based replacement for select boxes which supports searching and infinite scrolling of results.
• JqPlot :  jqPlot could generate pure client-side JS charts in web pages and therefore all nGrinder data chart are implemented with JqPlot.
• CodeMirror :CodeMirror is an excellent code editor with syntax highlighting in the browser.We use it in nGrinder Script Editor.

Controller Tier

• FreeMarker :FreeMarker is a Java-based template engine focusing on MVC framework and nGrinder web pages is used it as a template engine.
• Spring Security :  Spring Security is not only a powerful and highly authentication and access-control framework, but also one of the most mature and widely used security framework. Therefore it protects nGrinder from illegal access.
• Spring MVC : Spring MVC provides rich functionality for building robust web applications. nGrinder uses wonderfully its annotation based configuration.
• GSon :Gson is a Java library that can be used to convert Java Objects into its JSON representation, and vice versa.
• SVNKit Dav : We use this for serving the underlying SVN content thought web.

Service Tier

• Grinder : Grinder is a java load Testing framework which nGrinder mainly use. Without Grinder, no nGrinder exists
• Atlassian Plugin Framework :We use this framework to make the plugin development easier. It's not so heavy weight and easy to use!
• Spring  : Spring Framework provides a comprehensive programming and configuration model for J2ee applications and nGrinder use as a its base architecture.
• EhCache : Ehcache has excellent Spring integration. nGrinder uses this to speed up the data retrieval and to synchronize data between controllers in cluster mode.

Data Tier

• Spring Data :Spring Data simplifies the development of JPA-based data access layer. nGrinder uses RDBMS but this enables the RDBMS access without single line of SQL.
• Hibernate : Hibernate is a powerful technology for persisting data,and it is Spring Data back-end within nGrinder.
• H2 : H2 is one of two databases nGrinder supports and H2 is used as a default DB.
• Cubrid :Cubrid is one of two databases nGrinder supports. It's stable enough and supports HA.
• Liquibase : Liquibase is an open source that automates database schema updates. It supports almost all kinds of major databases.
• SVNKit : SVNKit is a pure Java toolkit which implements all subversion features and provides APIs to work with Subversion.

Depending on how you have configured Tomcat, you may use a different URL to access the main page of nGrinder. If you didn’t change ${TOMCAT_HOME}/conf/server.xml and didn't change nGrinder controller war file name to ROOT.war, nGrinder should have the following URL.

http://your_server_host:8080/ngrinder-controller

Once you access the URL, you can see the login page.

Login with Default Account

nGrinder creates 4 users at the first run. The predefined IDs are as follows.
Each has name that reflects its role.

• admin, user, superuser, system
• Initial password is the same as name

There are 4 user types (referred to as Roles)

To configure the system, you need to log in with an admin account.

NOTE:  Please don’t forget to change your password when you log in for the first time. If you leave your account with the default password, your nGrinder instance can be used by a hacker for a DDOS attack.

Manage Agents

Agent management in the admin menu offers agent operation and monitoring functions. Even when you have configured the agent to connect the controller, you need to approve each agent in the controller to make it effective. This step is required to prevent abnormal agents trying to connect to the controller from abusing the controller.

If you are admin, the menu above should be visible. You can click the Agent Management menu to manage agents. 

Then, you should see the screen shown above. This shows the statuses of the attached agent. If we just allow all agent connections, there would be a potential risk. For this reason, nGrinder only uses approved agents. If you confirm that the agents listed here are OK, you can approve them. Some agents can be only used for specific users. Please refer to [User Agent] for details.

You can monitor agent performance status by clicking each IP.

This graph shows the performance of each agent in the last 1 minute.

User Management

Admin can create/delete and modify users through the user management menu. Each user can also modify his/her own profile as well.

Script Console

This is a useful admin tool for debugging. Admin can input Jython code to monitor and control nGrinder internal status. for example:

print agentManager.getAllAttachedAgents()

It is important to know what you can refer in the script. The following variables are available.

• applicationContext (org.springframework.context.ApplicationContext)
• agentManager (org.ngrinder.perftest.service.AgentManager)
• consoleManager (org.ngrinder.perftest.service.ConsoleManager)
• userService (org.ngrinder.user.service.UserService)
• perfTestService (org.ngrinder.perftest.service.PerfTestService)
• fileEntryService (org.ngrinder.script.service.FileEntryService)
• config (org.ngrinder.infra.config.Config)
• pluginManager (org.ngrinder.infra.plugin.PluginManager)

Please refer to nGrinder javadoc to learn the available classes and methods.

Log monitoring

Sometime it’s important to see the server log to identify a problem and send a bug reports to the developers.

Instead of catalina.out which tomcat uses by default, nGrinder intercepts the logging system and writes a log in the ${NGRINDER_HOME}/logs/ngrinder.log file.

You can see the currently printing logs in the Log Monitoring menu. This is refreshed every 5 seconds in order to show the latest available logs.

Test Policies

Sometimes, you need to configure whole test execution policy. For example, you can set the maximum count of agents that each test can use. Or you can limit the max concurrent number of tests to minimize network loads. These kinds of policies can be configured by configuring the ${NGRINDER_HOME}/system.properties file.

For further details, please refer [Advanced Controller Configuration].

We proudly release nGrinder 3.1.1 to speed up the test execution and enhance the usability. With some bug fix, the following improvements are included. To ugrade to this version, you should delete the current version first and install this. In addition, you should delete the previous ${NGRINDER_HOME}/process_and_thread_policy.js in advance to experience the better process and thread calculation introduced by nGrinder 3.1.1.

1. Release Info

2. Changes

However if you want to simulate real user behaviors and really want to run the script with the huge count of virtual users, follow the below instructions. For example... if you want to have 10000 vusers,

1. Prepare the 2 good agent machines. You may need machines with 2 cores and 6G memory.
2. Write the script with Groovy (Groovy script is more efficient than Jython)
3. Increase the vuser limit by setting agent.max.vuser to 5000 in system.conf file.
4. Simulate thinking time with grinder.sleep(how_much_sleep_in_milliseconds) in the script
5. When you configure the vuser count, please set the process around 10 and thread around  500.

It is always possible for a test to cause the memory related problem. So you need to set process and thread count very carefully.
Please refer [Process And Thread] for detail.  If you need more vusers, please consider add more agents.

Actually in our company NHN, we set the vuser limit as 2000 with 4GB ram and 2 core CPU agent machines. nGrinder agents are very stable with that number. We expect that 8000 vusers can be run if you have 8GB and 4 core CPU machines. 

If you don't know how much vusers you should use, there is a good vuser calculator in our competitor's site.    It's really good products. :-) 

If you put the virtual user count in Vuser per agent field, nGrinder calculates the appropriate processes and threads. For example if you put 100 in the field, it will turn to 99. It’s because 3 processes and 33 threads will be used. You can configure process-thread count calculation logic by configuring process_and_thread_policy.js. 

Processes more than 10 easily cause thrashing by memory swapping in the agent if your agent has less than 4GB memory. If there are many number of processes, they would exceed the agent machine’s physical memory size. Please set it less than 10. If you increase the thread number more than 200, it might cause problems either. The threads belong to one process share the same heap. It’s obvious to see the OOM error if a lot of threads occupies the process memory.

Therefore, you have to set process and thread count with care. If you have no idea on this, Just let the nGrinder calculate automatically.

NEWS!! we have some experiment how many vusers a agent can handle. 4000 vusers can be supported by a single 4GB memory agent if each test doesn't use takes much memories. If you like to increase the max vuser limitation, please configure agent.max.vuser in system.conf.

In Mac and Linux, the embedded browsers(JxBrowser) are also initialized. However we found there are some frequent crash during navagation and HTTPS messages are not recorded because there are no way to work around the private certificate issue. Therefore, we highly recommend you to use the separated browsers(Firefox or Safari) connecting to the proxy instead of using the embedded browser. Following shows how to do it.

As you can see below, the proxy is initiated using 10288 port.

If you like to record HTTP/HTTPS with Firefox, Open Firefox and go to Preferences and Select Advanced ==> Settings in the popup window.

Set up the both HTTP and HTTPS proxies for both to the given port.

Then, navigate the web using the separate browser. Don’t forget that you should click Start Record button before navigation.

Then click "Stop Recording" in nGrinder Recorder. Then you can see the recorded script.

Please don’t forget to disable proxy settings when stopping recording. Otherwise you can not browse the Internet afterwards.

When you navigate HTTPS sites, you may see the following alert page. It’s because HTTPS messages are encrypted by the self-signed certificate which nGrinder issued. To ignore this, you should click “I Understand the Risks” and add the exception.

nGrinder Recorder initiates JXBrowser and TCPProxy when the user starts nGrinder Recorder. A proxy is initiated with the default proxy port 10288. If the port is occupied by the other programs, it automatically selects the available port. Then this proxy port is assigned to JXBrowser instances(embeddable native web browser component). All requests from JXBrowsers go through the attached proxy port. The proxy monitors the messages sent to the proxy port and put the connection target information into ConnectionFilter component so that a user can select which messages should be recorded depending on the target web server. When a user starts recording, all messages which is not filtered are saved in HTTPRecording component. TCPProxy looks message contents both in requests and responses, and saves meaningful message information (like URL, HTTP headers and form data) into HTTPRecording component. Recording includes messages not only HTTP but also HTTPS as well.

However when you browse HTTPS web site, you may see alert message box. Basically HTTPS messages are  protected by the certificate from the target web server. The proxy in the middle of messaging can not intercept HTTPS messages to see the message content in the normal way. TCPProxy which nGrinder Recorder is based on solves this by using the private certificate between the browser and TCPProxy. HTTPS messages between a browser and DelegateSSLEngine(in TCPproxy) is encoded by the private certificate issued by TCPProxy. Therefore DelegateSSLEngine can see the message contents. Then, it sends and receives the message to/from the target web server by encoding/decoding the requests/responses with the web server issued certificate. It causes the browser side warning message. See more details about this in Recoder Limitation.

When a user stop the recording, the recorded messages are analyzed to generate more generic script. In this step, the relations between requests/responses are linked as much as possible. Finally these are passed to HTTPRecordingResultProcessorWithFreeMarker component. Finally, HTTPRecordingResultProcessorWithFreeMarker processes them with the script template written in FreeMarker formats to generate Jython or Groovy scripts.

We don’t put much energy on this product. We believe the manually written script is always better than generated script. Therefore we didn't try to make this as a complete and bug free sw.

• [Recorder-Architecture]
• [Recorder-Limitation]
• [Recorder-ScreenShot]
• [Recorder-Installation]
• [Recorder-Quick-Start]
• [Recorder-Recording-Guide]
• [Recorder-Recording-Guide-For-Non-Window-User]
• [Recorder-Generated Test Script Structure]

• nGrinder Recorder can run in Windows / OS X / Linux because it is implemented in Java. At least you requires followings to run nGrinder Recorder.
  • Java Runtime (over 1.6)
  • GUI enabled env (not supporting headless OS)
• However, Some major features of nGrinder Recorder can work in Windows due to some bugs in internal components.
  We’ll continue to work to make some disabled features work in the other OS. 
• Check out [Recorder-Limitation] first. 

Installation

• Currently nGrinder Recorder is provided in the form of java webstart.  You can install it just by double-clicking thd downloaded jnlp file from URL in the browser.
  • http://ngrinder-demo.nhnopensource.org/jnlp/recorder/ngrinder-recorder.jnlp
• Then the following warning message will show up. 
  
• Check “Always trust content from the publisher” and click Run. nGrinder Recorder requires full permission in your PC because the underlying JXBrowser requires it.
• Then..  If your PC’s firewall monitors the port opening, the security alert message may come up. You should allow nGrinder Recorder to open the port appropriately.
  
• If you can see the following window, You’re ready to use nGrinder Recorder. 
  
• Copy the downloaded jnlp file into somewhere in your desktop to run nGrinder next time.

Configuration

• Just like nGrinder Controller and Agent, nGrinder Recorder uses the configuration in the ${user.home}/.ngrinder_recorder folder.  You can open recorder.conf file to configure followings. However it's not necessary.

  key	value
  recorder.additional.headers	The recorder only records the limited set of headers. If you want to record more headers, you can specify this value with header names separated by comma. Check the detailed info here.
  keystore.password	A user can provide the same certificate which is used already for the webserver HTTPS communication.  A user should locates the keystore in the ${user.home}/.ngrinder_recorder/keystore file. Then specify the password of this keystore here.
  keystore.type	A user can specify the keystore type. The default value is jks.

For now, I’d like to explain the basic Groovy script structure. This is applied whether you choose Groovy Script or Groovy Maven Project.
Following is a base template for groovy script.

import static net.grinder.script.Grinder.grinder
import static org.junit.Assert.*
import static org.hamcrest.Matchers.*
import net.grinder.plugin.http.HTTPRequest
import net.grinder.script.GTest
import net.grinder.script.Grinder
import net.grinder.scriptengine.groovy.junit.GrinderRunner
import net.grinder.scriptengine.groovy.junit.annotation.BeforeThread
import net.grinder.scriptengine.groovy.junit.annotation.BeforeProcess

import org.junit.BeforeClass
import org.junit.Test
import org.junit.runner.RunWith

import HTTPClient.HTTPResponse

/**
 * A simple example using the HTTP plugin that shows the retrieval of a
 * single page via HTTP.
 *
 * This script is auto generated by ngrinder.
 *
 * @author ${userName}
 */
@RunWith(GrinderRunner)
class Test1 {

    public static GTest test;
    public static HTTPRequest request;

    // This method is executed once per a process. 
    @BeforeProcess
    public static void beforeClass() {
        test = new GTest(1, "${name}");
        request = new HTTPRequest();
        test.record(request);
        grinder.logger.info("before process.");
    }

    // This method is executed once per a thread. 
    @BeforeThread
    public void beforeThread() {
        grinder.statistics.delayReports=true;
        grinder.logger.info("before thread.");
    }

    // This method is continuously executed until you stop the test 
    @Test
    public void test(){
        HTTPResponse result = request.GET("${url}");
        if (result.statusCode == 301 || result.statusCode == 302) {
            grinder.logger.warn("Warning. The response may not be correct. The response code was {}.", result.statusCode);
        } else {
            assertThat(result.statusCode, is(200));
        }
    }
}

The Groovy test case in nGrinder should be annotated with @RunWith(GroovyRunner). When you run this script in JUnit Runner of each IDE, this part makes GroovyRunner to control the JUnit behavior and mount the grinder context on JUnit.
@Test annotated methods are subject to be executed repeatedly. As you can see above, you can assert the test result using JUnit assertion. If the assertion is failed, the last test bounded to this thread will be evaluated as a failure.

Instead of @BeforeClass @Before @AfterClass @After annotations which are used  in the generic JUnit tests, Groovy test case in nGrinder uses its own annotation to control script behaviors.

The execution flow can be illustrated by following.

In the test case sample, Please look below part carefully.

    public static GTest test;
    public static HTTPRequest request;

    // This method is executed once per a process. 
    @BeforeProcess
    public static void beforeProcess() {
        // Instead of Test in Jython, GTest is used here
        // It's because the identifier "Test" is alredy used by the @Test 
        // GTest is the replacement to avoid the naming confliction.
        test = new GTest(1, "test name");
        request = new HTTPRequest();
        test.record(request);
        grinder.logger.info("before process.");
    }

In @BeforeProcess annotated method, it defines GTest instance and records the HTTPRequest instance to sense any calls on HTTPRequest instance and save it in the static member variable(request).
It’s the first step of test statistic preparation. Any method calls on request variable will increase the TPS. If you want to record multiple HTTPRequest instances using multiple GTest named differently, You can just add GTest instance in another member variables and invoke record method on different HTTPRequest instance.  

You can define the multiple test methods in a single test case class as well. Just add @Test annotation on those like the following. 

    @Test
    public void testGoogle(){
        HTTPResponse result = request.GET("http://www.google.com");
        if (result.statusCode == 301 || result.statusCode == 302) {
            grinder.logger.warn("Warning. The response may not be correct. The response code was {}.", result.statusCode);
        } else {
            assertThat(result.statusCode, is(200));
        }
    }

    @Test
    public void testYahoo(){
        HTTPResponse result = request.GET("http://www.yahoo.com");
        if (result.statusCode == 301 || result.statusCode == 302) {
            grinder.logger.warn("Warning. The response may not be correct. The response code was {}.", result.statusCode);
        } else {
            assertThat(result.statusCode, is(200));
        }
    }

However, you should aware nGrinder GroovyRunner creates only one test case object per a thread not like that generic JUnit test cases which creates a separate test case object for each @Test annotated method. So if each @Test annotated methods share member variables, they can affect each other. It’s intended. In nGrinder test case, it’s very likely each @Test method relies on the previous @Test annotated method execution result. Therefore, if you save the previous test result in the member variable and refer it to decide to continue to next test method, you can easily implement the test dependency.

    private boolean googleResult;
    
    @Test
    public void testGoogle(){
        googleResult = false;
        HTTPResponse result = request.GET("http://www.google.com");
        if (result.statusCode == 301 || result.statusCode == 302) {
            grinder.logger.warn("Warning. The response may not be correct. The response code was {}.", result.statusCode);
        } else {
            assertThat(result.statusCode, is(200));
        }
        googleResult = true;
    }

    @Test
    public void testYahoo(){
        if (!googleResult) {
            grinder.logger.warn("Just return. Because prev google test is failed.");
            return;
        }
        HTTPResponse result = request.GET("http://www.yahoo.com");
        if (result.statusCode == 301 || result.statusCode == 302) {
            grinder.logger.warn("Warning. The response may not be correct. The response code was {}.", result.statusCode);
        } else {
            assertThat(result.statusCode, is(200));
        }
    }

How can you use library and resources?  If you don’t use [Groovy Maven Structure] yet, you can refer [Jython Script Structure] to how to include them in your script. It’s exactly same as Jython way. However if you use Maven structure plus Groovy, you can enjoy the easier library dependency setting and run the JUnit test in your IDE. Interested in this? Then continue to see the [Groovy Maven Structure] to know how to enable this for you.

Groovy Maven Project is structured for users to easily import the project into IDE. If you already have an IDE which supports Groovy and Maven, you can easily import it. However if you don’t have it yet, please follow the instruction in [Install Groovy IDE] first. 

If you have installed the Eclipse based Groovy IDE and created Groovy Maven Project in nGrinder subversion, you can import the Groovy Maven Project using the subversion client in Eclipse.

Notice that there is the subversion URL field in the script management page in nGrinder. The above screenshot shows the our test instance. In this instance, the created Groovy Maven project can be accessed through http://ngrinder-staging.nhncorp.com:8080/svn/admin/project URL according to the above page .
OK. Let’s import the project.

1. Open your Eclipse and Choose File ==> Others ==> Maven ==> Check out Maven Projects from SCM
   
   
2. Select svn in the SVN URL and type the SVN URL provided in nGrinder. Then click Finish.
   
3. Then the project will be imported.
   
4. Enable Groovy on this project.
   
5. Remove groovy libraries from classpath by selecting "Groovy ==> Remove Groovy libraries from classpath".
   It's because there is a conflict between groovy libraries provided by Maven and Eclipse.
   
6. Select the Groovy test case and Select Run As ==> JUnit Test.
   
7. Then you might see the following error. It’s because the specialized nGrinder JUnit Runner(named GrinderRunner) needs a special JVM argument for instrumentation
   
8. Check the error message and add the JVM argument following the instruction. Select Run As ==> Run Configurations ==> Select Test1 ==> Select Arguments panel.
   
   Please be aware that the argument varies depending on the user account. So provide the exact argument which is shown in the JUnit error message. This setting should be provided for every JUnit test running set. If you like to make it as a default vm option for every test run. You can set it up in the global configuration. Please check [Install Groovy IDE] for details.
9. Then run as JUnit test again.
   
10. In my case, the access to please_modify_this.com causes the error. Ater changing this to http://www.google.com . It shows the green bar now.
    
11. You can add the break point in the test case. Then select Debug As ==> JUnit Test.
    
12. Then you can debug the test case using the generic Eclipse debugging feature.
    
13. When you think the test case writing is almost completed, you may want to see the result in the repeated run situation. You can provide the @Repeat annotation on the test case class for this need. Then run as JUnit test again.
    
14. Then you can see that test case is executed multiple times you specified. Please note that it’s 100 repetition by single thread. (not multi thread or multi process)
    
15. If you think the test case is ready, commit it into subversion selecting Team ==> Commit. Finally run this test in nGrinder.

When you create Groovy Maven project, you can use IDE to write and debug test cases. Groovy maven project use maven’s pom.xml to structure src folders and specify dependencies. Therefore if your IDE supports Groovy and Maven project, you can use it. For who don’t have IDE yet. This explains how to install Eclipse based Groovy Maven IDE.

1. Download and unzip the lastest “Eclipse IDE for Java Developers” which fits to your OS.
• http://www.eclipse.org/downloads/packages/eclipse-ide-java-developers/junosr2
• The above version already includes Maven Eclipse plugin but Groovy and Subversion. You need to separately install them. 
• Firstly, run the downloaded eclipse.
2. Install Groovy
1. Go to “Help” ==> “Eclipse Market Place”
2. Type "groovy" in the search field. Then click the install button in Groovy Eclipse for Juno. (Aha! It's my name Juno :-) Kidding)
   
3. Then install all.
   
4. Then click “Next”s and “Finish”s and restart Eclipse until the installation is completed.
3. Install Subversive 
1. Go to “Help” ==> “Eclipse Market Place”
2. Type "subversive" in the search field. Then click install below.
   
3. Select Subversive SVN JDK Ignore Extensions and Team Provider. Then select “Next” buttons until the installation is started.
   
4. Restart Eclipse and and select SVN Kit 1.7.8. Subversion repo provided by nGrinder is compatible with above SVN Kit 1.7.8.
   Then click “Next”s and “Finish”s and restart Eclipse until the installation is completed.
   
5. Now it’s time to connect Maven and Subversion by installing m2e subversive connector. Click File menu ==> New ==> Checkout Maven Projects from SCM.
   
6. Then click m2e Marketplace in the next dialog.
   
7. Check the m2e subversive and click Finish button.
   
   Then click “Next”s and “Finish”s and restart Eclipse until the installation is completed.
8. Groovy JUnit Running requires the special JVM argument for every test run. It can be very annoying. If you plan to use this Eclipse instance only for nGrinder Test Case, it might be a good choice to provide the default JVM options for every JVM execution in this Eclipse. 
   Go To Window ==> Preference ==> Java ==> Installed JRE. Then click Edit on the default JRE.
   
   Finally you can provide the default VM arguments in the above field. The VM arguments you should provide here varies depending on the user configuration. Please check [Import Groovy Maven Project in IDE] to know which VM arguments you should provide.

Now your IDE is ready to run and debug your nGrinder groovy test cases. Go to  [Import Groovy Maven Project in IDE] 

When you run the groovy maven project, it might take time to run the test at the first time. It's because maven project requires to download the related libraries from maven repo. Please be patient. :-).

If you choose to use Groovy Script when you create a script, it works similar to Jython script except the JUnit styled test cases. However if you want not only to use Groovy but also to execute JUnit runner in your IDE or easily configure dependencies, you can choose to make Groovy Maven Project first.

If you create Groovy Maven Project, you will have the following directories containing three files under the specified project folder.

You can import this project into you IDE using SVN checkout. You may need to check the below links.

• [Install Groovy IDE]
• [Import Groovy Maven Project in IDE]

In pom.xml, You can specify the additional library dependencies.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>ngrinder</groupId>
    <artifactId>${name}</artifactId>
    <version>0.0.1</version>

    <properties>
        <jdk-version>1.6</jdk-version>
        <source-encoding>UTF-8</source-encoding>
    </properties>

    <repositories>
        <repository>
            <id>nhnopensource.maver.repo</id>
            <url>https://github.com/nhnopensource/nhnopensource.maven.repo/raw/master/releases</url>
        </repository>
    </repositories>

    <dependencies>
        <!-- Following dependency is mandatory -->
        <dependency>
            <groupId>org.ngrinder</groupId>
            <artifactId>ngrinder-groovy</artifactId>
            <version>3.2</version>
            <scope>provided</scope>
        </dependency>

        <!-- Add the dependencies -->
        <!--  <dependency>
            <groupId>commons-io</groupId>
            <artifactId>commons-io</artifactId>
            <version>2.0.1</version>
        </dependency>
         -->
        <!-- Put your private library like this -->
        <!--  
        <dependency>
            <groupId>your_lib</groupId>
            <artifactId>your_lib</artifactId>
            <version>your_lib_version</version>
            <scope>system</scope>
            <systemPath>${project.basedir}/lib/hello.jar</systemPath>
        </dependency>
        -->
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>${jdk-verion}</source>
                    <target>${jdk-version}</target>
                    <encoding>${source-encoding}</encoding>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-eclipse-plugin</artifactId>
                <version>2.9</version>
                <configuration>
                    <additionalProjectnatures>
                        <projectnature>
                            org.eclipse.jdt.groovy.core.groovyNature
                        </projectnature>
                        <projectnature>
                            org.eclipse.m2e.core.maven2Nature
                        </projectnature>
                    </additionalProjectnatures>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

If you like to add the private libraries which do not exist in the public maven repository, you can put them in the lib folder directly and specify the systemPath option like the above sample. lib folder will be automatically detected and copied into agents as well when the controller prepares the script distribution.
When a script is selected in the performance test configuration page, the controller automatically detects whether the script is under the folder ${name}/src/main/java and there is pom.xml in the base folder. If the controller detects it, the controller will copy the files in the ${name}/src/main/resources and ${name}/src/main/java folder recursively into controller’s distribution preparation folder before distributing them to agents. The below figure illustrates how each folder in svn are copied to the distribution preparation folder.

- from svn foler + pom.xml
                 + src/main/java/Test1.groovy
                                /package_names/the_other_groovyscripts
                 + src/main/resources/resources1.txt
                                     /subfolder_names/the_other_resources
                 + lib (private libraris)


- to dist folder + Test1.groovy 
                 + resources1.txt 
                 + package_names/the_other_groovy_scripts 
                 + subfolder_names/the_other_resources 
                 + lib (copied from maven dependencies and files in lib folder existing subversion)

In generic Jython script and Groovy script, the resources are loaded using command "open(“./resources/resource1.txt”)" or "new File("./resources/resource1.txt"). However this won’t work in Groovy Maven project. We had to replace this path based resource discovery into the classpath based resource discovery to make Groovy JUnit test workable in IDE as well, 
All resources in ${project_name}/src/main/resources/.. are copied into the same folder where the test script is copied but keeping the sub directory hierachy. You can load the resources using code below.

import org.codehaus.groovy.reflection.ReflectionUtils;
....


class YourTest {
    String text;

    @BeforeThread
    public void beforeThread() {
       // In groovy, InputStream contains text field.
       text = loadResourceFromClassPath("/resource1.txt").text;
    }

    @Test
    public void doTest() {
       ....
    }

    // This is groovy way to load resource from classpath
    public loadResourceFromClassPath(String resourcePath) {
        return ReflectionUtils.getCallingClass(0).getResourceAsStream(resourcePath);
    }
}

• [Groovy Script Structure]
• [Groovy Maven Structure]
• [Install Groovy IDE]
• [Import Groovy Maven Project in IDE]

You can start recording by clicking Start Recording button on the right panel. Then, navigate the site you want to visit in the browser panel.

On the right panel, www.google.com / www.google.co.kr are shown, These are hosts to which the browser connected. Basically connecting to www.google.com is forwarded to www.google.co.kr  in Korea . So it shows two connections. Just Ignore www.google.com by unchecking www.google.com host now.

From now on, the messages sent to www.google.com will not be recorded. Then type the text you want to search in the browser.

Google will automatically switches the main page into the search page due to its instant search feature.

While doing this, you may notice that the count of recorder messages to www.google.co.kr is increased and new hosts ssl.gstatic.com and t1.gstatic.com show up. These two hosts may have some static resources like javascript, images. Just uncheck these hosts.

Then click Stop Recording..

Then you can see the generated script only contains the connection to www.google.co.kr.

There in the main __call__ function which nGrinder threads will run eventually. It calls three different pages. (page1 / page2 / page3). All HTTP calls are organized by page methods which are generated per the user click events on the web page. There are too much calls in this recording more than 600 lines. We can shorten these by ignoring some static resources retrieved from www.google.co.kr. Uncheck followings in the right panel.

Then click Start Recording and Stop Recording again. Then you will see little bit shorten script. Then you can shorten unnecessary part more by editing sript directly.

In this case, while typing “ngrinder” in the search field, n, ng, ngr, ngri, ngrin…. queries were sent to the server. If you need to record all of these, just leave it. However in my case it is useless, I deleted all except "ngrinder" query. Then copy and paste the generated script into Script Editor in nGrinder controller and run validation check.

Is it complex? nGrinder Recorder is a helper application not perfect one.

• i18n. We're supporting English, Korean, Chinese currently. If you like to provide your own language translation. Please fork our nGrinder project and do pull request with translation. Check the English message file here. https://github.com/nhnopensource/ngrinder/blob/master/ngrinder-controller/src/main/resources/messages_en.properties
  Because we are not natvie English speaker, there are a lot of typo mistakes and not smooth expression. If you find some, please let us know.
• OS compatibility. When we release, we test nGrinder on Windows and CentOS mostly. However there are many OS we didn't test much. For example Ubuntu, OS X, Fedora, etc... If you expect that nGrinder runs on your OS well, please participate the our BETA testing. We'll give you the BETA version to you and we'll make it works on your OS. Please contact junoyoon at gmail.com
• The Grinder plugins : The underlying engine, The Grinder supports plugins so that statistics on tests using the other protocol can be collected. If you like to have your own plugin and make it as a default nGrinder plugin as well. Please share it to us.