Open Source RDBMS - Seamless, Scalable, Stable and Free

한국어 | Login |Register

Current Events
Join our developers event to win one of the valuable prizes!
posted last year
viewed 19484 times
Share this article

Test Automation using Hudson and Selenium at NHN Services

NHN, the company behind CUBRID open source database development, is automating tests for diverse browsers (Chrome, Firefox, Internet Explorer, Opera, iOS Safari and Android) that run on a variety of operating systems (Linux, Microsoft Windows, Mac OS, iOS and Android) with a set of test codes by using Hudson and Selenium WebDriver. In this article, I will briefly summarize the test automation process we use at NHN's NAVER portal and the benefits of Hudson and Selenium WebDriver for automation.

How to Automate Tests

I registered a test code to Hudson, Continuous Integration (CI) tool, and set the test code to be executed by using Selenium WebDriver and JUnit whenever source is committed in SVN. It takes about 2 seconds to test a browser. When a problem occurs, the problem is reported within 10 minutes.

Hudson is commonly used in most development departments of NHN. By clicking Build Now, users can view the test progress status. The report system provides a good environment for "Fast Fail, Fast Feedback".

multi_browser_test_with_one_set_of_test_code.png

Figure 1: Multi-browser Test with One Set of Test Code.

Automating Firefox Test in Linux

Install xserver package on Linux server with Hudson. Modify run level to 5 and then add the following setting to an account where CI runs in order to execute a browser in a text console.

Xvfb :1 -screen 0 1024x768x24 > /home1/irteam/log/xvfb.log &

Install the latest Firefox version under the /usr/lib64 directory and replace the symbolic link. Now you can execute automation tests by running Firefox in Hudson.

Automating iPhone and iPad Tests in Mac OS X and Simulator

For tests, install Hudson on Mac OS X. It requires the iPhone or iPad Simulator, so you should install Xcode in advance.

Automating Internet Explorer and Opera Tests in Microsoft Windows

For more details on how to automate tests in Microsoft Windows, see http://seleniumhq.org/docs/03_webdriver.html.

Why Is Browser Test Automation with Hudson Required?

If tests for several browsers are automatically executed whenever a source code is changed, it would clearly help developers to reduce bugs and keep the source tree that can be released anytime. In addition, QA time can be significantly reduced. There are many benefits of browser tests with Hudson, in addition to quality and cost. The following are the benefits of Hudson.

Keeping a Living Document and Building up Domain Knowledge

Whenever performing a test in Hudson, a document with illustration and description can be automatically generated by using Javadoc package.htm in the test code. This document is called a "living document" which is continuously updated and used by operators, developers and QAers for a variety of purposes such as hand-off of domain knowledge and preservation of revision history.

javadoc_document_created_from_tests.png

Figure 2: Javadoc Document Created from Tests.

Visualizing Service Quality by Browser and Domain Knowledge

Anyone who clicks Build Now in Hudson can check the service quality by browser.

verifying_mobile_news_service_in_firefox.png

Figure 3: Verifying Mobile News Service in Firefox.

For example, in some cases, the news service should be provided differently by day due to the service characteristics. It is very difficult to hand off all domain information to all related staffs. For example, the KOSPI/KOSDAQ indexes are not showed on the news on weekends but after the opening of the market on Monday morning. Anyone who clicks the Build Now button can see the news service operation process by status.

Tests with Hudson help users to hand off domain knowledge and build up the information as well as enhance quality and cut costs.

Best Practices and Examples

We once needed to update the version of Spring Framework used for the mobile news service. We modified the version information in the pom.xml file of Maven and performed the unit test and the integration test for web server source, and the result was no problem. Developers opened some new web pages by running the web server and checked that all of them were fine.

However, there were more than 80 runtime errors. Fortunately, we had the multi-browser test in Hudson test, so we could fix all bugs and problems before releasing the service.

Selenium WebDriver

Why Selenium WebDriver?

We use Selenium WebDriver because of its reliable quality through a version up process for a long time, and it is easy-to-use. The test is performed by running a browser, so it is easy to perform UI test by browser.

The virtual browser test such as HtmlUnit cannot support a variety of browser versions (it is limited to Firefox 3.6 and lower and Internet Explorer 8 and lower). In addition, it does not support mobile browsers. Therefore, I recommend Selenium WebDriver over HtmlUnit, even though Selenium WebDriver requires more effort.

With Java API provided by Selenium WebDriver at the JUnit test code, you can easily produce the browser test code. Code 1 below is now used for the mobile news service test.

Code 1. News Service Test Code.

@Test(timeout = 1000 * 60)
public void 뉴스홈에_최근3일_최종편집시간이_표시된다() {
    Driver.get(driver, "http://m.news.naver.com/home.nhn");
    String expectedResult = "최종편집";
    String result = Driver.getTextByClass(driver, "last_update");
    assertThat(result.startsWith(expectedResult), is(true));
}

Benefits of Selenium WebDriver

Benefits of Selenium WebDriver are various: it allows developers to produce the test code with a variety of languages (Java, C#, Python, PHP, Ruby, Perl) and provides fast feedback by using the implicit wait function. The following are the benefits of Selenium WebDriver.

Test by Specifying ID/Class/XPath

If considering HTML maintenance, ID is the best choice to specify HTML/CSS elements, the second best is Class, and finally, XPath. Selenium WebDriver supports all three.

Multi-browser Tests with One Set of Test Code

You can test several browsers with one set of test code because you can change the test browser by replacing WebDriver only. As of 2012, WebDriver supports Firefox, Internet Explorer, Chrome, Opera, iPhone Safari, iPad Safari and Android browsers. Code 2 below is a part of code that creates WebDriver.

Code 2. Example of Creating WebDriver.

public WebDriverFactory() throws Exception {
    String browsetype = TestConfigParam.getBrowseType();

    if ("firefox".equals(browsetype)) {
        this.driver = new FirefoxDriver();
    } else if ("chrome".equals(browsetype)) {
        this.driver = new ChromeDriver();
    } else if ("ie".equals(browsetype)) {
        this.driver = new InternetExplorerDriver();
    } else if ("iphone".equals(browsetype)) {
        this.driver = new IPhoneDriver();
    } else if ("ie".equals(browsetype)) {
        this.driver = new InternetExplorerDriver();
    } else if ("android".equals(browsetype)) {
        this.driver = new AndroidDriver();
    } else if ("htmlunit".equals(browsetype)) {
        this.driver = new HtmlUnitDriver(false);
    } else {
        this.driver = new FirefoxDriver();
        driver.manage().timeouts().implicitlyWait(Driver.TIMEOUT, TimeUnit.SECONDS);
    }
}

Conclusion

So far, we have reviewed test automation applied to NAVER services. NHN has already automated tests for many services by using Hudson and Selenium WebDriver. I guess other companies have automated tests in the similar ways. Therefore, I hope this article to be a chance to communicate with others to create a Best Practice, rather than showing it for test methods.

By Hyehwan Ahn, Software Engineer at News Service Development Team, NHN Corporation.

I am a developer who is interested in automation of routine tasks in releasing Alpha/Beta/QA/Distribution phases to improve efficiency.



comments powered by Disqus