Web application test automation with Robot Framework

Posted on May 15, 2013 by Marko

Software quality has always been important but seems that lately it has become more generally acknowledged fact that quality assurance and testing aren’t things to be left behind. With Java EE Web applications you have different ways to achieve test coverage and test that your application works with tools like JUnit, Mockito and DBUnit. But what about testing your web application with different browsers? One great way is to use Robot Framework which is a generic test automation framework and when combined with Selenium 2 it makes both writing your tests and running them quite intuitive.

Contents

Introduction
Installing test tools
Executing Robot Framework tests
Using different browsers
Reading the results
Summary
- References

Introduction

Robot Framework which is a generic test automation framework for acceptance testing and its tabular test data syntax is almost plain English and easy to understand. Its testing capabilities can be extended by test libraries implemented either with Python or Java, and users can create new higher-level keywords from existing ones using the same syntax that is used for creating test cases. Robot Framework itself is open source and released under Apache License 2.0, and most of the libraries and tools in the ecosystem are also open source. The development of the core framework is supported by Nokia Siemens Networks.

Robot Framework doesn’t do any specific testing activity but instead it acts as a front end for libraries like Selenium2Library. Selenium2Library is a web testing library for Robot Framework that leverages the Selenium 2 (WebDriver) libraries from the Selenium project. In practice it starts the browser (eg. IE, Firefox, Chrome) and runs the tests against it natively as a user would. There’s no need to manually click through the user interface.

Robot Framework has good documentation and by going through the “Web testing with Robot Framework and Selenium2Library” demo you see how it’s used in web testing, get introduction to test data syntax, how tests are executed, and how logs and reports look like. For more detailed view about Robot Framework’s features you can read the User Guide.

Installing test tools

The “Web testing with Robot Framework and Selenium2Library” demo is good starting point for getting to know Robot Framework but it more or less skips the details of setting up the system and as the installation instructions are a bit too verbose here is an example how to install and use Robot Framework and Selenium 2 in 64-bit Windows 7.

Python installation

First we need Python as a precondition to run Robot Framework and we install Python version 2.7.x as Robot Framework is currently not compatible with Python 3.x. From the Python download page select Python 2.7.4 Windows X86-64 Installer.

For using the RIDE editor we also need wxPython. From the download page select wxPython2.8-win64-unicode-py27 for 64-bit Python 2.7.

Next we need to set up the PATH environment variable in Windows.

Open Start > Settings > Control Panel > System > Advanced > Environment Variables
Select System variables > PATH > Edit and add e.g. ;\Python27;C:\Python27\Scripts at the end of the value.
Exit the dialog with OK to save the changes.

We also need some Python package managers like distribute and pip which have Windows installers by Christoph Gohlke.

Distribute:

Distribute is a tool to easily download, build, install, upgrade, and uninstall Python packages. The installation is easiest by downloading it as binary. Just run the installer and point it to your Python installation directory.

Pip:

pip is a tool for installing and managing Python packages. The installation is easiest by downloading it as a binary. Just run the installer and point it to your Python installation directory.

Robot Framework and Selenium2Library installation

In practice it is easiest to install Robot Framework and Selenium2Library along with its dependencies using pip package manager. Once you have pip installed, all you need to do is running these commands in your Command Prompt:

1. pip install robotframework
2. pip install robotframework-selenium2library

It’s good to notice that pip has a “feature” that unless a specific version is given, they install the latest possible version even if that is an alpha or beta release. A workaround is giving the version explicitly. like pip install robotframework==2.7.7

RIDE installation

RIDE is a light-weight and intuitive editor for Robot Framework test case files. It can be installed by using Windows installer (select robotframework-ride-1.1.win-amd64.exe) or with pip using:

pip install robotframework-ride

The Windows installer does a shortcut to the desktop and you can start it from Command Prompt with command ride.py.

Now you have everything you need to create and execute Robot Framework tests.

Executing Robot Framework tests

As described in WebDemo running tests requires the demo application located under demoapp directory to be running. It can be started by executing it from the command line:

python demoapp/server.py

After the demo application is started, it is be available at http://localhost:7272 and it needs to be running while executing the automated tests. It can be shut down by using Ctrl-C.

In Robot Framework each file contains one or more tests and is treated as a test suite. Every directory that contains a test suite file or directory is also a test suite. When Robot Framework is executed on a directory it will go through all files and directories of the correct kind except those that start with an underscore character.

WebDemo’s test cases are located in login_tests directory and to execute them all type in your Command Prompt:

pybot login_tests

Running the tests opens a browser window which Selenium 2 is driving natively as a user would and you can see the interactions.
When the test is finished executing four files will have been generated: report.html, log.html and output.xml. On failed tests selenium takes screenshots which are named like selenium-screenshot-1.png. The browser can also be run on a remote machine using the Selenium Server.

You can also run an individual test case file and use various command line options (see pybot –help) supported by Robot Framework:

pybot login_tests/valid_login.txt
pybot --test InvalidUserName --loglevel DEBUG login_tests

If you selected Firefox as your browser and get an error like “Type Error: environment can only contain strings” that’s a bug in Selenium’s Firefox profile. You can fix it with a “monkey patch” to C:\Python27\Lib\site-packages\selenium\webdriver\firefox\firefox_profile.py.

Using different browsers

The browser that is used is controlled by ${BROWSER} variable defined in resource.txt resource file. Firefox browser is used by default, but that can be easily overridden from the command line.

pybot --variable BROWSER:Chrome login_tests
pybot --variable BROWSER:IE login_tests

Browsers like Chrome and Internet Explorer require separate Internet Explorer Driver and Chrome Driver to be installed before they can be used. InternetExplorerDriver can be downloaded from Selenium project and ChromeDriver from Chromium project. Just place them both somewhere in your PATH.

With Internet Explorer Driver you can get an error like “‘Unexpected error launching Internet Explorer. Protected Mode settings are not the same for all zones. Enable Protected Mode must be set to the same value (enabled or disabled) for all zones.’”. As it reads in the driver’s configuration you must set the Protected Mode settings for each zone to be the same value. To set the Protected Mode settings in Internet Explorer, choose “Internet Options…” from the Tools menu, and click on the Security tab. For each zone, there will be a check box at the bottom of the tab labeled “Enable Protected Mode”.

Reading the results

After the tests have run there are couple of result files to read: report.html and log.html.

The report.html shows the results of your tests and its background is green when all tests have passed and red if any have failed. It also shows “Test Statistics” for how many tests have passed and failed. “Test Details” shows how long the test took to run and, if it failed, what the fail message was.

The log.html gives you more detailed information about why some test fails if the fail message doesn’t make it obvious. It also gives a detailed view of the execution of each of the tests.

Summary

From the short experience I have played with Robot Framework it seems to be powerful tool for designing and executing tests and good way to improve your application’s overall quality.

Next it’s time to get to know the Robot Framework syntax better, write some tests and run Selenium Server. Also the Maven plugin and RobotFramework-EclipseIDE plugin looks interesting.

References

Robot Framework documentation
Robot Framework User Guide
Web testing with Robot Framework and Selenium2Library demo
RIDE: light-weight and intuitive editor for Robot Framework test case files

Patching RichFaces 3.3.3 AJAX.js for IE9

Posted on April 24, 2013 by Marko

There are always some problems when working with 3rd party frameworks when the world moves forward but the framework you’re using doesn’t. After JSF 2 was released the RichFaces development moved to 4.x version and they dropped support for the older versions although many users are still using the older versions as it’s not trivial or free to update to JSF 2 & RichFaces 4. Now if users have problems with the older versions they just have to patch it themselves as the RichFaces team isn’t going to fix e.g. the IE9 issues.

The problem with Internet Explorer 9 is that with RichFaces 3.3.3.Final the Ajax components doesn’t work and there are errors shown in the JavaScript Console. For example when testing RichFaces Ajax demos the JavaScript is assigning A.outerHTML = new XMLSerializer().serializeToString(C) which gives the following error:

SCRIPT16386: No such interface supported

Fortunately it’s relatively easy to fix it for IE9 by patching the AJAX.js JavaScript file in RichFaces. The process is discussed in RichFaces’s RF-9485 -JIRA ticket and JBoss forum message. In short the patching for AJAX.js contains following steps:

Upgrading Sarissa Framework and patching RichFaces AJAX.js file

Resources:

Sarissa framework: http://sourceforge.net/projects/sarissa/files/
RichFaces sources: http://anonsvn.jboss.org/repos/richfaces/branches/community/3.3.X/

1) Instead of patching Sarissa as suggested in the ticket it’s better to upgrade Sarissa to 0.9.9.6 as it has some additional IE9 issues fixed.

Download Sarissa:
http://sourceforge.net/projects/sarissa/files/sarissa/Sarissa%200.9.9.6/sarissa-0.9.9.6.jar/download

Extract sarissa.js from the jar:
sarissa-0.9.9.6.jar/gr/abiss/js/sarissa/sarissa.js

Replace first part of AJAX.js (till EOF comment) or replace sarissa.js in richfaces souce resources.
richfaces-impl-3.3.3.Final.jar/org/ajax4jsf/javascript/scripts/AJAX.js or richfaces-ui-3.3.3.Final\framework\impl\src\main\javascript\ajaxjsf\

2) For richfaces-impl work in non-compatibility mode in IE9, A4J.AJAX.replacePage method should be fixed in RichFaces sources in AJAX.js or JSFAJAX.js because custom document.open method does not work in IE9 document mode.

Replace (line 2125 in AJAX.js, line 1121 in JFSAJAX.js)
	if(isIE) {  
With
	if(isIE&&!Sarissa._SARISSA_IS_IE9) {

3) You will also need to change or comment the next line in AJAX.js or JSFAJAX.js (richfaces-ui-3.3.3.Final\framework\impl\src\main\javascript\ajaxjsf\)

Replace (line 2119 in AJAX.js, line 1044 in JFSAJAX.js)
	LOG.debug("Hidden JSF state fields: "+idsSpan);
to
	LOG.debug("Hidden JSF state fields: ");//+idsSpan);

Otherwise you get the following error in IE9:

	SCRIPT438: Object doesn't support this property or method
	For this line 2648 in framework pack:
	LOG.debug("Hidden JSF state fields: "+Q);

4) Now you have patched AJAX.js and you can either build a new RichFaces Jar, replace the original AJAX.js in RichFaces Jar with the patched one or just override it in your html.

The easiest way is just include the patched AJAX.js file as the last script-include within HEAD and it will take control over the bundled version of RichFaces.

In your template.jspx or similar:

<html>
	<head>
	<a4j:loadScript src="/js/AJAX_IE9fix.js" />
	...
	</head>
	...

5) Done.

Other ways to fix IE9 compatibility issues has been discussed in tickets RF-10774 as forcing IE9 to use another document mode. Also suggested solutions are to implement a browser sniffer and tell the Sarissa library to use its own XMLSerializer when Internet Explorer 9 is detected and forcing IE9 to run in an emulated mode but neither of those worked for me.

Using the WebLogic Maven Plug-In for Deployment

Posted on April 19, 2013 by Marko

Using the WebLogic Maven plug-In for deployment is much easier and quicker than going through the WebLogic Server’s AdminServer and Oracle Documentation provides good examples how to do it.

In short, generating WebLogic Maven Plug-In contains following steps:

1. Build the plug-in JAR file using the WebLogic JarBuilder Tool (wljarbuilder) under MW_HOME/wlserver_10.3/server/lib/ with the following command:

java -jar wljarbuilder.jar -profile weblogic-maven-plugin

2. Extract the pom.xml file from weblogic-maven-plugin.jar under the MW_HOME/wlserver_10.3/server/lib directory, and then copy the pom.xml file to MW_HOME/wlserver_10.3/server/lib.

	
jar xvf MW_HOME/wlserver_10.3/server/lib/weblogic-maven-plugin.jar META-INF/maven/com.oracle.weblogic/weblogic-maven-plugin/pom.xml

in Linux:	
cp META-INF/maven/com.oracle.weblogic/weblogic-maven-plugin/pom.xml MW_HOME/wlserver_10.3/server/lib

or in Windows:
copy META-INF/maven/com.oracle.weblogic/weblogic-maven-plugin/pom.xml MW_HOME/wlserver_10.3/server/lib

3. Provision the weblogic-maven-plugin.jar in your local Maven repository with the following command.

mvn install:install-file -Dfile=MW_HOME/wlserver_10.3/server/lib/weblogic-maven-plugin.jar -DpomFile=pom.xml

4. Done.

The Maven plug-in can be used e.g. from application’s POM file and be bound to some phase of the Maven life cycle. For example it can be bound to “install” phase and every time you run the “mvn install” command, the deployment plug-in is also called. In my opinion better way to use the plug-in is to add it as a profile so you can call it just when you want with command like “mvn clean install -Pdeploy-wls”.

Maven Project pom.xml File

<profile>
	<id>deploy-wls</id>
	<build>
		<plugins> 
			<plugin> 
				<groupId>com.oracle.weblogic</groupId>
				<artifactId>weblogic-maven-plugin</artifactId> 
				<version>10.3.6.0</version> 
				<configuration> 
					<adminurl>t3://localhost:7001</adminurl>
					<user>weblogic</user> 
					<password>weblogic123</password> 
					<upload>true</upload> 
					<targets>myServer</targets>
					<action>deploy</action> 
					<remote>false</remote> 
					<verbose>true</verbose> 
					<source>${project.build.directory}/${project.build.finalName}.${project.packaging}</source> 
					<name>${project.build.finalName}</name> 
				</configuration> 
				<executions> 
					<execution> 
						<phase>install</phase> 
						<goals> 
							<goal>deploy</goal> 
						</goals> 
					</execution> 
				</executions> 
			</plugin> 
		</plugins> 
	</build>
</profile>

The user credentials in the POM file are provided as clear-text but for more security you can use secure configuration authentication mechanism which stores the user name and password in encrypted form in an external file, and then uses it to supply the user credentials with which to connect to the WebLogic Server domain, along with the key that was used to encrypt the file.

Looking back, planning ahead

Posted on December 31, 2012 by Marko

Again a year has passed and it’s time to take a short look back and plan for the next year. A year ago I wrote that in the year 2012 the blog would be more active and I partly managed to keep up that promise with 9 articles. Not quite at least an article per month as I planned.

The articles in 2012 started with technical issues like running Crucible as a service, JSF 1.2 and value from dropdown menu and using CAcert.org signed certificates for TLS. In the Autumn as a Fujitsu LIFEBOOK4Life Insider I reviewed Stylistic M532 Android tablet and wrote about my visit to Fujitsu Forum 2012. There were more technical issues to write about software development but again the time ran out. It’s so much easier to say couple of words in Twitter or Google+ than write a coherent and insightful post about it :)

The year 2013 is almost here so it’s time to plan ahead. There are already some articles about Apache Wicket and running different development tools on Linux in the backlog waiting to be finished and some technical tasks like upgrading the theme and maybe changing the VPS provider are to be done. Will see if I manage to achieve one article per month so subscribe to the RSS feed and stay tuned.

Happy New Year!

Insights to future workplace from Fujitsu Forum 2012

Posted on November 27, 2012 by Marko

This year my Autumn holiday was a bit different as I was one of the four bloggers who were invited by Fujitsu’s LIFEBOOK4Life campaign to visit Fujitsu Forum 2012 in Munich to hear about how the future workplace might look like and to experience new technologies to support that. We also got a tour at Fujitsu’s factory in Augsburg which provided some views how computers are made from mainboards to final product. The three days in Germany were insightful and fun. How can you not like to hear about new ideas, see innovative things, meet new people and of course experience the excellent German food and beer.

Fujitsu Forum is the largest IT-event in Europe which is visited by professional users and IT decision-makers, as well as Fujitsu channel and technology partners. In 2012 more than 12,000 IT experts from around the world attended the Fujitsu Forum in Munich. And I was one of them invited by Fujitsu LIFEBOOK4Life campaign with three other bloggers (called Insiders). The chance I wrote last time came true. Our trip to Fujitsu Forum 2012 was scheduled to contain breakout sessions and keynotes on Wednesday and a tour at the Fujitsu factory in Augsburg on Thursday.

This year the Forum’s motto was “Reshaping ICT – Reshaping Business”: How to combine business processes and IT to form a stable basis that will ensure growth and success in the future. Kind of redefinition of last year’s “Reshaping IT” theme. In short some buzzwords I heard most were: consumerization, BYOD, tablets, virtualization, in-sync, cloud and win 8. “One workplace – on any device”.

First day, Wednesday 7.11

photo by Kim Ekman, Vision 51

The first day at the Forum was full of exciting breakout sessions and keynotes and the exhibition area provided some innovative ideas and services from Fujitsu and technology partners. And of course you got to test new devices like Windows 8 enabled STYLISTIC Q572 -tablet with AMD Hondo platform and STYLISTIC Q702 hybrid tablet with Intel Core i3/i5. There were also some prototypes of future Fujitsu products.

The breakout sessions and keynotes provided some good insights about how the future workplace might look like, which decisions and technologies are needed to support that and what kind of tools are we using in the future. In short the three breakout sessions I listened “Has the Post-Pc Era Begun – What will be on your desktop tomorrow”, “The War Between Enduser Devices” and “Tablets & More – Cool scenarios for hot devices” can be summed to words: consumerization, byod, tablets, virtualization, in-sync, cloud and win 8. “One workplace – on any device”. The materials for all sessions can be found at Fujitsu Forum 2012 documentation page.

The three breakout sessions’ ideas about the future of workplace and tools were similar what I think about the issue: the work and workplace is changing to support mobility and working anywhere anytime. Some of the driving actors for this is that the borders between business and private life are blurring, consumerization (same devices home and work) and BYOD (bring your own device) are coming more common. We should think more about user-oriented workplace. Although the mobile computing is big the traditional desktop computing model isn’t going anywhere as there is always need for more computing power, larger screen, data protection and security.

In the future the devices we use will be more diverse and we have more of them. Thus there will be need for virtualization, syncing data and support for “One workplace – on any device” ideology. Different tasks have different needs for the device so we need a dynamic desktop experience. The sessions also provided some views about the tablets’ platforms and in Western Europe Windows, iOS and Android will be about equals in strength. It will be seen. It was said about Windows 8 that there will be a challenge with transition and touch monitors are needed to deliver significant impact on Win 8.

There were also some suggestions about the devices which can support our future workplace needs. At the desktop we could find the ESPRIMO X -series which has integrated desktop computer at the base of the monitor and the same form factor works in multiple use cases. And with the integrated Web camera with presence sensor you can lock the screen when leaving and login with face detection. Crafty. For the mobility aspects there are STYLISTIC Q572 and Q702 tablets and for laptop needs you have Ultrabooks LIFEBOOK U772 and more traditional LIFEBOOK T902.

After the day at the sessions and exhibition area it was time for lunch. A nice buffet with currywurst, burger and Weissbier. The final event for the day was Celebration Night with some comedians with guitars and then the stage was open for Amy McDonald. It was a nice gig by Amy but last year’s Anastacia was better :)

Second day, Thursday 8.11

The second day for us at the Fujitsu Forum 2012 was dedicated to for a tour at Fujitsu’s factory in Augsburg. It was kind of surprising to see how many workers there are although the automation percent was something like 85-90. Some mainboard components and checking is more easy to do by humans. Also the storage and putting things together were done manually.

After the visit to the factory it was more or less free time to roam the exhibition area because we didn’t have time for anymore sessions or keynotes. At the exhibition we found about Fraunhofer’s nLightened workplace with touch, rotate, scan, change light, adjust hight features and which runs on top of Linux. Again we saw the Made4You customization service which engraves with laser any product you want. Like my Twitter name and Fujitsu Forum 2012 hashtag to a metal cased pen.

The second day was interesting and it looked like there was some party starting when the Insiders and moderators had to head back home. It would have been fun to spend some more days in Munich but as usual the work calls.

Summary

Fujitsu Forum 2012 provided insights and ideas about how the future workplace might look like and we got to experience new technologies to support that. The three days in Munich went (too) fast and it was again great to visit Fujitsu Forum and to see other Insiders and LIFEBOOK4Life crew. Thanks LIFEBOOK4Life and Fujitsu.

If you want to read more check out what Antonia wrote about our trip to campaign site and Aba from FujitsuFans.com has made some previews and reports from the Forum.

Rule of Tech

Technology shall have no dominion