Software quality has always been important but seems that lately it has become more generally acknowledged fact that quality assurance and testing aren’t things to be left behind. With Java EE Web applications you have different ways to achieve test coverage and test that your application works with tools like JUnit, Mockito and DBUnit. But what about testing your web application with different browsers? One great way is to use Robot Framework which is a generic test automation framework and when combined with Selenium 2 it makes both writing your tests and running them quite intuitive.
Contents
- Introduction
- Installing test tools
- Executing Robot Framework tests
- Using different browsers
- Reading the results
- Summary
Introduction
Robot Framework which is a generic test automation framework for acceptance testing and its tabular test data syntax is almost plain English and easy to understand. Its testing capabilities can be extended by test libraries implemented either with Python or Java, and users can create new higher-level keywords from existing ones using the same syntax that is used for creating test cases. Robot Framework itself is open source and released under Apache License 2.0, and most of the libraries and tools in the ecosystem are also open source. The development of the core framework is supported by Nokia Siemens Networks.
Robot Framework doesn’t do any specific testing activity but instead it acts as a front end for libraries like Selenium2Library. Selenium2Library is a web testing library for Robot Framework that leverages the Selenium 2 (WebDriver) libraries from the Selenium project. In practice it starts the browser (eg. IE, Firefox, Chrome) and runs the tests against it natively as a user would. There’s no need to manually click through the user interface.
Robot Framework has good documentation and by going through the “Web testing with Robot Framework and Selenium2Library” demo you see how it’s used in web testing, get introduction to test data syntax, how tests are executed, and how logs and reports look like. For more detailed view about Robot Framework’s features you can read the User Guide.
Installing test tools
The “Web testing with Robot Framework and Selenium2Library” demo is good starting point for getting to know Robot Framework but it more or less skips the details of setting up the system and as the installation instructions are a bit too verbose here is an example how to install and use Robot Framework and Selenium 2 in 64-bit Windows 7.
Python installation
First we need Python as a precondition to run Robot Framework and we install Python version 2.7.x as Robot Framework is currently not compatible with Python 3.x. From the Python download page select Python 2.7.9 Windows X86-64 Installer.
For using the RIDE editor we also need wxPython. From the download page select wxPython2.8-win64-unicode-py27 for 64-bit Python 2.7.
Next we need to set up the PATH environment variable in Windows if you didn’t setup it when you installed Python.
Open Start > Settings > Control Panel > System > Advanced > Environment Variables
Select System variables > PATH > Edit and add e.g. ;\Python27;C:\Python27\Scripts at the end of the value.
Exit the dialog with OK to save the changes.
Starting from Python 2.7.9, the standard Windows installer by default installs and activates pip.
Robot Framework and Selenium2Library installation
In practice it is easiest to install Robot Framework and Selenium2Library along with its dependencies using pip package manager. Once you have pip installed, all you need to do is running these commands in your Command Prompt:
1. pip install robotframework 2. pip install robotframework-selenium2library
It’s good to notice that pip has a “feature” that unless a specific version is given, they install the latest possible version even if that is an alpha or beta release. A workaround is giving the version explicitly. like pip install robotframework==2.7.7
RIDE installation
RIDE is a light-weight and intuitive editor for Robot Framework test case files. It can be installed by using Windows installer (select robotframework-ride-1.1.win-amd64.exe) or with pip using:
pip install robotframework-ride
The Windows installer does a shortcut to the desktop and you can start it from Command Prompt with command ride.py.
Now you have everything you need to create and execute Robot Framework tests.
Executing Robot Framework tests
As described in WebDemo running tests requires the demo application located under demoapp directory to be running. It can be started by executing it from the command line:
python demoapp/server.py
After the demo application is started, it is be available at http://localhost:7272 and it needs to be running while executing the automated tests. It can be shut down by using Ctrl-C.
In Robot Framework each file contains one or more tests and is treated as a test suite. Every directory that contains a test suite file or directory is also a test suite. When Robot Framework is executed on a directory it will go through all files and directories of the correct kind except those that start with an underscore character.
WebDemo’s test cases are located in login_tests directory and to execute them all type in your Command Prompt:
pybot login_tests
Running the tests opens a browser window which Selenium 2 is driving natively as a user would and you can see the interactions.
When the test is finished executing four files will have been generated: report.html, log.html and output.xml. On failed tests selenium takes screenshots which are named like selenium-screenshot-1.png. The browser can also be run on a remote machine using the Selenium Server.
You can also run an individual test case file and use various command line options (see pybot –help) supported by Robot Framework:
pybot login_tests/valid_login.txt pybot --test InvalidUserName --loglevel DEBUG login_tests
If you selected Firefox as your browser and get an error like “Type Error: environment can only contain strings” that’s a bug in Selenium’s Firefox profile. You can fix it with a “monkey patch” to C:\Python27\Lib\site-packages\selenium\webdriver\firefox\firefox_profile.py.
Using different browsers
The browser that is used is controlled by ${BROWSER} variable defined in resource.txt resource file. Firefox browser is used by default, but that can be easily overridden from the command line.
pybot --variable BROWSER:Chrome login_tests pybot --variable BROWSER:IE login_tests
Browsers like Chrome and Internet Explorer require separate Internet Explorer Driver and Chrome Driver to be installed before they can be used. InternetExplorerDriver can be downloaded from Selenium project and ChromeDriver from Chromium project. Just place them both somewhere in your PATH.
With Internet Explorer Driver you can get an error like “‘Unexpected error launching Internet Explorer. Protected Mode settings are not the same for all zones. Enable Protected Mode must be set to the same value (enabled or disabled) for all zones.’”. As it reads in the driver’s configuration you must set the Protected Mode settings for each zone to be the same value. To set the Protected Mode settings in Internet Explorer, choose “Internet Options…” from the Tools menu, and click on the Security tab. For each zone, there will be a check box at the bottom of the tab labeled “Enable Protected Mode”.
Reading the results
After the tests have run there are couple of result files to read: report.html and log.html.
The report.html shows the results of your tests and its background is green when all tests have passed and red if any have failed. It also shows “Test Statistics” for how many tests have passed and failed. “Test Details” shows how long the test took to run and, if it failed, what the fail message was.
The log.html gives you more detailed information about why some test fails if the fail message doesn’t make it obvious. It also gives a detailed view of the execution of each of the tests.
Summary
From the short experience I have played with Robot Framework it seems to be powerful tool for designing and executing tests and good way to improve your application’s overall quality.
Next it’s time to get to know the Robot Framework syntax better, write some tests and run Selenium Server. Also the Maven plugin and RobotFramework-EclipseIDE plugin looks interesting.
References
Robot Framework documentation
Robot Framework User Guide
Web testing with Robot Framework and Selenium2Library demo
RIDE: light-weight and intuitive editor for Robot Framework test case files
Leave a Reply