Tag Archives: howto

Web application test automation with Robot Framework

Posted on by
Reply

Software quality has always been important but seems that lately it has become more generally acknowledged fact that quality assurance and testing aren’t things to be left behind. With Java EE Web applications you have different ways to achieve test coverage and test that your application works with tools like JUnit, Mockito and DBUnit. But what about testing your web application with different browsers? One great way is to use Robot Framework which is a generic test automation framework and when combined with Selenium 2 it makes both writing your tests and running them quite intuitive.

Contents

Introduction

Robot Framework which is a generic test automation framework for acceptance testing and its tabular test data syntax is almost plain English and easy to understand. Its testing capabilities can be extended by test libraries implemented either with Python or Java, and users can create new higher-level keywords from existing ones using the same syntax that is used for creating test cases. Robot Framework itself is open source and released under Apache License 2.0, and most of the libraries and tools in the ecosystem are also open source. The development of the core framework is supported by Nokia Siemens Networks.

Robot Framework doesn’t do any specific testing activity but instead it acts as a front end for libraries like Selenium2Library. Selenium2Library is a web testing library for Robot Framework that leverages the Selenium 2 (WebDriver) libraries from the Selenium project. In practice it starts the browser (eg. IE, Firefox, Chrome) and runs the tests against it natively as a user would. There’s no need to manually click through the user interface.

Robot Framework has good documentation and by going through the “Web testing with Robot Framework and Selenium2Library” demo you see how it’s used in web testing, get introduction to test data syntax, how tests are executed, and how logs and reports look like. For more detailed view about Robot Framework’s features you can read the User Guide.

Installing test tools

The “Web testing with Robot Framework and Selenium2Library” demo is good starting point for getting to know Robot Framework but it more or less skips the details of setting up the system and as the installation instructions are a bit too verbose here is an example how to install and use Robot Framework and Selenium 2 in 64-bit Windows 7.

Python installation

First we need Python as a precondition to run Robot Framework and we install Python version 2.7.x as Robot Framework is currently not compatible with Python 3.x. From the Python download page select Python 2.7.4 Windows X86-64 Installer.

For using the RIDE editor we also need wxPython. From the download page select wxPython2.8-win64-unicode-py27 for 64-bit Python 2.7.

Next we need to set up the PATH environment variable in Windows.

Open Start > Settings > Control Panel > System > Advanced > Environment Variables
Select System variables > PATH > Edit and add e.g. ;\Python27;C:\Python27\Scripts at the end of the value.
Exit the dialog with OK to save the changes.

We also need some Python package managers like distribute and pip which have Windows installers by Christoph Gohlke.

Distribute:

Distribute is a tool to easily download, build, install, upgrade, and uninstall Python packages. The installation is easiest by downloading it as binary. Just run the installer and point it to your Python installation directory.

Pip:

pip is a tool for installing and managing Python packages. The installation is easiest by downloading it as a binary. Just run the installer and point it to your Python installation directory.

Robot Framework and Selenium2Library installation

In practice it is easiest to install Robot Framework and Selenium2Library along with its dependencies using pip package manager. Once you have pip installed, all you need to do is running these commands in your Command Prompt:

1. pip install robotframework
2. pip install robotframework-selenium2library

It’s good to notice that pip has a “feature” that unless a specific version is given, they install the latest possible version even if that is an alpha or beta release. A workaround is giving the version explicitly. like pip install robotframework==2.7.7

RIDE installation

RIDE is a light-weight and intuitive editor for Robot Framework test case files. It can be installed by using Windows installer (select robotframework-ride-1.1.win-amd64.exe) or with pip using:

pip install robotframework-ride

The Windows installer does a shortcut to the desktop and you can start it from Command Prompt with command ride.py.

Now you have everything you need to create and execute Robot Framework tests.

Executing Robot Framework tests

As described in WebDemo running tests requires the demo application located under demoapp directory to be running. It can be started by executing it from the command line:

python demoapp/server.py

After the demo application is started, it is be available at http://localhost:7272 and it needs to be running while executing the automated tests. It can be shut down by using Ctrl-C.

In Robot Framework each file contains one or more tests and is treated as a test suite. Every directory that contains a test suite file or directory is also a test suite. When Robot Framework is executed on a directory it will go through all files and directories of the correct kind except those that start with an underscore character.

WebDemo’s test cases are located in login_tests directory and to execute them all type in your Command Prompt:

pybot login_tests

Running the tests opens a browser window which Selenium 2 is driving natively as a user would and you can see the interactions.
When the test is finished executing four files will have been generated: report.html, log.html and output.xml. On failed tests selenium takes screenshots which are named like selenium-screenshot-1.png. The browser can also be run on a remote machine using the Selenium Server.

You can also run an individual test case file and use various command line options (see pybot –help) supported by Robot Framework:

pybot login_tests/valid_login.txt
pybot --test InvalidUserName --loglevel DEBUG login_tests

If you selected Firefox as your browser and get an error like “Type Error: environment can only contain strings” that’s a bug in Selenium’s Firefox profile. You can fix it with a “monkey patch” to C:\Python27\Lib\site-packages\selenium\webdriver\firefox\firefox_profile.py.

Using different browsers

The browser that is used is controlled by ${BROWSER} variable defined in resource.txt resource file. Firefox browser is used by default, but that can be easily overridden from the command line.

pybot --variable BROWSER:Chrome login_tests
pybot --variable BROWSER:IE login_tests

Browsers like Chrome and Internet Explorer require separate Internet Explorer Driver and Chrome Driver to be installed before they can be used. InternetExplorerDriver can be downloaded from Selenium project and ChromeDriver from Chromium project. Just place them both somewhere in your PATH.

With Internet Explorer Driver you can get an error like “‘Unexpected error launching Internet Explorer. Protected Mode settings are not the same for all zones. Enable Protected Mode must be set to the same value (enabled or disabled) for all zones.’”. As it reads in the driver’s configuration you must set the Protected Mode settings for each zone to be the same value. To set the Protected Mode settings in Internet Explorer, choose “Internet Options…” from the Tools menu, and click on the Security tab. For each zone, there will be a check box at the bottom of the tab labeled “Enable Protected Mode”.

Reading the results

After the tests have run there are couple of result files to read: report.html and log.html.

The report.html shows the results of your tests and its background is green when all tests have passed and red if any have failed. It also shows “Test Statistics” for how many tests have passed and failed. “Test Details” shows how long the test took to run and, if it failed, what the fail message was.

The log.html gives you more detailed information about why some test fails if the fail message doesn’t make it obvious. It also gives a detailed view of the execution of each of the tests.

Summary

From the short experience I have played with Robot Framework it seems to be powerful tool for designing and executing tests and good way to improve your application’s overall quality.

Next it’s time to get to know the Robot Framework syntax better, write some tests and run Selenium Server. Also the Maven plugin and RobotFramework-EclipseIDE plugin looks interesting.

References

Robot Framework documentation
Robot Framework User Guide
Web testing with Robot Framework and Selenium2Library demo
RIDE: light-weight and intuitive editor for Robot Framework test case files

Running FishEye & Crucible as a service in Linux

Posted on by
Reply

Atlassian’s tools for supporting software development are great but they aren’t really admin friendly to start with. For example FishEye & Crucible doesn’t ship with scripts to start it at system boot time but with the help of Atlassian’s Wiki, sysadmin tasks and scripts you can run it as a normal service. First we create a dedicated user for crucible and second we add a new service for it. I have done this on CentOS 5.7 x86_64.

Setting up the service account

As the root user, create a separate “FishEye & Crucible” service account at root shell:

# useradd -c "FishEye & Crucible service account" -d /home/crucible -m crucible

To make it easier for this to work also after FishEye & Crucible upgrades we create a symbolic link to the latest version (modify “/opt/fecru” to match your deployment).

# ln -s /opt/fecru/fecru-2.7.15 /opt/fecru/latest

Then, ensure that this user is the filesystem owner of the FishEye & Crucible instance (modify “/opt/fecru” to match your deployment).

# chown -R crucible:crucible /opt/fecru

Running Crucible as a crucible user

Save the following script to /etc/init.d/crucible. Be sure to edit the FISHEYE_HOME value to the location where your FishEye/Crucible instance resides:

#!/bin/bash
# RUN_AS: The user to run fisheye & crucible as. Its recommended that you create a separate user account for security reasons
RUN_AS=crucible
 
# FISHEYE_HOME: The path to the FishEye & Crucible installation. Its recommended to create a symbolic link to the latest version so the process will still work after upgrades.
FISHEYE_HOME="/opt/fecru/latest"
# FISHEYE_INST: The path where the data itself will be stored.
export FISHEYE_INST="/opt/fecru/fecru-data"

fisheyectl() {
        if [ "x$USER" != "x$RUN_AS" ]; then
                # If running without FISHEYE_INST
                # su - "$RUN_AS" -c "$FISHEYE_HOME/bin/fisheyectl.sh $1"
                su - "$RUN_AS" -c "FISHEYE_INST=$FISHEYE_INST $FISHEYE_HOME/bin/fisheyectl.sh $1"
        else
                "$FISHEYE_HOME/bin/fisheyectl.sh $1"
        fi
} 

case "$1" in
        start)
                fisheyectl start
                ;;
        stop)
                fisheyectl stop
                ;;
        restart)
                fisheyectl stop
                sleep 10
                fisheyectl start
                ;;
        *)
                echo "Usage: $0 {start|stop|restart}"
esac
 
exit 0

After saving the script, modify it’s permissions so that it can be executed:

# chmod 755 /etc/init.d/crucible

Running Crucible as a service

Now that we have an init script we can add it as a service and be able to configure the system to run the script on startup (more precisely, ensure that Crucible runs in runlevels 3, 4 and 5):

chkconfig --add crucible
chkconfig crucible on

Verify that the script has been installed correctly:

# chkconfig --list crucible

After this has been done you can manually start or stop the service by using these commands:

service crucible stop
service crucible start

And you’re done.

JSF 1.2 and getting selected value from dropdown

Posted on by
Reply

JSF 1.2 has some weird features which you just have to know if you haven’t read the documents. One example is getting a value from h:selectOneMenu dropdown after onchange event. The first what comes to mind is to use binding attribute with RichFaces’ a4j:support for rerendering elements after the event but it doesn’t work like you thought it would. In some cases using the binding attribute works just fine but as the binding attribute should refer to a request scoped bean property, not a session scoped one, you might get “Duplicate id error” when switching pages back and forth.

Fortunately there is valueChangeListener in h:SelectOneMenu which you can trick to do almost the same. It is executed during Validations phase, before the “Update Model Values” phase and is intended to get a handle of both the old and new value so that you can do some business stuff based on the real change. However, you can use it to invoke actions on a dropdown change only by combining it with onchange="submit()" and immediate="true" and the selected value is to be obtained by ValueChangeEvent#getNewValue(). (StackOverflow, BalusC)

For example:

Jspx:
<h:selectOneMenu value="#{fooBean.object.value}" 
	valueChangeListener="#{fooBean.statusChanged}" 
	onchange="submit()" immediate="true">
	<f:selectItem itemLabel="" itemValue=""/>
	<f:selectItems value="#{fooBean.selectValuesList}"/>
</h:selectOneMenu>

Java:
public void statusChanged(ValueChangeEvent event) {
	if (event.getNewValue() != null && 
		StringUtils.hasText((String) event.getNewValue())) {
		// ... Do something with the new value
	}
}

The negative side of using onchange="submit()" is that the form is submitted, validated and you don’t get the same dynamic feeling like with a4j:support.

In JSF 2 things are easier as you don’t need the valueChangeListener and you can use the listener attribute of <f:ajax> instead.

Exclude JQuery libraries from Eclipse’s JavaScript Validation

Posted on by
Reply

Eclipse likes to validate JavaScript when doing Dynamic Web Modules and thus may give you false positive validation errors on 3rd party JavaScript libraries like JQuery. Although you can turn off the validation altogether but better solution is to configure it to exclude files as Alexander shows us at Stackoverflow.

Eclipse Indigo (3.7) has the option to selectively remove some JavaScript sources from validation. The information about JavaScript source inclusion/exclusion is saved into .settings/.jsdtscope file.

  1. Right click your project
  2. Select Properties → JavaScript → Include Path
  3. Select Source tab
  4. Expand JavaScript source folder
  5. Highlight Excluded pattern
  6. Click Edit button
  7. Click Add button next to Exclusion patterns box
  8. You may use wildcard pattern, or click Browse button to add the source by name.
    • Exclude all JQuery files with pattern like: js/jquery-*

The configuration with JQuery files excluded from validation looks like this:

eclipse_exclude-javascript

Using CAcert.org signed certificates for TLS

Posted on by
Reply

Setting up Transport Layer Security (TLS), or as previously known as Secure Sockets Layer (SSL), for Apache, Postfix and IMAP like Dovecot is fairly easy. You just need some digital certificates and configuration. If you don’t want to pay for certificates from trusted sources like Thawte or you just don’t need that kind of trust (for development purposes), you can always produce your own certificates. But there is also a middle way: using CAcert.org signed certificates.

Background
Wikipedia tells us that CAcert.org is a community-driven certificate authority that issues free public key certificates. CAcert automatically signs certificates for email addresses controlled by the requester and for domains for which certain addresses (such as “hostmaster@example.com”) are controlled by the requester. Thus it operates as a robot certificate authority. CAcert certificates can be used like any other SSL certificates although they are considered weak because CAcert does not emit any information in the certificates other than the domain name or email address. To create higher-trust certificates, users can participate in a web of trust system whereby users physically meet and verify each other’s identities. They are also not as useful in web browsers as certificates issued by commercial CAs such as VeriSign, because most installed web browsers do not distribute CAcert’s root certificate. Thus, for most web users, a certificate signed by CAcert behaves like a self-signed certificate.

Generating Certificates
The procedure to sign your certificate at CAcert is rather simple. This guide assumes that the certificates are in /etc/ssl/cacert/ and you are as root.

0. Join CAcert.org and fill in your details. After email verification and login, add domain and service will try to verify that you can read mail on one of following accounts: root, hostmaster, postmaster, admin, webmaster or email addresses that can be found on whois data of domain that you provided.

1. Generate a private key that is not file encrypted:

openssl genrsa -out domainname.key 1024
chown root:root domainname.key
chmod 0400 domainname.key

Private keys should belong to “root” and be readable only by root.

You could also create a private key that is encrypted: openssl genrsa -des3 -out domainname.key 1024

2. Create a CSR with the RSA private key (output will be PEM format). Do not enter extra attributes at the prompt and leave the challenge password blank (press enter): 

openssl req -new -key domainname.key -out domainname.csr

3. Verify the contents of the CSR or private key:

openssl req -noout -text -in domainname.csr
openssl rsa -noout -text -in domainname.key

4. Send your public key to be signed by and request new server certificate from CAcert.org web site (Class 1 certificate). When you are asked for CSR paste content of domainname.csr. It should look like this:

-----BEGIN CERTIFICATE REQUEST-----
MIIB3TCCAUYCAQAwgZwxCzAJBgNVBAYTAkZJMRAwDgYDVQQIEwdVdXNpbWFhMQ8w
...clip...
MQ==
-----END CERTIFICATE REQUEST-----

You can verify the content of request before sending it

openssl req -in domainname.csr -text -verify -noout

5. Copy the Server Certificate from the CAcert.org webpage and put it in domainname.crt file and add permissions.

chmod a=r domainname.crt

Check at least the contents of Validity and Subject fields:

openssl x509 -in domainname.crt -text -noout

6. Get CAcert.org root certificate

wget -nv https://www.cacert.org/certs/root.crt -O cacert-org.crt
chmod a=r cacert-org.crt

Check the contents:

openssl x509 -in cacert-org.crt -text -noout

After that you’re ready to configure your services like Apache, Postfix and Dovecot to use the new certificate. Read about it later.