Using Apache for proxying connections to Crucible

Atlassian’s web applications are great tools for software development and they are relatively easy to setup because they come with Jetty servlet container and HQSQL database. You only have to install Java. Some of the applications can be also run like any normal deployable WAR-packaged web application for example with Apache Tomcat which gives you more control and administration options. But unfortunately code review tool Crucible isn’t one of those applications and maybe will never be.

Proxying connections to Crucible

By default Crucible runs in port 8060 which isn’t nicely looking for users. It’s way better to use ports 80 or 443 which are normal HTTP and HTTPS ports and are omitted from browser’s address bar. Of course you can configure that in the Administration screens, or by editing Crucible’s config.xml and restarting Crucible but if you run Crucible as a non root or also have other software running on the same server that isn’t an option.

One solution is to use Apache HTTP server to proxy connections from port 443 to Crucible’s listening port. I did it for Crucible and FishEye on CentOS x86_64 but things are mostly the same also on other Linuxes. I also disabled the HTTP port and used just the SSL enabled HTTPS with self generated certificates.

First we setup Apache for proxying connections to Crucible and then we generate some SSL certificates for HTTPS. If you haven’t Apache installed you can do it with yum like: yum install httpd.x86_64 mod_ssl openssl

1. Set HTTPS proxying in /etc/httpd/conf.d/ssl.conf

...

SSLProxyEngine on
ProxyRequests Off
ProxyPreserveHost On
ProxyPassReverse /crucible ajp://127.0.0.1:8060/crucible
proxyPass /crucible ajp://127.0.0.1:8060/crucible

RewriteEngine On
...

2. Generating SSL Certificate for Apache

# openssl genrsa -out localhost.key 1024
# openssl req -new -key localhost.key -out localhost.csr
# openssl x509 -req -days 365 -in localhost.csr -signkey localhost.key -out localhost.crt
# mv localhost.csr localhost.key /etc/pki/tls/private/
# mv localhost.crt /etc/pki/tls/certs/

3. Start httpd

# service httpd start

Configuring Crucible

4. Configure Crucible (http://hostname:8060/admin)

Edit Web Settings:
-----------------
Web context: crucible
Http Bind: (none)
Ajp13 Bind Address: ajp://127.0.0.1:8060/crucible

And you’re ready.

Using Let’s Encrypt SSL certificates on Centos 6

Let's Encrypt all the things

Let’s Encrypt is now in public beta, meaning, you can get valid, trusted SSL certificates for your domains for free. Free SSL certificates for everyone! As Let’s Encrypt is relatively easy to setup, there’s now no reason not to use HTTPS for your sites. The needed steps are described in the documentation and here’s short guide how to setup Let’s Encrypt in CentOS 6.x and automate the SSL certificate renewal.

Let’s Encrypt installation

The Let’s Encrypt Client is a fully-featured, extensible client for the Let’s Encrypt CA that can automate the tasks of obtaining certificates and configuring web servers to use them. The installation is simple but in my case on CentOS 6.x I first needed to update to Python 2.7 as Let’s Encrypt supports Python 2.7+ only.

Installing Python 2.7 in Centos 6.x

# Install Epel Repository
yum install epel-release
 
# Install IUS Repository
rpm -ivh https://rhel6.iuscommunity.org/ius-release.rpm
 
# Install Python 2.7 and Git
yum --enablerepo=ius install python27 python27-devel python27-pip python27-setuptools python27-virtualenv -y

Setting up Lets encrypt

Install Git if you don’t have it yet.

yum install git

If letsencrypt is packaged for your operating system, you can install it from there, and the other solution is to use the letsencrypt-auto wrapper script, which obtains some dependencies from your operating system and puts others in a python virtual environment:

# Get letsencrypt
git clone https://github.com/letsencrypt/letsencrypt
 
# See help
./letsencrypt/letsencrypt-auto --help

Running the client

You can either just run letsencrypt-auto or letsencrypt, and the client will guide you through the process of obtaining and installing certs interactively or you you can tell it exactly what you want it to do from the command line.

For example obtain a cert for your domain using the Apache plugin to both obtain and install the certs, you could do this:

./letsencrypt-auto --apache -d thing.com -d www.thing.com -d otherthing.net

(The first time you run the command, it will make an account, and ask for an email and agreement to the Let’s Encrypt Subscriber Agreement; you can automate those with –email and –agree-tos)

Although you can use the Apache plugin to obtain and install the certs it didn’t work for me. I got an error: “The apache plugin is not working; there may be problems with your existing configuration.” This seems to be an issue with Apache 2.2 and until it’s fixed you can use the webroot authentication method as explained in documentation.

./letsencrypt-auto certonly --webroot -w /var/www/example/ -d example.com

The webroot plugin works by creating a temporary file for each of your requested domains in ${webroot-path}/.well-known/acme-challenge. Then the Let’s Encrypt validation server makes HTTP requests to validate that the DNS for each requested domain resolves to the server running letsencrypt. Note that to use the webroot plugin, your server must be configured to serve files from hidden directories.

Now your certificate and chain have been saved at Let’s Encrypt configuration directory at “/etc/letsencrypt” and “/etc/letsencrypt/live/ contains symlinks to the latest certificates. Making regular backups of this folder is ideal.

All we have to do now is set it up in Apache.

Configure Apache to use Let’s Encrypt certs

In Let’s Encrypt configuration directory at “/etc/letsencrypt/live/ the .pem files are as follows (from the Letsencrypt documentation):

  • privkey.pem: Private key for the certificate.
    • This must be kept secret at all times! Never share it with anyone, including Let’s Encrypt developers. You cannot put it into a safe, however – your server still needs to access this file in order for SSL/TLS to work.
    • This is what Apache needs for SSLCertificateKeyFile
  • cert.pem: Server certificate only.
    • This is what Apache needs for SSLCertificateFile.
  • chain.pem: All certificates that need to be served by the browser excluding server certificate, i.e. root and intermediate certificates only.
    • This is what Apache needs for SSLCertificateChainFile.
  • fullchain.pem: All certificates, including server certificate. This is concatenation of chain.pem and cert.pem.

Now that we know which file is which we can configure our VirtualHost to use SSL with our new certs. Change the following lines in your Apache’s virtualhost’s SSL configuration:

...
SSLCertificateFile /etc/letsencrypt/live/<your-domain>/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/<your-domain>/privkey.pem
SSLCertificateChainFile /etc/letsencrypt/live/<your-domain>/chain.pem
...

Finally, restart apache

You can test that your SSL is working with SSL Labs.

Automate updating Let’s Encrypt certs

As you surely noticed Let’s Encrypt CA issues short lived certificates (90 days) and you have to renew the certificates at least once in 3 months. Nice way to force sysadmins to automate the process.

To obtain a new version of the certificate you can simply run Let’s Encrypt again but doing that manually is not feasible. Let’s Encrypt is working hard on automating the renewal process but until that we have to do it by ourselves.

Fortunately we don’t need to invent our own scripts as there’s excellent article about automating Let’s Encrypt and script for crontab.

Get the autole.sh -script from GitHub that automates tasks like:

  • Check the expire date of the certificate and renew when the remaining days are below a value
  • Check that the directory for the challenge is well mapped
  • Alert the admin if it’s not possible to renew the certificate

Now you can renew certain domain’s certificates with

./autole.sh www.mydomain.com

And to renew all your certificates use

./autole.sh --renew-all

Now you can add this to the crontab, run weekly, and your certificates will be ready and renew automatically. This cron job will execute the command every Monday at 08:30.

30 8 * * 1 /usr/local/sbin/autole.sh <your-domain> >> /var/log/autole.log

Now before I switch my WordPress over to HTTPS I have to do some find & replace in the database and fix the URL’s of the images to be protocol relative.

Patching RichFaces 3.3.3 AJAX.js for IE11

Couple of years ago I wrote about patching RichFaces 3.3.3 AJAX.js for IE9 and as the browser world has moved on, it’s now time to patch RichFaces 3.3.3 AJAX.js for Internet Explorer 11. Of course you could update your web application to JSF 2 and RichFaces 4 or PrimeFaces but it’s neither trivial nor free. The issue with RichFaces 3.3.3 still stands, development has moved to 4.x version and they’ve dropped support for the older versions although at least IE issues could be easily fixed. Fortunately patching RichFaces AJAX.js is relatively easy.

The problem with Internet Explorer 11 is that although you upgraded Sarissa Framework and patched RichFaces AJAX.js file for IE 9 it just isn’t enough anymore as IE 11 uses different User-agent string and disguises itself as “like Gecko”. The User-agent string in Win 8.1 for IE11 is “Mozilla/5.0 (Windows NT 6.3; Trident/7.0; rv:11.0) like Gecko”. The issue is also discussed on JBoss Forums and on Stack Overflow.

The different User-agent string breakes the “rerender” after Ajax Request. For example using a4j:function in combination with “rerender” is not working on IE 11 and the problem is that after Ajax request the result XML can’t be correctly append to the body. Also the rerendering is abnormal for h:inputTextarea, rich:modalPanel, h:inputTextarea and rich:calendar.

Fortunately the fix is simple as RicFaces issue RF-13443 tells.

Patching RichFaces 3.3.3 AJAX.js for IE 11

Step 1: Do the “Upgrading Sarissa Framework and patching RichFaces AJAX.js file” steps in my Patching RichFaces 3.3.3 AJAX.js for IE9 article.

Step 2: Edit the AJAX_IE9fix.js file you just created and add IE 11 checking to Sarissa._SARISSA_IS_IE and Sarissa._SARISSA_IS_IE9. The changed lines should look like the following:

Sarissa._SARISSA_IS_IE = (document.all && window.ActiveXObject && navigator.userAgent.toLowerCase().indexOf("msie") > -1 && navigator.userAgent.toLowerCase().indexOf("opera") == -1) || (navigator.userAgent.toLowerCase().indexOf("like gecko") > -1 && navigator.userAgent.toLowerCase().indexOf("11.") > -1);
 
Sarissa._SARISSA_IS_IE9 = Sarissa._SARISSA_IS_IE && (parseFloat(navigator.appVersion.substring(navigator.appVersion.indexOf("MSIE")+5))) >= 9 || (navigator.userAgent.toLowerCase().indexOf("like gecko") > -1 && navigator.userAgent.toLowerCase().indexOf("11.") > -1);

If all fails you can try to tell IE 11 to emulate IE 10 with adding X-UA-Compatible meta tag to head.

<meta http-equiv="X-UA-Compatible" content="IE=EmulateIE10"/>

The other thing I noticed with RichFaces 3.3.3 and modern browsers is that if you use ui:fragment which contains rich:suggestionbox and rerender it, the suggestionbox doesn’t work correctly. It gives an error: SCRIPT5007: Unable to get property 'parentNode' of undefined or null reference. For now I didn’t have time to figure out the issue and just changed my page structure and function.

Although it’s 2015 using JSF 1.2 and RichFaces 3.3.3 is still working quite nicely :)

Creating Vagrant Base Box with Veewee

Vagrant is a great tool for creating and configuring lightweight, reproducible, portable virtual machine environments but the first step for using Vagrant, downloading an existing “base box”, raises some questions. E.g. How are these unverified boxes built? So, you might end up building your own base box which is often time consuming and cumbersome. Fortunately there’s a tool called Veewee which aims to automate all the steps for building base boxes and to collect best practices in a transparent way.

Vagrant Base Box with Veewee

Veewee is a tool for easily (and repeatedly) building custom Vagrant base boxes, KVMs, and virtual machine images. You can use it to build a Vagrant box in Linux, Mac OS X and Windows but I found out that fulfilling the requirements on Windows is quite difficult (read Ruby and RVM) so just forget it.

To get you started there are some requirements you need to fulfill. First you’ll need to install at least one of the supported virtual machine providers like VirtualBox and second you need some development libraries.

On Ubuntu 15.04 Linux and using VirtualBox you need these packages:

$ apt-get install virtualbox git curl ruby ruby-dev libxslt1-dev libxml2-dev zlib1g-dev

Install RVM on Linux

For Ruby environment it’s recommended to use either rvm or rbenv. I chose the RVM and followed the RVM install documentation.

Install mpapis public key:

$ gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3

if keyserver fails, you can use $ curl -sSL https://rvm.io/mpapis.asc | gpg --import -

Install RVM stable with ruby:

$ \curl -sSL https://get.rvm.io | bash -s stable --ruby

Installing Veewee with RVM

With RVM already installed, ensure a ruby version that’s supported by Veewee is available on your machine:

$ source /home/marko/.rvm/scripts/rvm
$ rvm install ruby

Clone the veewee project from source:

$ cd <path_to_workspace>
$ git clone https://github.com/jedi4ever/veewee.git
$ cd veewee

Set the local gemset and ruby version within the current directory:

$ rvm use ruby@veewee --create

Run bundle install to install Gemfile dependencies for your local gemset:

$ gem install bundler
$ bundle install

Bundle install will take some time.

Building Vagrant Box with Veewee

Veewee uses definitions to build new virtual machines and ‘definition’ is derived from a ‘template’ and preconfigured templates are found in templates/ folder. Veewee Basics explains how you can create your own customized definition.

For my customized Vagrant Box I decided to use Tommy Muehle’s definition as a template as it contained what I wanted. Simple CentOS 6.6. Box with Puppet. I just changed the localization to Finland and made it bigger for WebLogic use case in mind. My definition for Vagrant Box can be found in GitHub.

To use my definition just clone the repository for CentOS 6.6 Box, copy the “centos-6.6-x86_64_puppet” folder to definitions/ folder under Veewee and make your own changes if needed. After you’re done run:

$ bundle exec veewee vbox build centos-6.6-x86_64_puppet

The build command runs Veewee scripts and automates the manual steps needed while installing a new Linux distribution.

Installing CentOS to Vagrant Box with Veewee

To export the Box for further use with Vagrant, run:

$ bundle exec veewee vbox export centos-6.6-x86_64_puppet

The above command is actually calling “vagrant package –base ‘centos-6.6-x86_64_puppet’ –output ‘boxes/centos-6.6-x86_64_puppet'”. The machine gets shut down, exported and will be packed in a centos-6.6-x86_64_puppet.box file inside the current directory.

And you’re all done. Now you can use your just created base box for Vagrant boxes. Import it into Vagrant’s box repository and use it to initialize a fresh project:

$ vagrant box add 'centos-6.6-x86_64_puppet' 'centos-6.6-x86_64_puppet.box'
$ vagrant init 'centos-6.6-x86_64_puppet'

Using Veewee to build a Vagrant Box is simple and what’s more important it’s automated and reproducible. Using Ruby and RVM on Windows 7 turned out to be practically impossible but old ThinkPad W510 with Ubuntu 15.04 worked nicely. Of course you could create a base box with Vagrant way which means installing and configuring your Linux manually. But why would you want to that if you can just automate it?

Disabling Derby in Oracle WebLogic 12c

Oracle WebLogic has some interesting traits to help developers frustrate. From Weblogic 10.3.4 and above the Apache Derby Database is included in the installation. That’s fine but from 12.1.2 release it also starts automatically which is usually unwanted, useless and waste of resources. Previous versions of WebLogic didn’t automatically start the Derby database.

Fortunately you can disable it as basically there is a simple IF statement in the “$WL_DOMAIN_HOME$\bin\setDomainEnv.cmd” file:

@REM Set DERBY_FLAG, if derby is available.
 
if exist %WL_HOME%\common\derby\lib\derby.jar (
    set DERBY_FLAG=true
)

If you want to prevent Derby form starting you have three options:

  • Rename “derby.jar” to something else
  • Delete the IF statement from start-up script
  • Set the DERBY_FLAG to false in the startWeblogic.cmd script

I couldn’t find Oracle’s documentation about Derby in Weblogic but those four options seems to work. I prefer the third option which is quite easy to configure. (via Oracle Community)

In my “$WL_DOMAIN_HOME$\bin\startWebLogic.cmd” I added

...
@REM Call setDomainEnv here.
 
@REM Disabling Derby
set DERBY_FLAG = false
...

Transferring Linux install media to USB thumb drive

Optical drives are nowadays less common on laptops which makes making bootable install medias a little easier. Yes, easier by requiring you to use USB thumb drives. There are different ways to achieve what you want and here is couple of examples how to transfer Linux install media to USB thumb drive on Windows and Mac OS X.

First you need to download the install media of your choice as ISO image which you want to transfer to your USB thumb drive. In theory you need a media which is meant for USB thumb drive or is a Hybrid ISO image but practically all modern distributions’ media are Hybrid ISO images.

Windows

Plug in the USB thumb drive and format it with FAT32 file system

Download UNetbootin and transfer ISO content to USB thumb drive using UNetbootin.

  1. Select Diskimage and click the “. . .” button to find and open the ISO file. Make sure Type is USB drive and the Drive letter is what your Windows detect and click OK. Your ISO content will be copied to USB thumb drive.
  2. Next you also need to copy the ISO file that you used with UNetbootin to the root of the USB thumb drive, e.g. the ISO file should be F:\linuxmint-16-cinnamon-dvd-64bit.iso

Now you can proceed to boot up the machine that you want to install with e.g. Linux Mint 16.

Linux

Creating bootable USB thumb drive in Linux is much simpler. You just need install media which is meant for USB thumb drive or is a Hybrid ISO image. With hybrid images, you can simply write the content of the ISO and make a bootable USB stick which can install Linux by using the ‘dd’ command.

You just need to have the ISO image and one command:

$ sudo dd bs=4M if=linux-image.iso of=/dev/sdX oflag=direct

Where sdX is the device where your USB thumb drive is, e.g. sdb. If your system doesn’t support ‘oflag=direct’, you can just leave it out as it is simply intended to speed up the process a bit. If you leave the bs-option off the default will be 512 bytes and takes forever.

Mac OS X

OS X’s DiskUtil can’t burn ISO images to USB and gives “Invalid argument” but fortunately just like with Linux also Mac OS X makes things easy for you. You just need the install media which is meant for USB thumb drive or is a Hybrid ISO image. With hybrid images, you can simply write the content of the ISO and make a bootable USB stick which can install Linux by using the ‘dd’ command.

$ dd if=linux-image.iso of=/dev/rdiskX bs=8192

Where rdiskX is the device where your USB thumb drive is, e.g. rdisk1. You can see the device (disk identifier) from DiskUtility.app by selecting your USB stick and checking the Device section.

It’s useful to notice that on OS X you can use the ‘/dev/rdiskX’ device which stands for raw disk and is much faster than the block level device ‘/dev/diskX’ which is buffered and where data undergoes extra processing. For example rdiskX gives “823132160 bytes transferred in 625.470137 secs (1316022 bytes/sec)” and diskX gives “823132160 bytes transferred in 1296.675998 secs (634802 bytes/sec)”.

If you get an error e.g. “dd: /dev/rdisk2: Resource busy” then you have to make sure the Mac OS X isn’t using it and thus unmount it’s partitions. Open the DiskUtility.app, and on your USB hard drive, unmount any of it’s partitions. Do not eject the USB hard drive.

Do a clean install of Windows 8 with an upgrade key

There are times when you have to do a clean install of your Windows 8 but if you have just an upgrade key you need to make couple of extra hoops before you can activate the new install. The upgrade key doesn’t prevent you installing to a clean disk but when you try to activate, you get an error 0x8007007B, saying your product key can only be used for upgrading. Another fine example how Microsoft makes things complicated for legitimate users.

Fortunately there’s a way to fix that issue as Lifehacker’s article tells:

  • Open the Registry Editor (Win + R, type regedit).
  • Navigate to “HKEY_LOCAL_MACHINE/Software/Microsoft/Windows/CurrentVersion/Setup/OOBE/” and change the MediabootInstall key’s value from 1 to 0.
  • Open the Command Prompt (Win + R, type cmd). Right-click on the Command Prompt icon and run it as an administrator.
  • Type slmgr -rearm and press Enter.
  • Reboot Windows.

After that is done and you get back into Windows, you should be able to run the Activation utility and activate Windows as normal, without getting an error.

Apparently you can also call Microsoft Support and they will walk you through the proper way doing this because they understand you may have bought a new drive. There is a dialog you can get to in system tools where it will ask you for a numerical code. The support personnel will give you the number, you click “OK” after typing it in, then go the activation again and it works. This process is likely doing the work around mention above, but through an approved administrative process.

By the way, restarting Windows 8 is most easily done by left-clicking once on an empty spot on the desktop and holding Alt + F4).

Setting up LAMP stack on OS X

Setting up LAMP stack for web development on OS X can be done with 3rd party software like MAMP but as Mac OS X comes with pre-installed Apache and PHP it’s easy to use the native setup. You just need to configure Apache, PHP and install MySQL.

Setup Apache2

Set up the Server Name to localhost to suppress the warning about fully qualified domain name and enable PHP module.

$ sudo vim /etc/apache2/httpd.conf
ServerName localhost:80
LoadModule php5_module libexec/apache2/libphp5.so

Create “virtual hosts” under your Sites. Change the username to your account’s username.

~$ sudo vim /etc/apache2/users/username.conf
<VirtualHost *:80>
  ServerName dev
  DocumentRoot /Users/username/Sites
  VirtualDocumentRoot /Users/username/Sites/%-2/htdocs
  UseCanonicalName Off
 
  <Directory "/Users/username/Sites/*/htdocs">
    AllowOverride All
    Order allow,deny
    Allow from all
  </Directory>
</VirtualHost>

Now Apache serves your projects from your home directory’s Sites folder. Apache will serve files from the htdocs folder like “~/Sites/projectname/htdocs”.

Now just restart Apache and check that it’s running.

$ sudo apachectl restart
$ ps aux | grep httpd

Setup PHP

$ sudo cp /etc/php.ini.default /etc/php.ini

Edit php.ini for easier debugging:

error_reporting  =  E_ALL | E_STRICT
display_errors = On
html_errors = On

Setup MySQL

MySQL can be installed directly from Oracle’s MySQL packages or by using Homebrew.

Install Homebrew

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Install MySQL using Homebrew

$ brew install mysql

Install the MySQL system tables and have it run as your system user:

$ unset TMPDIR
$ mysql_install_db --verbose --user=`whoami` --basedir="$(brew --prefix mysql)" --datadir=/usr/local/var/mysql --tmpdir=/tmp

Start MySQL and check that it’s running

$ mysql.server start
$ ps aux | grep mysql

Reset the root password. Change the “5.5.27” to your installed version number.

$ /usr/local/Cellar/mysql/5.5.27/bin/mysqladmin -u root password 'YOUR_NEW_PASSWORD'

As we are using the Homebrew package for MySQL and the default php.ini file then PHP is trying to connect to MySQL through the default_socket at /var/mysql/mysql.sock which doesn’t exist as MySQL is using /tmp/mysql.sock. Just change all instances of /var/mysql/mysql.sock to /tmp/mysql.sock.

$ sudo sed -i "" "s:/var/mysql/mysql.sock:/tmp/mysql.sock:g" /etc/php.ini

And you’re done.

Web application test automation with Robot Framework

Software quality has always been important but seems that lately it has become more generally acknowledged fact that quality assurance and testing aren’t things to be left behind. With Java EE Web applications you have different ways to achieve test coverage and test that your application works with tools like JUnit, Mockito and DBUnit. But what about testing your web application with different browsers? One great way is to use Robot Framework which is a generic test automation framework and when combined with Selenium 2 it makes both writing your tests and running them quite intuitive.

Contents

Introduction

Robot Framework which is a generic test automation framework for acceptance testing and its tabular test data syntax is almost plain English and easy to understand. Its testing capabilities can be extended by test libraries implemented either with Python or Java, and users can create new higher-level keywords from existing ones using the same syntax that is used for creating test cases. Robot Framework itself is open source and released under Apache License 2.0, and most of the libraries and tools in the ecosystem are also open source. The development of the core framework is supported by Nokia Siemens Networks.

Robot Framework doesn’t do any specific testing activity but instead it acts as a front end for libraries like Selenium2Library. Selenium2Library is a web testing library for Robot Framework that leverages the Selenium 2 (WebDriver) libraries from the Selenium project. In practice it starts the browser (eg. IE, Firefox, Chrome) and runs the tests against it natively as a user would. There’s no need to manually click through the user interface.

Robot Framework has good documentation and by going through the “Web testing with Robot Framework and Selenium2Library” demo you see how it’s used in web testing, get introduction to test data syntax, how tests are executed, and how logs and reports look like. For more detailed view about Robot Framework’s features you can read the User Guide.

Installing test tools

The “Web testing with Robot Framework and Selenium2Library” demo is good starting point for getting to know Robot Framework but it more or less skips the details of setting up the system and as the installation instructions are a bit too verbose here is an example how to install and use Robot Framework and Selenium 2 in 64-bit Windows 7.

Python installation

First we need Python as a precondition to run Robot Framework and we install Python version 2.7.x as Robot Framework is currently not compatible with Python 3.x. From the Python download page select Python 2.7.9 Windows X86-64 Installer.

For using the RIDE editor we also need wxPython. From the download page select wxPython2.8-win64-unicode-py27 for 64-bit Python 2.7.

Next we need to set up the PATH environment variable in Windows if you didn’t setup it when you installed Python.

Open Start > Settings > Control Panel > System > Advanced > Environment Variables
Select System variables > PATH > Edit and add e.g. ;\Python27;C:\Python27\Scripts at the end of the value.
Exit the dialog with OK to save the changes.

Starting from Python 2.7.9, the standard Windows installer by default installs and activates pip.

Robot Framework and Selenium2Library installation

In practice it is easiest to install Robot Framework and Selenium2Library along with its dependencies using pip package manager. Once you have pip installed, all you need to do is running these commands in your Command Prompt:

1. pip install robotframework
2. pip install robotframework-selenium2library

It’s good to notice that pip has a “feature” that unless a specific version is given, they install the latest possible version even if that is an alpha or beta release. A workaround is giving the version explicitly. like pip install robotframework==2.7.7

RIDE installation

RIDE is a light-weight and intuitive editor for Robot Framework test case files. It can be installed by using Windows installer (select robotframework-ride-1.1.win-amd64.exe) or with pip using:

pip install robotframework-ride

The Windows installer does a shortcut to the desktop and you can start it from Command Prompt with command ride.py.

Now you have everything you need to create and execute Robot Framework tests.

Executing Robot Framework tests

As described in WebDemo running tests requires the demo application located under demoapp directory to be running. It can be started by executing it from the command line:

python demoapp/server.py

After the demo application is started, it is be available at http://localhost:7272 and it needs to be running while executing the automated tests. It can be shut down by using Ctrl-C.

In Robot Framework each file contains one or more tests and is treated as a test suite. Every directory that contains a test suite file or directory is also a test suite. When Robot Framework is executed on a directory it will go through all files and directories of the correct kind except those that start with an underscore character.

WebDemo’s test cases are located in login_tests directory and to execute them all type in your Command Prompt:

pybot login_tests

Running the tests opens a browser window which Selenium 2 is driving natively as a user would and you can see the interactions.
When the test is finished executing four files will have been generated: report.html, log.html and output.xml. On failed tests selenium takes screenshots which are named like selenium-screenshot-1.png. The browser can also be run on a remote machine using the Selenium Server.

You can also run an individual test case file and use various command line options (see pybot –help) supported by Robot Framework:

pybot login_tests/valid_login.txt
pybot --test InvalidUserName --loglevel DEBUG login_tests

If you selected Firefox as your browser and get an error like “Type Error: environment can only contain strings” that’s a bug in Selenium’s Firefox profile. You can fix it with a “monkey patch” to C:\Python27\Lib\site-packages\selenium\webdriver\firefox\firefox_profile.py.

Using different browsers

The browser that is used is controlled by ${BROWSER} variable defined in resource.txt resource file. Firefox browser is used by default, but that can be easily overridden from the command line.

pybot --variable BROWSER:Chrome login_tests
pybot --variable BROWSER:IE login_tests

Browsers like Chrome and Internet Explorer require separate Internet Explorer Driver and Chrome Driver to be installed before they can be used. InternetExplorerDriver can be downloaded from Selenium project and ChromeDriver from Chromium project. Just place them both somewhere in your PATH.

With Internet Explorer Driver you can get an error like “‘Unexpected error launching Internet Explorer. Protected Mode settings are not the same for all zones. Enable Protected Mode must be set to the same value (enabled or disabled) for all zones.'”. As it reads in the driver’s configuration you must set the Protected Mode settings for each zone to be the same value. To set the Protected Mode settings in Internet Explorer, choose “Internet Options…” from the Tools menu, and click on the Security tab. For each zone, there will be a check box at the bottom of the tab labeled “Enable Protected Mode”.

Reading the results

After the tests have run there are couple of result files to read: report.html and log.html.

The report.html shows the results of your tests and its background is green when all tests have passed and red if any have failed. It also shows “Test Statistics” for how many tests have passed and failed. “Test Details” shows how long the test took to run and, if it failed, what the fail message was.

The log.html gives you more detailed information about why some test fails if the fail message doesn’t make it obvious. It also gives a detailed view of the execution of each of the tests.

Summary

From the short experience I have played with Robot Framework it seems to be powerful tool for designing and executing tests and good way to improve your application’s overall quality.

Next it’s time to get to know the Robot Framework syntax better, write some tests and run Selenium Server. Also the Maven plugin and RobotFramework-EclipseIDE plugin looks interesting.

References

Robot Framework documentation
Robot Framework User Guide
Web testing with Robot Framework and Selenium2Library demo
RIDE: light-weight and intuitive editor for Robot Framework test case files

Running FishEye & Crucible as a service in Linux

Atlassian’s tools for supporting software development are great but they aren’t really admin friendly to start with. For example FishEye & Crucible doesn’t ship with scripts to start it at system boot time but with the help of Atlassian’s Wiki, sysadmin tasks and scripts you can run it as a normal service. First we create a dedicated user for crucible and second we add a new service for it. I have done this on CentOS 5.7 x86_64.

Setting up the service account

As the root user, create a separate “FishEye & Crucible” service account at root shell:

# useradd -c "FishEye & Crucible service account" -d /home/crucible -m crucible

To make it easier for this to work also after FishEye & Crucible upgrades we create a symbolic link to the latest version (modify “/opt/fecru” to match your deployment).

# ln -s /opt/fecru/fecru-2.7.15 /opt/fecru/latest

Then, ensure that this user is the filesystem owner of the FishEye & Crucible instance (modify “/opt/fecru” to match your deployment).

# chown -R crucible:crucible /opt/fecru

Running Crucible as a crucible user

Save the following script to /etc/init.d/crucible. Be sure to edit the FISHEYE_HOME value to the location where your FishEye/Crucible instance resides:

#!/bin/bash
# RUN_AS: The user to run fisheye & crucible as. Its recommended that you create a separate user account for security reasons
RUN_AS=crucible
 
# FISHEYE_HOME: The path to the FishEye & Crucible installation. Its recommended to create a symbolic link to the latest version so the process will still work after upgrades.
FISHEYE_HOME="/opt/fecru/latest"
# FISHEYE_INST: The path where the data itself will be stored.
export FISHEYE_INST="/opt/fecru/fecru-data"

fisheyectl() {
        if [ "x$USER" != "x$RUN_AS" ]; then
                # If running without FISHEYE_INST
                # su - "$RUN_AS" -c "$FISHEYE_HOME/bin/fisheyectl.sh $1"
                su - "$RUN_AS" -c "FISHEYE_INST=$FISHEYE_INST $FISHEYE_HOME/bin/fisheyectl.sh $1"
        else
                "$FISHEYE_HOME/bin/fisheyectl.sh $1"
        fi
} 

case "$1" in
        start)
                fisheyectl start
                ;;
        stop)
                fisheyectl stop
                ;;
        restart)
                fisheyectl stop
                sleep 10
                fisheyectl start
                ;;
        *)
                echo "Usage: $0 {start|stop|restart}"
esac
 
exit 0

After saving the script, modify it’s permissions so that it can be executed:

# chmod 755 /etc/init.d/crucible

Running Crucible as a service

Now that we have an init script we can add it as a service and be able to configure the system to run the script on startup (more precisely, ensure that Crucible runs in runlevels 3, 4 and 5):

chkconfig --add crucible
chkconfig crucible on

Verify that the script has been installed correctly:

# chkconfig --list crucible

After this has been done you can manually start or stop the service by using these commands:

service crucible stop
service crucible start

And you’re done.