Using Bacon2D with Sailfish OS

Sailfish OS development for Jolla is quite versatile and you can use QML plugins to enhance your application. But if you’ve used a QML module which isn’t approved in Harbour and want to distribute your application via Jolla Store, you’ve to do some extra steps with it. Lately I ported Ubuntu Phone’s Falldown game for Sailfish OS and it uses Bacon2D framework to ease 2D game development. Bacon2D isn’t allowed in Harbour so I had to patch it to accept custom namespace and either package the dynamically linked library with my app’s RPM or use statically linked Bacon2D with my app. Here are my notes about it.

The examples will be with my Falldown game project and the code can be found on GitHub. The steps shown below are not the optimal way of doing things as you could streamline the process by including Bacon2D build to your project build and in RPM spec patch the needed files. Will see if I get around doing that and automate things.

Building Bacon2D for Sailfish OS

Bacon2D is a framework to ease 2D game development, providing ready-to-use QML elements representing basic game entities needed by most of games. Bacon2D uses Box2D, a 2D rigid body simulation engine for games. Box2D lets you make objects move in realistic ways and make the game world more interactive.

You can use Bacon2D in your Sailfish OS project by using it as dynamically or statically linked library. The easiest way is to just use the statically linked version so it’s statically compiled into your project. But it’s almost as easy to build the dynamically linked library.

The other issue is getting the Bacon2D built such way that you can use it in your project and submit your application to Jolla Store. As Bacon2D plugin isn’t allowed in Harbour you’ve to patch it to your own namespace, e.g harbour.falldown.bacon2d.

The next chapters cover the three different ways to build and use Bacon2D with Sailfish OS.

Get Bacon2D framework

Start with adding the Bacon2D repository to your project’s Git repository as submodule.

$ git submodule add https://github.com/Bacon2D/Bacon2D.git

Now the Bacon2D project files are in e.g. falldown/Bacon2D and falldown directory will be our base.

Now you’re ready to build and use Bacon2D with three optional ways:

After I got the needed things to work, I used the Option 3 way to get Bacon2D included in my application and hopefully it gets approved to Jolla Store.

Option 1: Using statically linked Bacon2D

In your project’s project file e.g. harbour-falldown.pro add to the top of the file include clause for including Bacon2D statically

include(./Bacon2D/src/Bacon2D-static.pri)

In your project’s main.cpp include plugins.h and register types before loading QML resources.

#include <plugins.h>
 
Plugins plugins;
plugins.registerTypes("Bacon2D");

Build your project in Qt Creator by selecting Build->Release, Deploy->Deploy as RPM Package and then running Build -> Clean, Build, Deploy.

Your Bacon2D powered app should now launch in your phone.

Option 2: Using dynamically linked Bacon2D

Open Bacon2D/Bacon2D.pro with Qt Creator and disable the test and examples subdirectories.

SUBDIRS += src

Build Bacon2D for Armv7: Select Kit – MerSDK-SailfishOS-armv7hl – Release – Deploy by copying binaries. The build will fail with “Error 127” but the libbacon2dplugin.so is built so you can use it now.

Copy the “build-Bacon2D-MerSDK_SailfishOS_armv7hl-Release/src/imports” directory which contains QML files and libbacon2dplugin.so to your project’s lib directory, e.g. falldown/lib.

$ cp -r build-Bacon2D-MerSDK_SailfishOS_armv7hl-Release/src/imports/Bacon2D lib

In e.g. harbour-falldown.pro add the lib directory to be added inside RPM.

DEPLOYMENT_PATH = /usr/share/$${TARGET}
lib.files += lib
lib.path = $${DEPLOYMENT_PATH}
INSTALLS += lib

In your main.cpp add import path to Bacon2D files.

view->engine()->addImportPath(SailfishApp::pathTo("qml/imports/").toLocalFile());

In Qt Creator select Build->Release, Deploy->Deploy as RPM Package. Run Build -> Clean, Build, Deploy.

Your Bacon2D powered app for Sailfish OS should now launch in Jolla.

Option 3: Using statically linked Bacon2D the Harbour way

Including statically linked Bacon2D to be built with you Sailfish OS application can be done different ways and I chose not to write my own plugins.cpp to register QML types and just patched the plugins.cpp which Bacon2D provides.

In harbour-falldown.pro add to the top of the file include clause for including Bacon2D statically

include(./Bacon2D/src/Bacon2D-static.pri)

Patch Bacon2D Bacon2D/src/plugins.cpp to accept e.g. harbour.falldown.bacon2d namespace. Change every qmlRegisterType line from “Bacon2D” to your namespace.

qmlRegisterType<Layer>("harbour.falldown.bacon2d", 1, 0, "Layer");

In your project’s src/main.cpp include plugins.h and register types before loading QMLresources.

#include <plugins.h>
 
Plugins plugins;
plugins.registerTypes("harbour.falldown.bacon2d");

Put Bacon2D QML files under e.g. qml/components and change them to import e.g. harbour.falldown.bacon2d 1.0. Somehow I didn’t get the Bacon2D type to work which defines enums.

Build your project in Qt Creator by selecting Build->Release, Deploy->Deploy as RPM Package and then running Build -> Clean, Build, Deploy.

Your Bacon2D powered app should now launch in your phone.

Option 4: Using dynamically linked Bacon2D the Harbour way

You can link against shared libraries which ships with your Sailfish OS app in the RPM but because of Harbour rules you have to install the library under e.g. /usr/share/harbour-falldown/lib.

But before you can use the dynamically linked Bacon2D for Sailfish OS app intended to be distributed via Jolla Harbour you have to patch Bacon2D to accept e.g. harbour.falldown.bacon2d namespace and build the library.

In Bacon2D project change the following files.

Bacon2D/Bacon2D.pro

PROJECT_NAME = harbour.falldown.bacon2d

Bacon2D/src/src.pro

TARGETPATH = harbour/falldown/bacon2d
DESTDIR = $$OUT_PWD/imports/harbour/falldown/bacon2d/

Bacon2D/src/plugins.cpp

qmlRegisterType<Layer>("harbour.falldown.bacon2d", 1, 0, "Layer");

Bacon2D/src/qmldir

module harbour.falldown.bacon2d

In Bacon2D/src directory change every QML file to import e.g. harbour.falldown.bacon2d 1.0

Build Bacon2D with Qt Creator for Armv7: Kit – MerSDK-SailfishOS-armv7hl – Release – Deploy by copying binaries

Copy Bacon2D QML files and libbacon2dplugin.so under lib directory and to the structure which match you import namespace, e.g.: lib/harbour/falldown/bacon2d/.

$ cp -r build-Bacon2D-MerSDK_SailfishOS_armv7hl-Release/src/imports/harbour lib

In your project’s project file, e.g. harbour-falldown.pro add the lib directory to be added inside RPM

DEPLOYMENT_PATH = /usr/share/$${TARGET}
lib.files += lib
lib.path = $${DEPLOYMENT_PATH}
INSTALLS += lib

In your spec file, e.g. rpm/harbour-falldown.spec add following exclude as Harbour RPM packages should not provide anything.

# >> macros
%define __provides_exclude_from ^%{_datadir}/.*$
# << macros

Also to use QML modules which ships together with the application but you have to prefix the name of the imports with e.g. “harbour.falldown.bacon2d”. And you have to install them under /usr/share/harbour-falldown (loadable QML plugins or the QML files).

In src/main.cpp add import path to Bacon2D QML files and plugin before loading qml resources.

view->engine()->addImportPath(SailfishApp::pathTo("lib/").toLocalFile());

Build your project in Qt Creator by selecting Kit – MerSDK-SailfishOS-armv7hl – Release – Deploy – Deploy as RPM Package and then running Clean, Build, Deploy.

Your Bacon2D powered app should now launch in your phone.

Build secure Web applications by reading Iron-Clad Java

Building secure Web applications isn’t easy and contains many aspects that the development team has to consider and take into account. “Iron-Clad Java: Building Secure Web Applications” book is good starting point to learn concepts, tactics, patterns and anti-patterns to develop, deploy and maintain secure Java applications. With 304 pages the book is more about getting an overview and pointers for further reading and research but works quite nicely in that regard.

“Iron-Clad Java: Building secure Web applications”

As the name suggests, “Iron-Clad Java: Building Secure Web Applications” by Jim Manico and August Detlefsen, is targeted for Java developers and is suitable reading also for less technical people in the team like project managers and managers as it doesn’t go too deeply to technical aspects or code. After reading the book even the managers should get an appreciation and the vocabulary to discuss security with their staff. The reader should get a solid understanding of what is wrong with many web apps in general and what corrective measures to take to mitigate the issues. The book was published September 2014 and has 304 pages (ISBN-13: 978-0071835886).

The book covers topics like secure authentication and session management processes, access control design, defending against cross-site scripting (XSS) and cross-site request forgery (CSRF), protecting sensitive data while stored or in transit, preventing SQL injection, ensuring safe file I/O and upload, using effective logging, error handling, and intrusion detection methods and also guide for secure software development lifecycle (secure-SDLC). The topics are written with theory and practice and so that they are approachable for developers new to security, for those that might be a little inexperienced but still providing useful nuggets for experienced developers.

In good and bad the book gives somewhat opinionated answers what technics and tools you can use to address security issues but overall the advice is solid and un-biased and more or less framework agnostic. So, the lessons learned should apply to any project. For me, writing examples with e.g. JSP and Struts makes me question if also the other tools the book suggest (which I wasn’t familiar with) are suitable for modern Java EE application development. Something to look into after reading the book. Also OWASP seemed to have answer to almost every security aspect.

Anyways, the book’s advice isn’t about using certain technologies but giving you something to think about and educating about security aspects in your Java Web application. What matters is that the book gives explanations of why you need to implement a specific control for a given issue, how you could do it and what the impacts are. This is what many security professionals miss when speaking to developers. The book tells you what the security problem is and then why and how you should fix that so it makes sense for developers.

Taking care for Web application security isn’t just for architects and developers but it’s also a topic which importance whole team should know and understand. The “Iron-Clad Java: Building secure Web applications” gives good overview to security and is suitable for the whole development team to read.

Starting iOS development with Swift

Mobile application development differs between platforms and after doing couple of applications for the Sailfish OS powering Jolla phone it was finally time to see what other platforms have to provide. I develop Java applications at work so it was logical to look into iOS and learn some Swift. The Internet is full of resources of how to start developing for iOS and here’s my take to the topic. Now I just need an iPhone to run my app on a real device :)

Getting started

Coming from the Java EE and Web application world it’s good to read some documentation about mobile application development for iPhone and iOS before starting to code. You need to learn the basics concepts about iOS platform and Swift language and good starting point is to check Apple’s resources for developers and iOS developer library and read the guide how to start developing iOS apps (although it’s with Objective-C). To learn Swift you can read guide to Swift language or if you like books there’s also The Swift Programming Language book.

You can also start learning iOS development with several free or paid online courses. Coding with Chris “how to make an iPhone app” series of videos is a good starting point although it’s designed for people who have no programming experience. It provides nice overview to the tools and how to start developing. You can also follow the App Development: Teaching Swift by Apple Education with code examples or if you’ve the money and need diploma see the Udemy course for iOS developers or Udacity’s iOS developer Nanodegree.

It’s also good to read Human Interface Guidelines for iOS-based devices although the guidelines don’t provide any practical examples. It’s a good resource to learn how iOS applications should work, tells you how your app should look and behave and how to use the UI elements that UIKit provides. As I have done apps for Sailfish OS it was good to adapt my thinking to see things in the iOS way.

In practice the best way to learn is to just write code and experiment. Getting to know XCode and Interface Builder takes some time. After using Eclipse and IntelliJ IDEA for Java development both XCode and Apple’s graphical UI editor are somewhat confusing at first.

There’s also couple of iOS development newsletters to follow: iOS Dev Weekly, This Week In Swift. Also the In depth Mac and iOS articles archives is a good resource.

And if you’re using IRC there’s #cocoa-init channel on irc.freenode.net focused on asking and answering questions for beginning developers on iOS and OS X. For more general iOS development you can join #iphonedev channel on irc.freenode.net. To join them you need to register your nick.

Development in practice

Getting started with iOS can be challenging as Swift and Objective-C are used mainly only on Apple’s platform and it has its own names for almost everything. I haven’t yet gathered my own good practices so it’s great that you can read about iOS good practices from Futurice.

Basic tools

For iOS development your options with tools are somewhat limited as you need Mac computer running OS X (10.9.4 or later) for being able to run Apple Xcode and iOS SDK. The other option is to use JetBrains Appcode (99e) which is better (what I’ve understood). Xcode can be installed from Mac App Store and it comes with iOS SDK. Also although you can run your applications in iOS Simulator it helps to have a real device which helps you to understand how apps interact with users and what the look and feel should be. The documentation and examples gets you far but nothing beats to have first hand experience of the platform.

I found it beneficial to watch e.g. the Coding with Chris “how to make an iPhone app” series of videos for getting around XCode development environment.

Xcode and UI builder

Developing for iOS

iPhone applications can be written with Objective-C or with newer Swift programming language. Objective-C is built on top of the C programming language and provides object-oriented capabilities and a dynamic runtime. Swift in the other hand can be described as Objective without the C and is a replacement for the Objective-C language and works side-by-side with it. Wikipedia has good short description of Swift.

Although Swift is relatively new and what I’ve read isn’t quite as robust as Objective-C it’s good starting point for developing iOS apps. Having used some Objective-C for OS X apps I wouldn’t recommend it to anyone if you don’t actually need it.

iOS platform

Apple’s operating system for iPhone, iOS, provides many frameworks for developers and as a developer you’ve to decide which version to target as it affects your application and capabilities. Apple’s iOS developer page provides short overview of what it has to offer. The current version is iOS 8 and i.a. Shinobicontrols has written iOS 8 Day by Day eBook which consists of 39 blog posts covering the most significant features available to developers in iOS 8.

As a developer you have to choose which version of iOS you target and whether your app is universal or only for iPhone or iPad as it affects your code and potential users. Do you need the new features in iOS 8 or is iOS 7 enough and is it worth to make solutions to fit both versions? And do you need different user interface for bigger screen in iPad or is universal version enough? iOS 8 is supported on iPhone 4s or newer and newer iPads and what I read about 72% of devices are using iOS 8 and 25% are on iOS 7. So, I think it’s enough to target iOS 8 as it provides you more options how to implement your app.

Developing with Swift

The best way to get to know Swift is just read some code, watch tutorials and of course write code. To learn Swift you can read guide to Swift language and you can watch from Youtube e.g. rm2kdev series about Swift starting from the basics and example of doing a To Do List app.

One nice element of the Swift system which helps you to get hang of it, is its ability to be cleanly debugged and run within the development environment, using a read–eval–print loop (REPL). It gives you interactive properties more in common with the scripting capabilities of Python than traditional systems programming languages. It’s useful to play with Swift in Xcode Playground and see what your code does.

Knowing Swift language is just one part and the bigger part in my opinion is to know how to use the UI elements that UIKit provides to create good user experience. When I started developing my app with Swift majority of time went to figuring out how to construct the user interface with Xcode UI builder and what the elements should be, not actually writing much code in Swift. User experience section in iOS Developer Library and UI Elements in iOS Human Interface Guidelines provide good starting points for reading about user experience but doesn’t help you much with coding especially when the sample codes are in Objective-C. Basic UI elements like “pull to refresh”, “swipe between views”, “split view” and “slide-out navigation” are more easily found by googling.

We all have our own ways to learn new platform and programming language and I find it beneficial to just create small application which experiment with different concepts and interactions. I’m gathering my own experiments with iOS technologies to GitHub and building a news reader application for High.fi on the way.

So, what are you waiting for? Download Xcode and start developing your own app for iOS :)

Summary

Documentation

iOS 8

Other resources

Courses and guides

Book: Real World Java EE Night Hacks

Reading software development related books can be said to be unnecessary as all the information can be also found from the Internet but sometimes it’s easier to read all the related topics from one place. Adam Bien’s “Real World Java EE Night Hacks: Dissecting the Business Tier” is a book which walks through best practices and patterns used to create a Java EE 6 application and covers several important topics from architecture to performance and monitoring to testing. The book has 167 pages with source code so the topics are more about getting the idea than explaining them thoroughly. So if you’re new to Java EE 6 and patterns this book is for you. It gets you started and gives you topics to research more.

Real World Java EE Night Hacks

“Real World Java EE Night Hacks” walks through best practices and patterns used to create a real world Java EE application called “X-ray.” It’s a high-performance blog statistics application add-on for Apache Roller which is built with “vanilla” Java EE 6. It tells you about the core principles of Java EE like EJB 3.1, CDI, JPA, JTA, JAX-RS, Dependency Injection, Convention over Configuration, interceptors, transactions and binds them in “X-ray” application with source code to follow. The book is also more than just Java EE as it covers concepts like unit and integration testing, performance measuring and monitoring, continuous integration, real-time monitoring and timers and batch processing.

The book is easy to read although it isn’t for beginners as it requires you to know the Java jargon and main topics of Java EE. The book covers all the important topics regarding what you would need to know when building Java EE application but doesn’t explain or cover them thoroughly. It’s understandable as you would need more than one book to go them all through in sufficient detail. It’s more about telling you that there are this kinds of things to consider and how to apply them with Java EE application. It’s a starting point for your own research. It would’ve been also nice to have more pictures and diagrams in it.

In overall, the “Real World Java EE Night Hacks” is a decent book about implementing Java EE concepts and application architecture with best practices and patterns but it still feels a bit meager especially as the example isn’t an application you would first think of Java EE application to be.

WordPress theme development with Vagrant on OS X

Developing WordPress themes requires you to have either remote machine with the needed software or installing e.g. PHP and MySQL to your local machine. Although setting up the development environment (LAMP stack) in OS X is quite easy there’s also better option, to separate it from your machine and make it more like it’s on some hosting provider. And for that it was time to get to know how Vagrant works and how to utilize it for WordPress theme development on OS X. Also this way you can make separate environments for different projects. Here’s short article to get you started.

In short: We setup Homebrew to install packages, Virtualbox will be used to run an Ubuntu Linux virtual machine and then we will use Vagrant to start, stop and build the virtual machine from Vagrantfile.

Setting up Vagrant for WordPress development

Step 0: Install Homebrew

We will use Homebrew to install the needed packages on OS X. Homebrew is the missing package manager for OS X.

$ ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Step 1: Install VirtualBox

You can install Virtualbox manually or by using Homebrew.

$ brew cask install virtualbox

Step 2: Install Vagrant

Vagrant is software for creating and configuring lightweight, reproducible, and portable virtual development environments. It can be seen as a wrapper around virtualization software such as VirtualBox and around configuration management software such as Ansible, Chef, Salt or Puppet. Vagrant essentially builds virtual images from references (boxes) of a type of operating system with applications and dependent software.

You can install Vagrant from Vagrant site or by using Homebrew.

$ brew install Caskroom/cask/vagrant

Step 3: Vagrantfile for WordPress environment

To get started quickly we want to start with some premade Vagrantfiles and there are multiple choices for WordPress development.

One good choice could be WordPress Theme Review VVV which also installs Theme Unit test data and other plugins to test your theme. Other simple option is to use Vagrantpress which is a packaged development environment for developing WordPress themes and modules.

  • Clone the project, $ git clone https://github.com/chad-thompson/vagrantpress.git
  • Run $ vagrant plugin install vagrant-hostsupdater from command line
  • Rename the vagrantpress to the name you want for your project.
  • Navigate to the directory you just renamed.
  • Run the command $ vagrant up from the directory. It will ask your password to setup the hostname.
  • Open your browser to http://vagrantpress.dev

Working with the environment

To log in to the local WordPress installation: http://vagrantpress.dev/wp-admin/. The username is admin, the password is vagrant.

Your WordPress installation files can be seen in the directory you created for the VagrantPress.

You can access phpMyAdmin: http://vagrantpress.dev/phpmyadmin/ with username wordpress, password wordpress.

Using Vagrant

Using Vagrant is quite simple and there’s nice Getting started guide. In short there’s couple of commands you need. Also the virtual machine is located on your home directory under ~/.vagrant.d/boxes.

To start or resume working on any project, to install every dependency that project needs, and to set up any networking and synced folders, type this in your project directory.

$ vagrant up

See your Vagrant virtual machines’ status.

$ vagrant global-status

You can ssh to your project’s machine running in Vagrant.

$ vagrant ssh

You can also ssh to Vagrant’s Linux. The password is vagrant as also is the root password.

$ ssh vagrant@127.0.0.1 -p2222

When you’re done working for the day to suspend your VM:

$ vagrant halt

When you’re done playing around, you can remove all traces of it. Note: it also removes all the changes you made to the virtual machine, like your WordPress settings and data.

$ vagrant destroy.

If you ever have any issues with the VM, you should try to provision it with puppet again.

$ vagrant reload --provision

As Vagrant is running the virtual machines with Virtualbox you can also manage and see their status by starting Virtualbox GUI. Or using following commands.

$ VBoxManage list runningvms
$ VBoxManage list vms
$ VBoxManage controlvm <uuid> poweroff
$ VBoxManage unregistervm <uuid>

Setting up Bower and Gulp in Windows

Doing things manually is fine once but if you can automate things it’s better. With little tools you can speed up development and reduce recursive mundane tasks such as starting a project or setting up boilerplate code. I recently came across Bower which is a package manager for the web. With Bower you can fetch and install packages from all over, and it takes care of finding, downloading, and saving the stuff you’re looking for. The other interesting tool to help you get going is Gulp which enables you to automate and enhance your workflow. Let’s see how to put things together on Windows, nothing special just steps to get you started.

Gulp tasks

Install Git

Bower needs Git to work so first install Git if you don’t have it. I chose Git for Windows which gives you BASH emulation used to run Git from the command line, graphical user interface for using Git and Shell integration.

Just click through the installation.

Install Node.js

Bower depends on Node.js and NPM so you need to get Node.js. Just download the installation package from Node.js site and click through it.

Install Bower

After you have Node.js installed we can install Bower with npm. You might need to restart your Windows to get all the path variables setup so Npm can find them.

Open up the Git Bash or Command Prompt and Bower is installed globally by running the following command.

$ npm install -g bower

Once you have Bower installed you then can install packages and dependencies using these commands:

# Using a local or remote package
bower install <package>
 
# Using a specific version of a package
bower install <package>#<version>
 
# Search packages
$ bower search <package>

By default packages will be put in the bower_components directory which can be changed if you prefer. If you want your packages downloaded into js/libs you can achieve this by creating a .bowerrc file

.bowerrc

{
    "directory": "js/libs"
}

You can also create a bower.json file which allows you to define the packages needed along with dependencies and then simply run bower install to download packages. In our example we setup a simple Backbone.js application which uses Bootstrap.

bower.json

{
    "name": "Foobar",
    "version": "0.1.0",
    "dependencies": {
          "jquery": "~2.0.3",
          "underscore": "~1.5.0",
          "bootstrap": "~3.3.2",
          "backbone": "~1.1.2"
    }
}

Our bower.json describes that we want some JavaScript libraries and as we have defined the version with ~ it can have bigger minor versions, e.g. jquery version can be between 2.0.3 < 2.1.0. Read more about semantic versioner for npm.

Now after creating that file inside the app directory you can run the following command:

$ bower install

After that you should see all your JavaScript packages under bower_components folder.

Install Gulp

To automate and enhance your workflow you can use Gulp for example to copy the files where you want them. There are nice recipes to show how to benefit of Gulp.

Install Gulp globally with npm:

$ npm install --global gulp

Install Gulp also in your project devDependencies:

$ npm install --save-dev gulp

Now we can setup our Gulp dependencies which pull from npm. Create a new package.json file in your project root and just add an empty object, {} and save it.

Next we install gulp-bower plugin which we can use to install Bower packages.

$ npm install --save-dev gulp-bower

This will install all the needed dependencies in a node_modules folder and also automatically update our package.json file with these dependencies.

Finally we need to setup the gulpfile.js which defines our tasks we want to perform. First we define what we installed in npm step above and create a config object to hold various settings. The bowerDir is just the path to the bower_components. Finally we add task for running bower and default task. Our bower tasks basically runs bower install but by including in the gulpfile other contributors only have to run gulp bower and have them all setup and ready.

gulp.js

var gulp = require('gulp'),
    bower = require('gulp-bower');
 
var config = {
     bowerDir: './bower_components'
}
 
gulp.task('bower', function() {
    return bower()
        .pipe(gulp.dest(config.bowerDir))
});
 
$ gulp.task('default', ['bower']);

The default task runs the bower task and all the user has to do to setup the needed packages is to run

$ gulp

In our case running gulp just runs our bower task which downloads the JavaScript packages we need. Pretty simple.

Gulp is powerful tool and has many use cases but also needs some to get all things working like you want and even then you might need to make compromises. One crafty task for Gulp and Bower is to customize your Bootstrap theme. Also Mark Goodyear has written good article about Getting started with gulp which shows some typical use cases.

Essential IntelliJ IDEA keyboard shortcuts

Recently I switched from using Eclipse to IntelliJ IDEA as our Java EE application’s front-end was done with JavaScript and the support for front-end technologies in Eclipse is more or less non-existent. The switch for long time Eclipse user wasn’t easy as IDEA works a bit differently but the change was worth it. The biggest difference in daily work with IDE is the shortcuts which are quite different in IDEA. In theory you can use Eclipse keymap for shortcuts but it just doesn’t work like it should and in practice you have to learn the IDEA way. There are many posts in the Internet about keyboard shortcuts in IDEA but there’s always place for more :) So, here’s my list of shortcuts to keep in your finger memory.

Learn keymap with Key Promoter

To learn your way around IntelliJ IDEA’s keyboard shortcuts there’s nice “Key Promoter” plugin to train yourself. It prompts whenever you use the mouse when you could’ve used the keyboard instead (similar to Eclipse’s Mousefeed).

To install the plugin:

  1. Ctrl+Alt+S to pull up the Settings screen
  2. Filter on “plugin”. Click “Plugins”, then “Browse Repositories” at the bottom
  3. Filter on “key promoter”
  4. Double click to install
  5. Essential IntelliJ IDEA keyboard shortcuts

    IntelliJ IDEA keymap

    You may be tempted to just go with the Eclipse keymap but it’s better to learn the IDEA way although it’s quite irritating at start. You also should change some default IDEA keyboard shortcuts to better ones like “closing editor window” with Ctrl+F4 which is too cumbersome compared to the de facto Ctrl+W. And changing “comment current line or selection” with Ctrl+/ which is impossible with Finnish keyboards to Ctrl+7.

    If you want to know how Eclipse shortcuts map to IDEA there’s nice post about IntelliJ IDEA shortcuts for Eclipse users and I added some in my list.

    Recent Viewed or edited Files: CTLR + E / CTRL + SHIFT + E
    Shows you a popup with all the recent files that you have opened or actually changed in the IDE. If you start typing, you can filter the files.

    Go to Class or file: CTRL + N and CTRL + Shift + N
    Allows you to search by name for a Java file in your project. If you combine it with SHIFT, it searches any file. Adding ALT on top of that it searches for symbols. (Eclipse: Ctrl+Shift+T and Ctrl+Shift+R)

    Find and Replace in Path: CTRL + SHIFT + F / CTRL + SHIFT + R
    Allows you to find in files or replace in files. (Eclipse: Ctrl+H)

    Incremental Find: F3 / CTRL + L
    When using CTLR-F to find in current file the F3 lets you to loop through the results. (Eclipse: Ctrl+K)

    Delete line: CTRL + Y
    Delete current line under cursor. Yank it. (Eclipse: Ctrl+D)

    Goto line: CTRL + G
    Go to given line number. (Eclipse: Ctrl-L)

    Syntax Aware Selection: CTRL + W
    Allows you to select code with context. Awesome when you need to select large blocks or just specific parts of a piece of code.

    Complete Statement: CTRL + SHIFT + ENTER
    This will try to complete your current statement. How? By adding curly braces, or semicolon and line change.

    Smart Type Completion: CTRL + SHIFT + SPACE
    Like auto complete (CTRL + SPACE) but if you add a SHIFT you get the smart completion. This means that the IDE will try to match expected types that suit the current context and filter all the other options.

    Reformat source code and optimize imports: CTRL + ALT + L
    Allows you to reformat source code to meet the requirements of your code style. Lays out spacing, indents, keywords etc. Reformatting can apply to the selected text, entire file, or entire project.

    Quick Fix: Alt+Enter
    (Eclipse: Ctrl+1)
    Gives you a list of intentions applicable to the code at the caret.

    Paste one of the previous values from clipboard: CTRL + SHIFT + V
    Shows you a dialog to select previous value from the clipboard to be pasted.

    Comment or uncomment line or block: Ctrl+7 / Ctrl+Shift+7
    Allows you to comment or uncomment the current line or selected block of source code. This is originally Ctrl + / (Slash) which is impossible with Finnish keyboard layouts.

    Show Diff (in Changes): CTRL + D
    In Changes tab compares the file with latest repository version.

    Highlight Usages: CTRL + SHIFT + F7
    Place the cursor in a element and after pressing the cursor the IDE will highlight all the occurrences of the selected element.

    Go to Declaration: CTRL + B
    If you place the cursor in a class, method or variable and use the shortcut you will immediately jump to the declaration of the element.

    Navigate Between Methods: ALT + UP/DOWN Arrows
    Jump between methods.

    Refactoring String Fragments: CTRL + ALT + V
    Refactor hardcoded string into variable/field/constant. Select the section of the String you want to extract, and use the normal “Extract…” shortcuts to extract it into a variable.

    Other useful keyboard shortcuts

    There are many useful keyboard shortcuts and you can print them from Help > Default Keymap Reference. Here are some more shortcuts which are also handy.

    Update application: CTRL + F10
    Current file structure: CTRL + F12
    Bookmarks: F11 and SHIFT + F11
    Generate Getters & Setters (in editor): ALT + INSERT
    Create New _* (in project navigator): ALT + INSERT
    Refactor – Rename: SHIFT + F6
    Open Settings CTRL + Alt + S
    Duplicate line: CTRL + D
    Move line: CTRL + ALT + UP / DOWN
    Find Command: CTRL + SHIFT + A
    Show usages in a pop-up list: CTRL + Alt + F7
    Extract Variable/Method/Constant/Field: CTRL + ALT + V/M/C/F
    Quick JavaDoc Popup: CTRL + Q
    Tab switcher: CTRL + TAB
    Jump to Project Navigator: ALT + 1
    Jump back to last tool window/panel: F12
    Jump to beginning/end of block (e.g., method start/end): CTRL + [ and CTRL + ]
    Toggle uppercase/lowercase of selection: CTRL + SHIFT + U
    Toggle collapse/expand: CTRL + .
    Toggle full screen editor (hide other tool windows): CTRL + SHIFT + F12

    Not a keyboard shortcut exactly but the “iter” smart template is great. If you want to iterate though something using a for loop type “iter” then TAB to use the live template. It will figure out the most likely variable you want to iterate over and generate a for loop for it. In Eclipse it worked more logically with just typing for and then autocomplete.

Getting Git Right in Helsinki

Software development is fun if you have tools which work great and support what you’re doing. So it was finally great to get hear Sven Peters talking about better software development in teams as Atlassian’s Getting Git Right landed to Helsinki (24.11.2014). Event about Git and of course about Atlassian’s tools.

Getting Git right by svenpet and durdn

Getting Git Right’s main theme was about happy developers, productive teams and how Git and Atlassian’s tools help to achieve that. Sven Peters and Nicola Paolucci presented how to be a happier developer with Git, and how to ship software faster and smarter. It’s good to remember that developing software is after all a social challenge, not a technical one. And Git helps you with it. The presentation slides are available at SlideShare and you can also watch it on Youtube (different event).

Git: You can rewrite history. Timemachine without paradoxes

Nicola Paolucci gave a nice and 5 minute talk about Git and it’s internals. Lot’s of technical details. The main points were “Fast and compact”, “Freedom and safety”, “Explore and understand”, “Control and assemble”. With Git you can rewrite the history safely to e.g. clean commits. Paolucci showed also some tools to help working with Git on the command line like hooks they use and using “better” prompt like liquid-prompt. For GUI you can use Atlassian’s SourceTree.

Git datamodel

Merging

Interesting part of the event was talk about what is efficient and the best Git workflow? The answer is “we don’t know”. It depends as there are different cultures, different products and different teams. There’s no right way but there are some good workflows which might work for you.

One is to use branch per issue, e.g. hot-fix/jira-30-user-avatars, feature/jira-27-user-sign. The simplest workflow is to use feature branches with develop branch. Then the master is very stable. If you have multiple product versions then release branches are good and bug fixes are done to separate branch and merged to other branches.

They also presented how Atlassian’s Stash can help you to work with Git and branches. Like merging changes to branches can be done automatically with hooks or by using Stash. Stash looked nice for controlling and managing your repository with visual interface.

Code reviews: do they feel like this?

Git also helps you to improve code quality with e.g. code reviews. Code reviews shouldn’t be painful as it’s about team ownership, shared knowledge and aim for better code but often there’s developer guilt. It can be made easier by making code reviews part of your daily work by doing it in small patches like pull requests.

Development is also about communication and for that Atlassian presented HipChat. It looked quite nice tool for following what’s happening in a project with aggregating team chat and information from different tools. Following commits and continuous information brings you clear view what’s happening. There are also alternatives to HipChat like Slack or just basic IRC.

But why should you use Git? Benefits like more time to code, better collaboration, dev productivity and it’s the future doesn’t convince everyone. Like pointy haired bosses. So it’s good to remember that Git is also about economics. Delivering software faster, having less bugs and thus having happy customers. Shipping software faster and smarter.

Why Git? from Peters’ slides:

Why Git?

It’s also about economics

In Questions and Answers session there was talk about Atlassian’s strategy with Bitbucket and Stash. They said that both are going strong as they have different use cases. Stash has more enterprise features and you can have the repository on your own premises. Bitbucket is about hosting the repository in Atlassian’s platform and for small and medium team. What I have used Bitbucket it’s nice service but not as user friendly as GitHub. Another interesting question was about storing binary files in Git. There’s no optimal solution yet but just some workarounds like git-annex and git-media which allows managing files with git, without checking the file contents into git. In practice you shouldn’t store binary files in Git and you should separate product to code and assets.

Summary

Atlassians Getting Git Right was nice event and gave good overview about Git and how to use it in software development team. It would have been nice to hear something about the alternatives to Atlassian’s tools (BitBucket, Stash, SourceTree and HipChat) which helps you to do better software development. I can’t deny that Atlassian’s tools work nicely together but sometimes the price is just too high.

Now it’s time to start using Git also on work projects and as all participants got “Just do Git” T-shirt it’s easier :) Thanks to Atlassian and Ambientia for arranging this event.

Authentication with LDAP provider in WebLogic gets stuck

Lately we upgraded our Java EE applications to new platform and began seeing stuck threads and slow starting times. The platform was upgraded from OC4J to WebLogic 12c and also the underlying LDAP service was changed to Oracle Access Manager. Looking at the server logs the one possible reason for stuck threads was quite clear: LDAP requests.

Fortunately the stuck threads problem with LDAP was a known problem with Oracle Weblogic Server 10.3.2 and later and covered in Oracle Support doc 1436044.1. The LDAP provider fails to authenticate for some users and the server logs show Stuck Threads in LDAP requests:

<10.9.2014 11.43.46 EEST> <Error> <WebLogicServer> <BEA-000337> <[STUCK] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "607" seconds working on the request "Workmanager: default, Version: 0, Scheduled=true, Started=true, Started time: 607304 ms
", which is more than the configured time (StuckThreadMaxTime) of "600" seconds in "server-failure-trigger". Stack trace:
        java.lang.Object.wait(Native Method)
        java.lang.Object.wait(Object.java:503)
        netscape.ldap.LDAPMessageQueue.waitForMessage(Unknown Source)
        netscape.ldap.LDAPMessageQueue.waitFirstMessage(Unknown Source)
        netscape.ldap.LDAPConnection.sendRequest(Unknown Source)
        netscape.ldap.LDAPConnection.search(Unknown Source)
        weblogic.security.providers.authentication.LDAPAtnDelegate.getDNForUser(LDAPAtnDelegate.java:3771)
        weblogic.security.providers.authentication.LDAPAtnDelegate.userExists(LDAPAtnDelegate.java:2384)
        weblogic.security.providers.authentication.LDAPAtnLoginModuleImpl.login(LDAPAtnLoginModuleImpl.java:199)
        com.bea.common.security.internal.service.LoginModuleWrapper$1.run(LoginModuleWrapper.java:110)
        java.security.AccessController.doPrivileged(Native Method)
        com.bea.common.security.internal.service.LoginModuleWrapper.login(LoginModuleWrapper.java:106)
        sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        java.lang.reflect.Method.invoke(Method.java:606)
        javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)
        javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
        javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
        javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
        java.security.AccessController.doPrivileged(Native Method)
        javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)
        javax.security.auth.login.LoginContext.login(LoginContext.java:595)
        ...

The cause for this is that authentication requests are hanging whenever the LDAP server is slow. By default, connections and searches to the LDAP server do not time out, so if the LDAP server is slow, authentication requests may take a very long time to retry. This can be seen as many threads stuck doing LDAP searches.

The solution is to set a timeout on LDAP requests for example as below (described in Oracle Support doc 1436044.1):

  1. Log in to the WLS Administration Console.
  2. Navigate to Security Realms -> myrealm -> Providers -> “your_ldap_authenticator.”
  3. Select the following values:
    • Connect Timeout 30
    • Results Time Limit 5000
    • Uncheck “Keep Alive Enabled”
  4. Save and apply changes. Restart the required servers if prompted.

NOTE: The optimal values may differ from environment to environment. But we can try the values specified here as starting places, and they will help in most cases like this. Our original values for the LDAP authenticator settings were: Connect Timeout: 0; Results Time Limit: 0; Keep Alive Enabled unchecked.

This is still a partial solution as you should investigate why the LDAP is slow. For now this solves our problem but has some side effects with user authentication.

Distribute project’s artifacts in Maven Central with OSSRH

You have developed some crafty Java library or Maven plugin and now you want to distribute it to other users through Maven Central repository? Using Sonatype Open Source Software Repository Hosting Service is a nice way for open source projects to achieve that and there’s two options to get your artifacts in it: 1. Release with Sonatype’s process or 2. Use Bintray’s process. In short, both options end up with the same results, your project’s artifacts are in Maven Central repository, but using Sonatype’s process and Maven release plugin makes it easy to release new version after you have set it up.

Using Sonatype’s service for pushing artifacts to Maven Central

Sonatype Open Source Software Repository Hosting Service (OSSRH) provides Maven repository hosting service for open source projects. You can deploy snapshots, stage releases, and promote your releases so they will be synced to Maven Central.

The process with Sonatype has a bit more steps than Bintray but if you follow the user guide, quite easy. And now as they have made some changes to their release process it’s easier more or less the same than using Bintray. All you need to do is to sign up a Sonatype JIRA account, create a JIRA ticket and make some POM/settings configuration and then use Maven release plugin to perform release. The procedure is described in Sonatype OSS Maven Repository Usage Guide but here is a short recap.

Performing release deployment with the Maven release plugin

Using Maven to perform release deployment to OSSRH is described in Sonatype’s guide which tells you how to do it manually or through the Maven release plugin which I find convenient.

First start with Initial Setup and create your JIRA account and new project ticket. The new project ticket triggers creation of your repositories. While your repositories are taken care of you should review the requirements for components in Central.

The artifacts need to be GPG signed so if you yet haven’t got a GPG key follow this guide to create one. Then you need to have your GPG key in a keyserver so start by sending your GPG key to keyserver:

gpg --keyserver pool.sks-keyservers.net --send-keys <your KEYID>

If you have more than one GPG key in your keychain then it’s useful to set a default key to be used for signing. Edit your ~/.gnupg/gpg.conf and uncomment the “default-key KEYID” line and add your KEYID.

Your Maven project settings

Configure your Maven settings.xml to have Sonatype OSSRH credentials (in .m2/settings.xml).

<settings>
     ...
    <servers>
    <server>
      <id>ossrh</id>
      <username>your-jira-id</username>
      <password>your-jira-pwd</password>
    </server>
  </servers>
     ...
</settings>

Edit your’project’s pom.xml for Maven release

Add the distributionManagement section to point to OSSRH.

<distributionManagement>
	<snapshotRepository>
		<id>ossrh</id>
		<url>https://oss.sonatype.org/content/repositories/snapshots</url>
	</snapshotRepository>
	<repository>
		<id>ossrh</id>
		<url>https://oss.sonatype.org/service/local/staging/deploy/maven2/</url>
	</repository>
</distributionManagement>

The configuration for the Maven release plugin should include disabling the release profile that is part of the Maven Super POM, since we are using our own profile, and specify the deploy goal together with the activation of our release profile

<pluginManagement>
          <plugins>
                <plugin>
                     <groupId>org.apache.maven.plugins</groupId>
                     <artifactId>maven-release-plugin</artifactId>
                     <version>2.5</version>
                     <configuration>
                          <useReleaseProfile>false</useReleaseProfile>
                          <releaseProfiles>release</releaseProfiles>
                          <goals>deploy</goals>
                     </configuration>
                </plugin>
          </plugins>
</pluginManagement>

Add the nexus-staging plugin to define where’s Nexus and if we release it right after closing.

<plugin>
	<groupId>org.sonatype.plugins</groupId>
	<artifactId>nexus-staging-maven-plugin</artifactId>
	<version>1.6.3</version>
	<extensions>true</extensions>
	<configuration>
		<serverId>ossrh</serverId>
		<nexusUrl>https://oss.sonatype.org/</nexusUrl>
		<autoReleaseAfterClose>true</autoReleaseAfterClose>
	</configuration>
</plugin>

Add the release profile for signing with GPG & Javadoc and Sources Attachments

<profile>
	<id>release</id>
	<build>
		<plugins>
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-source-plugin</artifactId>
				<version>2.2.1</version>
				<executions>
					<execution>
						<id>attach-sources</id>
						<goals>
							<goal>jar-no-fork</goal>
						</goals>
					</execution>
				</executions>
			</plugin>
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-javadoc-plugin</artifactId>
				<version>2.9.1</version>
				<executions>
					<execution>
						<id>attach-javadocs</id>
						<goals>
							<goal>jar</goal>
						</goals>
					</execution>
				</executions>
			</plugin>
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-gpg-plugin</artifactId>
				<version>1.5</version>
				<executions>
					<execution>
						<id>sign-artifacts</id>
						<phase>verify</phase>
						<goals>
							<goal>sign</goal>
						</goals>
					</execution>
				</executions>
			</plugin>
		</plugins>
	</build>
</profile>

You can see the full pom.xml e.g from my markdown-page-generator-plugin’s pom.xml.

Publishing Snapshots

Snapshot deployment are performed when your version ends in -SNAPSHOT. You do not need to fulfil the requirements when performing snapshot deployments and can simply run

$ mvn clean deploy

Successfully deployed SNAPSHOT versions will be found in Snapshot repository. If you need to delete your artifacts, you can log in to Sonatype’s Nexus using the same credentials you use to access to the Sonatype JIRA.

Performing a Release Deployment

The Maven Release Plugin can be used to automate the changes to the Maven POM files, sanity checks, the SCM operations required, the actual deployment execution and you can perform a release deployment to OSSRH with the following steps.

Prepare the release by answering the prompts for versions and tags

$ mvn release:clean release:prepare

If prepare fails use

$ mvn release:rollback
$ mvn release:clean

The prepare will create a new tag in SCM, even in GitHub, automatically checking in on your behalf. For it to work you need to have working public key in GitHub for git-push.

Perform the release. The perform process will ask for your gpg.passphrase if you don’t give it with passphrase argument.

$ mvn release:perform

For some reason I can’t provide the GPG passphrase when Maven GPG plugin asks it and I have to use the -Dgpg.passphrase argument.

$ mvn -Darguments="-Dgpg.passphrase=&lt;your GPG key passphrase&gt; release:perform

If release:perform fails to error “gpg: signing failed: No pinentry” then on macOS you need to restart gpg-agent and define pinentry-program. gnupg21 comes with pineentry as dependency. Or see Gist about commit signing with GPG on macOS.

$ killall gpg-agent && gpg-agent --daemon --use-standard-socket --pinentry-program /usr/local/bin/pinentry

This execution will deploy to OSSRH and release to the Central Repository in one go, thanks to the usage of the Nexus Staging Maven Plugin with autoReleaseAfterClose set to true.

Now your artifacts are in the Releases repository. The updates to The Central Repository search can take up to two hours. Once your artifacts are released you will not be allowed to update or delete them.

The first time you promote a release, you need to comment on the OSSRH JIRA ticket you created so OSSRH can know you are ready to be synced.

Manually Releasing the Deployment to the Central Repository

If you don’t want to release artifacts automatically you can also do it manually. Set the autoReleaseAfterClose set to false so you can inspect and potentially release the deployed artifacts manually. Then to release your artifacts is done through Nexus as described in the guide.

Using Bintray for pushing artifacts to Maven Central

Bintray offers developers the fastest way to publish and consume OSS software releases. Whether you are distributing software packages or downloading ones”. In short it provides i.a. an alternative way to release artifacts to Maven Central and to Sonatype Open Source Software Repository.

I found out about Bintray after I had already used the Sonatype’s way to release artifacts so I haven’t tested this myself. Bintray’s process is more or less similar than Sonatype’s but if I hadn’t read the blog post about how to do it I wouldn’t have know where to start. In short the process is the following.

  1. Register to Bintray and set up auto-signing: Generate yourself a keypair, if you don’t have one. Add it to your profile, and setup your default Maven repo (or a new one) for signing with your GPG key: Bintray can then sign your jars automatically.
  2. Add your Sonatype account under “accounts”. If you don’t have one, follow this procedure
  3. Create and link your package: Import from a GitHub repo or create a new package for your Maven project.
  4. You can link your package to JCenter by clicking “Add to JCenter”.
  5. Set up Maven up to deploy to Bintray by copy-pasting the pom.xml snippets from “Set me up!” guide.

Then for each release:

  1. Deploy: Deploy files by running your build tool.
  2. Publish: Review the build artifacts and publish the version files if satisfied.
  3. Sync: On the version page go to the Maven Central tab, enter your Sonatype password and click “Sync” and you’re done!
  4. Your package is now in https://oss.sonatype.org/content/repositories/releases and will be synced to Maven Central (usually takes time). In case of a sync problem, Bintray will automatically take care of any needed cleanup.

Summary

For distributing your open source projects’ artifacts through Maven Central repository and OSSRH you have two options: 1. Release with Sonatype’s process or 2. Use Bintray’s process. Both options end up with the same results: your project’s artifacts are in Maven Central repository. In my opinion if you’re using Maven then it doesn’t really matter which option you choose.

Before Sonatype simplified their process for making releases to OSSRH the Bintray’s process was the only almost pain-free gateway to Maven Central. But with the new Sonatype way the two alternative ways are more or less the same and I could say that Sonatype’s new process feels a bit easier when using the Maven release plugin. In practice there’s not much differences, e.g. both supports SNAPSHOTs, Sonatype in OSSRH and Bintray in oss.jfrog.org. Bintray is more flexible especially if not using Maven so if you’re using other tools like Gradle then it’s the easier way to go.