Getting Git Right in Helsinki

Software development is fun if you have tools which work great and support what you're doing. So it was finally great to get hear Sven Peters talking about better software development in teams as Atlassian's Getting Git Right landed to Helsinki (24.11.2014). Event about Git and of course about Atlassian's tools.

Getting Git right by svenpet and durdn

Getting Git Right’s main theme was about happy developers, productive teams and how Git and Atlassian's tools help to achieve that. Sven Peters and Nicola Paolucci presented how to be a happier developer with Git, and how to ship software faster and smarter. It's good to remember that developing software is after all a social challenge, not a technical one. And Git helps you with it. The presentation slides are available at SlideShare and you can also watch it on Youtube (different event).

Git: You can rewrite history. Timemachine without paradoxes

Nicola Paolucci gave a nice and 5 minute talk about Git and it’s internals. Lot's of technical details. The main points were "Fast and compact”, "Freedom and safety”, "Explore and understand”, "Control and assemble”. With Git you can rewrite the history safely to e.g. clean commits. Paolucci showed also some tools to help working with Git on the command line like hooks they use and using "better" prompt like liquid-prompt. For GUI you can use Atlassian's SourceTree.

Git datamodel

Merging

Interesting part of the event was talk about what is efficient and the best Git workflow? The answer is "we don't know". It depends as there are different cultures, different products and different teams. There’s no right way but there are some good workflows which might work for you.

One is to use branch per issue, e.g. hot-fix/jira-30-user-avatars, feature/jira-27-user-sign. The simplest workflow is to use feature branches with develop branch. Then the master is very stable. If you have multiple product versions then release branches are good and bug fixes are done to separate branch and merged to other branches.

They also presented how Atlassian's Stash can help you to work with Git and branches. Like merging changes to branches can be done automatically with hooks or by using Stash. Stash looked nice for controlling and managing your repository with visual interface.

Code reviews: do they feel like this?

Git also helps you to improve code quality with e.g. code reviews. Code reviews shouldn't be painful as it's about team ownership, shared knowledge and aim for better code but often there's developer guilt. It can be made easier by making code reviews part of your daily work by doing it in small patches like pull requests.

Development is also about communication and for that Atlassian presented HipChat. It looked quite nice tool for following what’s happening in a project with aggregating team chat and information from different tools. Following commits and continuous information brings you clear view what's happening. There are also alternatives to HipChat like Slack or just basic IRC.

But why should you use Git? Benefits like more time to code, better collaboration, dev productivity and it's the future doesn't convince everyone. Like pointy haired bosses. So it's good to remember that Git is also about economics. Delivering software faster, having less bugs and thus having happy customers. Shipping software faster and smarter.

Why Git? from Peters' slides:

Why Git?

It's also about economics

In Questions and Answers session there was talk about Atlassian's strategy with Bitbucket and Stash. They said that both are going strong as they have different use cases. Stash has more enterprise features and you can have the repository on your own premises. Bitbucket is about hosting the repository in Atlassian's platform and for small and medium team. What I have used Bitbucket it's nice service but not as user friendly as GitHub. Another interesting question was about storing binary files in Git. There's no optimal solution yet but just some workarounds like git-annex and git-media which allows managing files with git, without checking the file contents into git. In practice you shouldn't store binary files in Git and you should separate product to code and assets.

Summary

Atlassians Getting Git Right was nice event and gave good overview about Git and how to use it in software development team. It would have been nice to hear something about the alternatives to Atlassian's tools (BitBucket, Stash, SourceTree and HipChat) which helps you to do better software development. I can't deny that Atlassian's tools work nicely together but sometimes the price is just too high.

Now it's time to start using Git also on work projects and as all participants got “Just do Git” T-shirt it's easier :) Thanks to Atlassian and Ambientia for arranging this event.

Authentication with LDAP provider in WebLogic gets stuck

Lately we upgraded our Java EE applications to new platform and began seeing stuck threads and slow starting times. The platform was upgraded from OC4J to WebLogic 12c and also the underlying LDAP service was changed to Oracle Access Manager. Looking at the server logs the one possible reason for stuck threads was quite clear: LDAP requests.

Fortunately the stuck threads problem with LDAP was a known problem with Oracle Weblogic Server 10.3.2 and later and covered in Oracle Support doc 1436044.1. The LDAP provider fails to authenticate for some users and the server logs show Stuck Threads in LDAP requests:

<10.9.2014 11.43.46 EEST> <Error> <WebLogicServer> <BEA-000337> <[STUCK] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "607" seconds working on the request "Workmanager: default, Version: 0, Scheduled=true, Started=true, Started time: 607304 ms
", which is more than the configured time (StuckThreadMaxTime) of "600" seconds in "server-failure-trigger". Stack trace:
        java.lang.Object.wait(Native Method)
        java.lang.Object.wait(Object.java:503)
        netscape.ldap.LDAPMessageQueue.waitForMessage(Unknown Source)
        netscape.ldap.LDAPMessageQueue.waitFirstMessage(Unknown Source)
        netscape.ldap.LDAPConnection.sendRequest(Unknown Source)
        netscape.ldap.LDAPConnection.search(Unknown Source)
        weblogic.security.providers.authentication.LDAPAtnDelegate.getDNForUser(LDAPAtnDelegate.java:3771)
        weblogic.security.providers.authentication.LDAPAtnDelegate.userExists(LDAPAtnDelegate.java:2384)
        weblogic.security.providers.authentication.LDAPAtnLoginModuleImpl.login(LDAPAtnLoginModuleImpl.java:199)
        com.bea.common.security.internal.service.LoginModuleWrapper$1.run(LoginModuleWrapper.java:110)
        java.security.AccessController.doPrivileged(Native Method)
        com.bea.common.security.internal.service.LoginModuleWrapper.login(LoginModuleWrapper.java:106)
        sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        java.lang.reflect.Method.invoke(Method.java:606)
        javax.security.auth.login.LoginContext.invoke(LoginContext.java:762)
        javax.security.auth.login.LoginContext.access$000(LoginContext.java:203)
        javax.security.auth.login.LoginContext$4.run(LoginContext.java:690)
        javax.security.auth.login.LoginContext$4.run(LoginContext.java:688)
        java.security.AccessController.doPrivileged(Native Method)
        javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:687)
        javax.security.auth.login.LoginContext.login(LoginContext.java:595)
        ...

The cause for this is that authentication requests are hanging whenever the LDAP server is slow. By default, connections and searches to the LDAP server do not time out, so if the LDAP server is slow, authentication requests may take a very long time to retry. This can be seen as many threads stuck doing LDAP searches.

The solution is to set a timeout on LDAP requests for example as below (described in Oracle Support doc 1436044.1):

  1. Log in to the WLS Administration Console.
  2. Navigate to Security Realms -> myrealm -> Providers -> "your_ldap_authenticator."
  3. Select the following values:
    • Connect Timeout 30
    • Results Time Limit 5000
    • Uncheck "Keep Alive Enabled"
  4. Save and apply changes. Restart the required servers if prompted.

NOTE: The optimal values may differ from environment to environment. But we can try the values specified here as starting places, and they will help in most cases like this. Our original values for the LDAP authenticator settings were: Connect Timeout: 0; Results Time Limit: 0; Keep Alive Enabled unchecked.

This is still a partial solution as you should investigate why the LDAP is slow. For now this solves our problem but has some side effects with user authentication.

Distribute project's artifacts in Maven Central with OSSRH

You have developed some crafty Java library or Maven plugin and now you want to distribute it to other users through Maven Central repository? Using Sonatype Open Source Software Repository Hosting Service is a nice way for open source projects to achieve that and there's two options to get your artifacts in it: 1. Release with Sonatype's process or 2. Use Bintray's process. In short, both options end up with the same results, your project's artifacts are in Maven Central repository, but using Sonatype's process and Maven release plugin makes it easy to release new version after you have set it up.

Using Sonatype's service for pushing artifacts to Maven Central

Sonatype Open Source Software Repository Hosting Service (OSSRH) provides Maven repository hosting service for open source projects. You can deploy snapshots, stage releases, and promote your releases so they will be synced to Maven Central.

The process with Sonatype has a bit more steps than Bintray but if you follow the user guide, quite easy. And now as they have made some changes to their release process it's easier more or less the same than using Bintray. All you need to do is to sign up a Sonatype JIRA account, create a JIRA ticket and make some POM/settings configuration and then use Maven release plugin to perform release. The procedure is described in Sonatype OSS Maven Repository Usage Guide but here is a short recap.

Performing release deployment with the Maven release plugin

Using Maven to perform release deployment to OSSRH is described in Sonatype's guide which tells you how to do it manually or through the Maven release plugin which I find convenient.

First start with Initial Setup and create your JIRA account and new project ticket. The new project ticket triggers creation of your repositories. While your repositories are taken care of you should review the requirements for components in Central.

The artifacts need to be GPG signed so if you yet haven't got a GPG key follow this guide to create one. Then you need to have your GPG key in a keyserver so start by sending your GPG key to keyserver:

gpg --keyserver pool.sks-keyservers.net --send-keys <your KEYID>

If you have more than one GPG key in your keychain then it’s useful to set a default key to be used for signing. Edit your ~/.gnupg/gpg.conf and uncomment the "default-key KEYID" line and add your KEYID.

Your Maven project settings

Configure your Maven settings.xml to have Sonatype OSSRH credentials (in .m2/settings.xml).

<settings>
     ...
    <servers>
    <server>
      <id>ossrh</id>
      <username>your-jira-id</username>
      <password>your-jira-pwd</password>
    </server>
  </servers>
     ...
</settings>

Edit your'project's pom.xml for Maven release

Add the distributionManagement section to point to OSSRH.

<distributionManagement>
	<snapshotRepository>
		<id>ossrh</id>
		<url>https://oss.sonatype.org/content/repositories/snapshots</url>
	</snapshotRepository>
	<repository>
		<id>ossrh</id>
		<url>https://oss.sonatype.org/service/local/staging/deploy/maven2/</url>
	</repository>
</distributionManagement>

The configuration for the Maven release plugin should include disabling the release profile that is part of the Maven Super POM, since we are using our own profile, and specify the deploy goal together with the activation of our release profile

<pluginManagement>
          <plugins>
                <plugin>
                     <groupId>org.apache.maven.plugins</groupId>
                     <artifactId>maven-release-plugin</artifactId>
                     <version>2.5</version>
                     <configuration>
                          <useReleaseProfile>false</useReleaseProfile>
                          <releaseProfiles>release</releaseProfiles>
                          <goals>deploy</goals>
                     </configuration>
                </plugin>
          </plugins>
</pluginManagement>

Add the nexus-staging plugin to define where's Nexus and if we release it right after closing.

<plugin>
	<groupId>org.sonatype.plugins</groupId>
	<artifactId>nexus-staging-maven-plugin</artifactId>
	<version>1.6.3</version>
	<extensions>true</extensions>
	<configuration>
		<serverId>ossrh</serverId>
		<nexusUrl>https://oss.sonatype.org/</nexusUrl>
		<autoReleaseAfterClose>true</autoReleaseAfterClose>
	</configuration>
</plugin>

Add the release profile for signing with GPG & Javadoc and Sources Attachments

<profile>
	<id>release</id>
	<build>
		<plugins>
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-source-plugin</artifactId>
				<version>2.2.1</version>
				<executions>
					<execution>
						<id>attach-sources</id>
						<goals>
							<goal>jar-no-fork</goal>
						</goals>
					</execution>
				</executions>
			</plugin>
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-javadoc-plugin</artifactId>
				<version>2.9.1</version>
				<executions>
					<execution>
						<id>attach-javadocs</id>
						<goals>
							<goal>jar</goal>
						</goals>
					</execution>
				</executions>
			</plugin>
			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-gpg-plugin</artifactId>
				<version>1.5</version>
				<executions>
					<execution>
						<id>sign-artifacts</id>
						<phase>verify</phase>
						<goals>
							<goal>sign</goal>
						</goals>
					</execution>
				</executions>
			</plugin>
		</plugins>
	</build>
</profile>

You can see the full pom.xml e.g from my markdown-page-generator-plugin's pom.xml.

Publishing Snapshots

Snapshot deployment are performed when your version ends in -SNAPSHOT. You do not need to fulfil the requirements when performing snapshot deployments and can simply run

$ mvn clean deploy

Successfully deployed SNAPSHOT versions will be found in Snapshot repository. If you need to delete your artifacts, you can log in to Sonatype's Nexus using the same credentials you use to access to the Sonatype JIRA.

Performing a Release Deployment

The Maven Release Plugin can be used to automate the changes to the Maven POM files, sanity checks, the SCM operations required, the actual deployment execution and you can perform a release deployment to OSSRH with the following steps.

Prepare the release by answering the prompts for versions and tags

$ mvn release:clean release:prepare

If prepare fails use

$ mvn release:rollback
$ mvn release:clean

The prepare will create a new tag in SCM, even in GitHub, automatically checking in on your behalf. For it to work you need to have working public key in GitHub for git-push.

Perform the release. The perform process will ask for your gpg.passphrase if you don't give it with passphrase argument.

$ mvn release:perform

For some reason I can't provide the GPG passphrase when Maven GPG plugin asks it and I have to use the -Dgpg.passphrase argument.

$ mvn -Darguments="-Dgpg.passphrase=&lt;your GPG key passphrase&gt; release:perform

If release:perform fails to error "gpg: signing failed: No pinentry" then on macOS you need to restart gpg-agent and define pinentry-program. gnupg21 comes with pineentry as dependency. Or see Gist about commit signing with GPG on macOS.

$ killall gpg-agent && gpg-agent --daemon --use-standard-socket --pinentry-program /usr/local/bin/pinentry

This execution will deploy to OSSRH and release to the Central Repository in one go, thanks to the usage of the Nexus Staging Maven Plugin with autoReleaseAfterClose set to true.

Now your artifacts are in the Releases repository. The updates to The Central Repository search can take up to two hours. Once your artifacts are released you will not be allowed to update or delete them.

The first time you promote a release, you need to comment on the OSSRH JIRA ticket you created so OSSRH can know you are ready to be synced.

Manually Releasing the Deployment to the Central Repository

If you don't want to release artifacts automatically you can also do it manually. Set the autoReleaseAfterClose set to false so you can inspect and potentially release the deployed artifacts manually. Then to release your artifacts is done through Nexus as described in the guide.

Using Bintray for pushing artifacts to Maven Central

"Bintray offers developers the fastest way to publish and consume OSS software releases. Whether you are distributing software packages or downloading ones". In short it provides i.a. an alternative way to release artifacts to Maven Central and to Sonatype Open Source Software Repository.

I found out about Bintray after I had already used the Sonatype's way to release artifacts so I haven't tested this myself. Bintray's process is more or less similar than Sonatype's but if I hadn't read the blog post about how to do it I wouldn't have know where to start. In short the process is the following.

  1. Register to Bintray and set up auto-signing: Generate yourself a keypair, if you don’t have one. Add it to your profile, and setup your default Maven repo (or a new one) for signing with your GPG key: Bintray can then sign your jars automatically.
  2. Add your Sonatype account under "accounts". If you don’t have one, follow this procedure
  3. Create and link your package: Import from a GitHub repo or create a new package for your Maven project.
  4. You can link your package to JCenter by clicking "Add to JCenter".
  5. Set up Maven up to deploy to Bintray by copy-pasting the pom.xml snippets from "Set me up!" guide.

Then for each release:

  1. Deploy: Deploy files by running your build tool.
  2. Publish: Review the build artifacts and publish the version files if satisfied.
  3. Sync: On the version page go to the Maven Central tab, enter your Sonatype password and click "Sync" and you’re done!
  4. Your package is now in https://oss.sonatype.org/content/repositories/releases and will be synced to Maven Central (usually takes time). In case of a sync problem, Bintray will automatically take care of any needed cleanup.

Summary

For distributing your open source projects' artifacts through Maven Central repository and OSSRH you have two options: 1. Release with Sonatype's process or 2. Use Bintray's process. Both options end up with the same results: your project's artifacts are in Maven Central repository. In my opinion if you're using Maven then it doesn't really matter which option you choose.

Before Sonatype simplified their process for making releases to OSSRH the Bintray's process was the only almost pain-free gateway to Maven Central. But with the new Sonatype way the two alternative ways are more or less the same and I could say that Sonatype's new process feels a bit easier when using the Maven release plugin. In practice there's not much differences, e.g. both supports SNAPSHOTs, Sonatype in OSSRH and Bintray in oss.jfrog.org. Bintray is more flexible especially if not using Maven so if you're using other tools like Gradle then it's the easier way to go.

Monitor and profile application with Java Mission Control

Monitoring Java applications is can be done with different tools and with JDK you get one good tool for it: Java Mission Control. Java Mission Control and Java Flight Recorder together create a complete tool chain to continuously collect low level and detailed runtime information enabling after-the-fact incident analysis. Starting with Oracle JDK 7 Update 40 (7u40) Java Mission Control (JMC) bundled with the HotSpot JVM. Let's take a short look what those tools are.

"Java Flight Recorder is a profiling and event collection framework built into the Oracle JDK. It allows Java administrators and developers to gather detailed low level information about how the Java Virtual Machine (JVM) and the Java application are behaving. Java Mission Control is an advanced set of tools that enables efficient and detailed analysis of the extensive of data collected by Java Flight Recorder. The tool chain enables developers and administrators to collect and analyze data from Java applications running locally or deployed in production environments." - Oracle Java Mission Control.

Java Mission Control and Java Flight Recorder are commercial features, which are available as part of the Oracle Java SE Advanced product. They are freely available for download for development and evaluation purposes, as per the terms in the Oracle Binary Code License Agreement, but require an Oracle Java SE Advanced license for production use.

Using Java Mission Control

Oracle Java Mission Control is a tool suite for managing, monitoring, profiling, and troubleshooting your Java applications and it consists of the JMX Console and the Java Flight Recorder. To get a good overview how you can use Java Mission Control check Java Mission Control demo video. The JMC Client is built to run on Eclipse and it's based on the features of the old JRockit Mission Control.

The JMX Console enables you to monitor and manage your Java application and the JVM at runtime but the main and most important feature is the Flight Recorder. Java Flight Recorder (JFR) records the behavior of the JVM at runtime and you can analyze the recording offline using the Java Flight Recorder tool. They say that the overall profiling overhead for your applications stays at less than 2%, usually much less.

Starting with Oracle JDK 7 Update 40 (7u40) it's bundled with the HotSpot JVM and although you can connect it to older JDK's like application running on JDK 6 the newer ones show more information and have more features. So no real fun with legacy applications. The Flight Recorder needs at least JDK 7 Update 40.

Start JMC from the Windows command prompt:
"c:\Program Files\Java\jdk1.8.0_20\bin\jmc.exe"

Java Mission Control can be connected to local or remote Java Application. Start your application with following Virtual Machine flags which enables the JMX remote without authentication and Mission Control:

-Dcom.sun.management.jmxremote 
-Dcom.sun.management.jmxremote.port=8999 
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false 
-XX:+UnlockCommercialFeatures 
-XX:+FlightRecorder

If you're using WebLogic then the JMX Remote settings are following:

-Djavax.management.builder.initial=weblogic.management.jmx.mbeanserver.WLSMBeanServerBuilder
-Dcom.sun.management.jmxremote.port=8999 -Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false 
-XX:+UnlockCommercialFeatures 
-XX:+FlightRecorder

JMC does not include or run with a security manager, so a user must ensure to run the JMC client in a secure environment.

After connecting JMC with your Java application it opens a familiar Eclipse based user interface. JMX Console has couple of tabs on the bottom which shows overview, MBeans, Memory and Thread information.

JMC: JMX Console Overview

JMC: JMX Console MBeans
JMC: JMX Console Memory

JMC: JMX Console Threads

The more useful tool is the Java Flight Recorder (JFR) for profiling your application. In Java Mission Control JVM Browser right click on the Java Virtual Machine you wish to start a Flight Recording.

Leave all the default settings and select the "Profiling - on server" template for your Event Settings. Just hit finish at this point. You can also click Next to go to the higher level event settings which are groupings of named settings in the template. You can select how often you want JFR to sample methods by changing the Method Sampling setting.

The recording will be downloaded automatically and displayed in Mission Control. Click the tab group for Code to start visualizing your Method Profiling Sample events. Switch to the method profiling tab to find a top list of the hottest methods in your application.

Too bad I don't have nice recording to show but here's couple of screenshots. Better overview of how to use Flight Recording can be seen from the Java Mission Control demo video.

JMC: Flight Recording

JMC: Flight Recording

Sailfish OS User Interface design practices

The operating system running on Jolla has different and refreshing approach to user experience than the mainstream mobile operating systems. Sailfish OS is quite new project which also shows in applications' user interfaces as common practices are not quite established. There are some guidelines and component usage examples how the applications should be built but they don't cover every aspect especially with complex applications. Here are some examples how different apps solve common design issues in Sailfish OS.

Sailfish OS user interface design practices

Sailfish OS UI design practices

Sailfish OS guides you to make your app to work in certain ways and component gallery in development environment shows some practices but at the end how the app's flow and user experience really works is up to the developer. There really isn’t comprehensive guide how elements and menu structures should work like there’s for Windows Phone and iOS. And thus Sailfish OS applications' look and feel are sometimes quite different.

Here are some examples how different user interface aspects are designed in couple of Jolla's own and third party developers' apps. The presented examples and comments are just to point out different ways to build the user interface in Sailfish OS applications and doesn't take a stance on which way is better or how it should be done.

Menu for different sections

There are couple of practices to present a menu for app's different pages. The default is to use the pull down and push up menus but with more complex apps it's not enough as the recommendation is to have only couple of items in that menu.

Tweetian (Twitter) uses bottom tab panel and you can flick left or right or press to change the page. In HalfTrail (map + GPS app) different sections are also as icons in bottom tab panel but activate only when pressed. In Haikala (high.fi news) settings page's can be flicked or pressed with textual buttons.

Tab panel for pages
Action buttons on bottom panel
Bottom tab panel

Haikala uses side panel for actions to switch category. IRC uses the side panel for actions to select different IRC channels. Previously also Sailimgur (imgur) used left side panel for navigating to different pages.

Side panel for pages
Side panel for info & actions
Side panel for info & actions

Cargo Dock (file manager) is based on having two different sides and you can copy files between them. Both sides are identical but separate pages. Jolla Store shows links to apps in grid view and also arrows for selecting certain category of apps.

Two sides to work with
Arrows for sections
Grid view for selecting items

Settings menu

Settings are usually shown in separate page like in Jolla's Settings and Hunger Meter (battery usage). Terminal app on the other hand uses hamburger menu which opens side panel.

Settings page
Settings in own page
Hamburger-menu settings

Showing controls and actions

Jolla's Media app uses docked panel to show controls. The panel is shown only when a song is playing. You can also have docked panel with pull up menu for showing more controls icons. Tweetian has different actions in pull up and pull down menus.

Controls in docked panel
Docked panel with pull up menu
Actions in Pull up menu

Other common way is to show controls in bottom or top panel with tool icons. This can be seen in HalfTrail with action buttons in bottom toolbar. In Paint there's a top toolbar with icons. Neither provide legend for icons although Paint has help in About page. MitaKuuluu (WhatsApp) also uses bottom toolbar for chat related actions. In Sailimgur you can choose to have to toolbar at the bottom or at the top. The bottom toolbar has better reachability if used with one hand.

Action buttons on bottom panel
Tools in top panel
Actions in bottom toolbar
Toolbar at the bottom
Top toolbar

Jolla Settings app uses switches to toggle different features. Terminal app on the contrary takes quite different approach to showing actions and settings. Terminal uses so called Hamburger menu which works quite nicely although isn't "Sailfish OS" style.

Actions as switches
Hamburger-menu
Hamburger-menu settings

Sailimgur shows actions for the photo when long pressed in a context menu. Actions related to a comment are shown as bottom context menu or icon buttons. Quickddit (Reddit) shows actions as icon buttons.

Actions in context menu
Actions in context menu
Actions as icon buttons

Back navigation

For the back navigation there isn't a physical button and you either flick left, click page stack icon or use some button and action in pull down menu.

In Jolla's Gallery app you scroll through photos in slideshow view and you get actions for back navigation only when pressing the photo and opening the Docked panel where you can flick back to the gallery overview. Alternative way for back navigation from photo slideshow is just press the photo as seen in e.g. Jolla Store.

Jolla Opas (Reittiopas) uses simple icon button in toolbar for navigating the route on the map and navigating back. And of course you can have the back navigation in pull down menu like in Sailimgur when navigating back from Web view.

Back navigation in docked panel
Back navigation with button
Closing view in pull down menu

Navigating between items

Navigating between items in a list is normally done by flicking left or right. But sometimes that's not usable or it's better to provide also alternative way.

Sailimgur has previous and next buttons at the bottom, Tidings (RSS feeds) provides them in pull down and pull up menus and Browser has them in bottom tool bar as icons.

Next & Previous as buttons
Next in pull down menu
Prev/next in toolbar

Search

Providing search is usually done as a separate page but it can be put on pull down menu or as normal element in page.

Warehouse (OpenRepos) application has search link in pull down menu which navigates you to separate search page which also shows the results. Sailimgur has the search element in pull down menu and it closes after user presses enter to show the results in main view. Jolla Opas is a route planning app so the search elements are in main page. The search results are shown (if found more than one destination) as blue circle with results number and can be selected in separate dialog page.

Search in own page
Search in menu
Search as main function
Inline search in main view

Showing gallery of images

Showing list of images is pretty simple but there are differences how applications do it. Gallery application uses one big grid of cropped thumbnails, Warehouse shows full thumbnail and selection list in side and Jolla Store shows limited list of cropped thumbnails which opens in own view.

Only thumbnails, press to open own view
Grid view of images
Large image, side preview list

Notifications about new features

Mobile applications update quite often but users don't usually check the changelog what's changed or if there's new features. So it's nice to provide a simple changelog in first start like Flashlight, Paketti (post packages) and Haikala.

New features notification
Showing changelog
Changelog dialog

Using the WebLogic 12c Maven Plug-In for Deployment

Using the WebLogic 12c 12.1.2 Maven plug-In for deployment is much easier and quicker than going through the WebLogic Server’s AdminServer and Oracle Documentation provides good examples how to do it. The weblogic-maven-plugin provides enhanced functionality to install, start and stop servers, create domains, execute WLST scripts, and compile and deploy applications.

The weblogic-maven-plugin plug-in is provided as a pre-built JAR file and accompanying pom file. In short, installing and configuring the WebLogic 12c Maven sync plug-in contains following steps:

Open command prompt and run commands:

cd D:\oracle\wls12120\oracle_common\plugins\maven\com\oracle\maven\oracle-maven-sync\12.1.2
mvn install:install-file -DpomFile=oracle-maven-sync.12.1.2.pom -Dfile=oracle-maven-sync.12.1.2.jar
mvn com.oracle.maven:oracle-maven-sync:push -Doracle-maven-sync.oracleHome=D:\oracle\wls12120

You can validate whether you have successfully installed the plug-in using the Maven help:describe goal.

mvn help:describe -DgroupId=com.oracle.weblogic -DartifactId=weblogic-maven-plugin -Dversion=12.1.2-0-0

And you're ready to deploy with Maven. A lot more easier than generating the WebLogic 11g R2 Maven Plug-in where you had to i.a. build the jar by yourself.

The Maven plug-in can be used e.g. from application's POM file and be bound to some phase of the Maven life cycle. For example it can be bound to "install" phase and every time you run the "mvn install" command, the deployment plug-in is also called. In my opinion better way to use the plug-in is to add it as a profile so you can call it just when you want with command like "mvn clean install -Pdeploy-wls".

Maven Project pom.xml File

<profile>
	<id>deploy-wls</id>
	<build>
		<plugins> 
			<plugin> 
				<groupId>com.oracle.weblogic</groupId>
				<artifactId>weblogic-maven-plugin</artifactId> 
				<version>12.1.2-0-0</version> 
				<configuration> 
					<adminurl>t3://localhost:7001</adminurl>
					<user>weblogic</user> 
					<password>weblogic123</password> 
					<upload>true</upload> 
					<targets>myServer</targets>
					<action>deploy</action> 
					<remote>false</remote> 
					<verbose>true</verbose> 
<source>${project.build.directory}/${project.build.finalName}.${project.packaging}</source> 
					<name>${project.build.finalName}</name> 
				</configuration> 
				<executions> 
					<execution> 
						<phase>install</phase> 
						<goals> 
							<goal>deploy</goal> 
						</goals> 
					</execution> 
				</executions> 
			</plugin> 
		</plugins> 
	</build>
</profile>

The user credentials in the POM file are provided as clear-text but for more security you can use secure configuration authentication mechanism which stores the user name and password in encrypted form in an external file, and then uses it to supply the user credentials with which to connect to the WebLogic Server domain, along with the key that was used to encrypt the file.

Maven plugin to generate HTML from Markdown now in Maven Central

Generating documents is always not the most pleasant job but it can be made easier by using easier markup language like Markdown. For that purpose I made last year a simple Markdown to HTML Page Generator Maven Plugin and now it's also available in Maven Central.

"Markdown to HTML Page Generator" is a simple Maven Plugin which reads the input directory for Markdown files with .md suffix and outputs HTML files to target directory. There are couple of configuration options to set the input/output directories and if you want to add header and footer files to make it structurally valid HTML. The plugin uses pegdown Markdown processor.

Maven plugin artifacts can be found in Sonatypes' OSS Maven repository:

Read more about the plugin and how to use it from markdown-page-generator-plugin's GitHub page.

Debugging Sailfish OS application power consumption

Lately my simple news feed reader, Ampiaiskala, got rejected from Jolla Store in Harbour Quality Assurance as it seemed that it was too power hungry while minimized. I was a bit surprised as 0.1 version was accepted and the functionality hadn't changed. So, it was time to get familiarized with PowerTOP and how to debug problems with power consumption.

First we need to install PowerTOP to Jolla and then just ran it via SSH over USB or WLAN while the app is minimized and screen is closed. After that it's about what all those numbers mean and where they come from.

Installing PowerTOP on Jolla

Steps to install PowerTOP from command line:

1. Add mer-tools repository:

$ ssu ar mer-tools

2. Refresh repository cache:

$ pkcon refresh

3. Install tool:

$ pkcon install powertop

4. Run tool:

$ devel-su powertop

Debugging power consumption

PowerTOP reports how much power each process uses and provides a list of all running processes which are waking up and how many times in a second (how many times each one sends a CPU interrupt). Only one process is monitored so it doesn't tell all about the system, e.g. when the process has interaction with audio or network.

After installing PowerTOP tool, usage is easy via SSH over USB or WLAN connection. Difference between these connections is that device does not suspend with PC USB connection enabled and with WLAN device goes to suspend if there's not any network traffic and display is turned off.

Connect to your phone e.g. with ssh nemo@192.168.1.58 and ran the PowerTOP as root: devel-su powertop. The command prompts you for the root password which you have given in Settings > System > Developer mode > SSH and root password.

Before starting to debug your app's power consumption it's useful to create a baseline by running PowerTOP after a cold startup, without opening any applications, and then run it a few more times throughout the day to get a comparison of different workloads.

Generate HTML page of PowerTOP results for baseline:

$ devel-su powertop --html=<filename>

After getting some baseline results it's just reading the numbers and figuring out if your application eats power or not. And if it is power hungry beast then figuring out why it does that.

Complementary to PowerTOP, there's Strace which lets you find out how a program is interacting with the operating system by monitoring system calls and signals. Just install it with $ pkcon install strace and ran it with strace -p or strace -ttT -p . It shows what your application does while it's expected to be idle. For example:

$ ps aux | grep ampiaiskala
$ devel-su strace -p 1301

The result looks something like this:

Don't ask me what all that means :)

Power consumption and Harbour QA testing

The reason I'm debugging my application for power usage is Jolla Harbour Quality Assurance which rejected my app with "It seems that app doesn't go to idle state when device does". The Quality Assurance procedure for power consumption is explained in Together being the following.

"Jolla's Harbour QA testing is mostly done with PC USB cable connection, application minimized to home screen and display is turned off. In this state application should not block suspend and wake ups should be minimal (< ~60, this is a reference value). A good practice is to take a "system state" before opening application and compare those results when application is minimized to home screen." "The phone suspend is blocked by wake locks and easy test for that is catting /sys/power/wake_lock file. The file should be empty normally if nothing from user space is not blocking suspend. If application is using e.g. audio, then file should include "mce_cpu_keepalive" string. Of course applications are allowed to do things in the background but that should be minimal for its own functionality. E.g are internet streaming."

After another Harbour QA rejection (2014-06-05) I asked in IRC from Jolla Sailor, nazanin, about the power consumption issue and what steps they are using so I could repeat them. The steps are the following:

  • First there are no apps open, lock the device, run powertop command
  • Wait for about 1 minute until the device status becomes stable
  • Unlock it, open the application you want to test, play around with it for a bit, until the number of wake ups reaches 300 or 400
  • Check if you already see the app in wakeup list in this status
  • Lock the device and observe how powertop updates
  • If after 1 minute or 2, the application drops down from the list, it's okay
  • If it doesn't, then wait more, for about 5-10 minutes
  • If you see the application is still there, that is when you can assume the test case is failed and application is causing power consumption

Possible reasons for excess power consumption

The procedure to figure out if your application is consuming power more than it should is relatively easy but finding out why it does that is a bit harder. Things like timers, animations, WebViews and using animated images are possible culprits and how the app behaves with none or bad network connectivity. I don't know there to be any good tips how to debug these issues other than going through your app, knowing what it does and figuring out possible issues.

My own cases for Harbour QA rejections due power consumption have been with using SilicaWebview (QtWebKit) and keeping it open when the app is minimized and thus eating too much CPU and preventing the device go to sleep. The solution would be that you suspend the WebKit rendering or close the WebView. But that's easier said than done. After discussing this issue in IRC with couple of Sailors (nazanin, kontio) for now there's no good solution to suspend the WebKit as it doesn't provide an easy API to suspend rendering. If it's reasonable for user experience you can pop the WebView page when app is minimized. You can bind the action e.g. to CoverPage and use onStatusChanged signal and check for "status == PageStatus.Activating.

I came across this WebKit issue with Haikala and Ampiaiskala as they open WebKit view for reading the news feed item's web page. For those I modified my code to allow the user to close the web view from the pull down menu when user goes back to the feed list. After that they passed the Harbour QA. Couple of months ago there was a similar case on SailfishDevel mailing list.

With my imgur app, Sailimgur, it was again using WebKit for viewing imgur's gallery page but also there's animated images running while showing the gallery or image page in the app. The app was seen under wake-ups when it was showing an animated image like .gif. If the app wasn't showing some funny gifs from imgur it dropped down from the list after a while as it should. Understandable effect as the UI is running the gif which consumes power and prevents the device go to sleep.

The partial solution is to manually stop the animated images when the app is minimized but in my opinion that should be something that Sailfish OS should take care automatically. The other thing is the animated image running in the SilicaWebView where QtWebKit keeps running in the background and doesn't suspend automatically. You could send SIGSTOP to suspend it when minimized and then continue with SIGCONT from C++ but it's a bit hassle as you would need a monitoring process to manage that. And it's not known if that works or not.

In the end after considering the options the Sailors said that they accept my app in the current state, since WebKit doesn't provide an easy API to suspend rendering. But have to think about what I could do to limit wake-ups.

Maven plugin to generate HTML from Markdown

Writing documentation or user guide isn't the most interesting part of the software project but it's still needed to be done and important for the end users. There are many ways to achieve documentation but using Markdown is good way to make it easier. After that you just need to convert it to HTML and as I couldn't find a suitable Maven plugin to do that nicely I wrote one myself: markdown-page-generator-plugin.

Software developers are a bit lazy to write good and up to date documentation and as I needed to write the API documentation for the web application as separate HTML pages I got to know the issue first hand. I like writing documents but typing the API documentation as static HTML wasn't very effective or fun. Fortunately it was quite easy to do it with Markdown which allows you to write documentation using an easy-to-read, easy-to-write plain text format which you can then convert to HTML. After that the only problem was to enable easy HTML generation directly from Maven Lifecycle and thus was the "Markdown to HTML Page Generator Maven Plugin" developed.

Markdown to HTML Page Generator Maven Plugin

"Markdown to HTML Page Generator" is a simple Maven Plugin which reads the input directory for Markdown files with .md suffix and outputs HTML files to target directory. There are couple of configuration options to set the input/output directories and if you want to add header and footer files to make it structurally valid HTML. The plugin uses pegdown Markdown processor.

Source code

The plugin code can be found from GitHub: markdown-page-generator-plugin.

Usage

Check out the plugin's GitHub page for configuration options.

Using the plugin could be easier but as it isn't in Maven repository you have to e.g. add it as Maven project to same Eclipse workspace as your project and build it. After that add it as a plugin to your build and it's run automatically when processing sources. You can also run it from command line with "mvn com.ruleoftech:markdown-page-generator-plugin:generate".

Setting up LAMP stack on OS X

Setting up LAMP stack for web development on OS X can be done with 3rd party software like MAMP but as Mac OS X comes with pre-installed Apache and PHP it's easy to use the native setup. You just need to configure Apache, PHP and install MySQL.

Setup Apache2

Set up the Server Name to localhost to suppress the warning about fully qualified domain name and enable PHP module.

$ sudo vim /etc/apache2/httpd.conf
ServerName localhost:80
LoadModule php5_module libexec/apache2/libphp5.so

Create "virtual hosts" under your Sites. Change the username to your account's username.

~$ sudo vim /etc/apache2/users/username.conf
<VirtualHost *:80>
  ServerName dev
  DocumentRoot /Users/username/Sites
  VirtualDocumentRoot /Users/username/Sites/%-2/htdocs
  UseCanonicalName Off
 
  <Directory "/Users/username/Sites/*/htdocs">
    AllowOverride All
    Order allow,deny
    Allow from all
  </Directory>
</VirtualHost>

Now Apache serves your projects from your home directory's Sites folder. Apache will serve files from the htdocs folder like "~/Sites/projectname/htdocs".

Now just restart Apache and check that it's running.

$ sudo apachectl restart
$ ps aux | grep httpd

Setup PHP

$ sudo cp /etc/php.ini.default /etc/php.ini

Edit php.ini for easier debugging:

error_reporting  =  E_ALL | E_STRICT
display_errors = On
html_errors = On

Setup MySQL

MySQL can be installed directly from Oracle's MySQL packages or by using Homebrew.

Install Homebrew

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Install MySQL using Homebrew

$ brew install mysql

Install the MySQL system tables and have it run as your system user:

$ unset TMPDIR
$ mysql_install_db --verbose --user=`whoami` --basedir="$(brew --prefix mysql)" --datadir=/usr/local/var/mysql --tmpdir=/tmp

Start MySQL and check that it's running

$ mysql.server start
$ ps aux | grep mysql

Reset the root password. Change the "5.5.27" to your installed version number.

$ /usr/local/Cellar/mysql/5.5.27/bin/mysqladmin -u root password 'YOUR_NEW_PASSWORD'

As we are using the Homebrew package for MySQL and the default php.ini file then PHP is trying to connect to MySQL through the default_socket at /var/mysql/mysql.sock which doesn't exist as MySQL is using /tmp/mysql.sock. Just change all instances of /var/mysql/mysql.sock to /tmp/mysql.sock.

$ sudo sed -i "" "s:/var/mysql/mysql.sock:/tmp/mysql.sock:g" /etc/php.ini

And you're done.