Weekly notes 10

Summer has been relative nice this far even here in Finland and my short holiday is just couple of days away. But before that it’s time to check this years Java tools and technologies landscape report, get some useful plugins for Atom, start developing a React application with no configuration and read about the benefits of Serverless architecture. And while traveling it’s good to listen to podcasts for developers.

Weekly notes, issue 25.7.2016


Create Apps with No Configuration
Developing a React app has lots of things to setup so using Create React App, officially supported way to create single-page React application, as a boilerplate generator is good choice. And with single command, and all the build dependencies, configs, and scripts are moved right into your project so you’re not lock-in.

A Better File Structure For React/Redux Applications
Something to think about how you organize your React code. Similar to how you could organize things with Java application.

Tools of the trade

Java Tools and Technologies Landscape Report 2016
ZeroTurnaround has just released its Java Tools and Technologies Landscape Report 2016, which analyzes the data about the tools and technologies Java developers use. Good to note that the survey received just over 2000 responses.

Atom treasures: a list of Atom plugins I can’t live without
AtomEditor is great for developers and better when extended with plugins. Found some new ones, like sync-settings. (from The Practical Dev)

New DevTools Web Performance Tooling Tips and Features (video)
Chrome’s DevTools is powerful but not always so easy to utilize. Paul Irish and Sam Saccone show off new tips, tricks and features in DevTools to help you debug the performance of your site.
(from HTML5 Weekly Issue 241)

Bash boilerplates
When hacking up Bash scripts, there are often things such as logging or command-line argument parsing that: You need every time, Come with a number of pitfalls you want to avoid, Keep you from your actual work. Here’s an attempt to bundle those things in a generalized way so that they are reusable as-is in most scripts.

Learning new things

79 Podcasts for Developers, Programmers & Software Engineers
Podcasts are incredibly useful for staying on top of all the latest happenings in software development.


Benefits and drawbacks of Serverless architecture
Serverless architectures refer to applications that significantly depend on third-party services. But what are the benefits and drawbacks to such a way of designing and deploying applications. (from Java Web Weekly 133)

Something different

Cheating at Pokemon Go with a Hackrf and GPS spoofing
Pokemon Go has taken the world with enthusiasm and it requires you to walk around and explore the city for Pokestops, Gyms and hatching eggs. But why do that if you can cheat? Since the game is GPS based with little tinkering you can spoof your GPS location using a HackRF software defined radio and simulate walking around.

Expanding your horizons on JVM with Kotlin

The power of Java ecosystem lies in the Java Virtual Machine (JVM) which runs variety of programming languages which are better suitable for some tasks than Java. One relatively new JVM language is Kotlin which is statically typed programming language that targets the JVM and JavaScript. You can use it with Java, Android and the browser and it’s 100% interoperable with Java. Kotlin is open source (Apache 2 License) and developed by a team at JetBrains. The name comes from the Kotlin Island, near St. Petersburg. The first officially considered stable release of Kotlin v1.0 was released on February 15, 2016.

Why Kotlin?

“Kotlin is designed to be an industrial-strength object-oriented language, and to be a better language than Java but still be fully interoperable with Java code, allowing companies to make a gradual migration from Java to Kotlin.” – Kotlin, Wikipedia

Kotlin’s page summaries the question “Why Kotlin?” to:

  • Concise: Reduce the amount of boilerplate code you need to write.
  • Safe: Avoid entire classes of errors such as null pointer exceptions.
  • Versatile: Build server-side applications, Android apps or frontend code running in the browser. You can write code in Kotlin and target JavaScript to run on Node.js or in browser.
  • Interoperable: Leverage existing frameworks and libraries of the JVM with 100% Java Interoperability.

“You can write code that’s more expressive and more concise than even a scripting language, but with way fewer bugs and with way better performance.” – Why Kotlin is my next programming language

One of the obvious applications of Kotlin is Android development as the platform uses Java 6 although it can use most of Java 7 and some backported Java 8 features. Only the recent Android N which changes to use OpenJDK introduces support for Java 8 language features.

For Java developers one significant feature in Kotlin is Higher-Order Functions, function that takes functions as parameters, which makes functional programming more convenient than in Java. But in general, I’m not so sure if using Kotlin compared to Java 8 is as much beneficial. It smooths off a lot of Java’s rough edges, makes code leaner and costs nothing to adopt (other than using IntelliJ IDEA) so it’s at least worth trying. But if you’re stuck with legacy code and can’t upgrade from Java 6, I would jump right in.

Learning Kotlin

Coming from Java background Kotlin at first glance looks a lot leaner, elegant, simpler and the syntax is familiar if you’ve written Swift. To get to know the language it’s useful to do some Kotlin Examples and Koans which get you through how it works. They also have “Convert from Java” tool which is useful to see how Java classes translate to Kotlin. For mode detailed information you can read the complete reference to the Kotlin language and the standard library.

If you compare Kotlin to Java you see that null references are controlled by the type system, there’s no raw types, arrays are invariant (can’t assign an Array to an Array) and there’s no checked exceptions. Also semicolons are not required, there’s no static members, non-private fields or wildcard types.

And what Kotlin has that Java doesn’t have? For starters there’s null safety, smart casts, extension functions and lots of things Java just got in recent versions like Null safety, streams, lambdas ( although which are “expensive”). On the other hand Kotlin targets Java 6 bytecode and doesn’t use some of the improvements in Java 8 like invoke-dynamic or lambda support. Some of JDK7/8 features are going to be included in Standard Library in 1.1 and in the mean time you can use small kotlinx-support library. It provides extension and top-level functions to use JDK7/JDK8 features such as calling default methods of collection interfaces and use extension for AutoCloseable.

And you can also call Java code from Kotlin which makes it easier to write it alongside Java if you want to utilize it in existing project and write some part of your codebase with Kotlin.

The Kotlin Discuss is also nice forum to read experiences of using Kotlin.

Tooling: in practice IntelliJ IDEA

You can use simple text editors and compile your code from the command line or use build tools such as Ant, Gradle and Maven but good IDEs make the development more convenient. In practice, using Kotlin is easiest with JetBrains IntelliJ IDEA and you can use their open source Community edition for free. There’s also Eclipse plugin for Kotlin but naturally it’s much less sophisticated than the IntelliJ support.

Example project

The simplest way to start with Kotlin application is to use Spring Boot’s project generator, add your dependencies, choose Gradle or Maven and click on “Generate Project”.

There are some gotchas with using Spring and Kotling together which can be seen from Spring + Kotlin FAQ. For example by default, classes are final and you have to mark them as “open” if you want the standard Java behaviour. This is useful to know with @Configuration classes and @Bean methods. There’s also Kotlin Primavera which is a set of libraries to support Spring portfolio projects.

For example Spring Boot + Kotlin application you should look at Spring.io writeup where they do a geospatial messenger with Kotlin, Spring Boot and PostgreSQL

What does Kotlin look like compared to Java?
Simple example of using Java 6, Java 8 and Kotlin to filter a Map and return a String. Notice that Kotlin and Java 8 are quite similar.

# Java 6
String result = "";
for (Map.Entry<Integer, String> entry : someMap.entrySet()) {
		result += entry.getValue();
# Java 8
String result = someMap.entrySet().stream()
		.filter(map -> "something".equals(map.getValue()))
# Kotlin
val result = someMap
  .filter { it == "something" }
# Kotlin, shorter 
val str = "something"
val result = str.repeat(someMap.count { it.value == str })
# Kotlin, more efficient with large maps where only some matching.
val result = someMap
  .map { it.value }
  .filter { it == "something" }

The last Kotlin example makes the evaluation lazy by changing the map to sequence. In Kotlin collections map/filter methods aren’t lazy by default but create always a new collection. So if we call filter after values method then it’s not as efficient with large maps where only some elements are matching the predicate.

Using Java and Kotlin in same project

To start with Kotlin it’s easiest to mix it existing Java project and write some classes with Kotlin. Using Kotlin in Maven project is explained in the Reference and to compile mixed code applications Kotlin compiler should be invoked before Java compiler. In maven terms that means kotlin-maven-plugin should be run before maven-compiler-plugin.

Just add the kotlin and kotlin-maven-plugin to your pom.xml as following

            <goals> <goal>compile</goal> </goals>
            <goals> <goal>test-compile</goal> </goals>

Notes on testing

Almost everything is final in Kotlin by default (classes, methods, etc) which is good as it forces immutability, less bugs. In most cases you use interfaces which you can easily mock and in integration and functional tests you’re likely to use real classes, so even then final is not an obstacle. For using Mockito there’s Mockito-Kotlin library https://github.com/nhaarman/mockito-kotlin which provides helper functions.

You can also do better than just tests by using Spek which is a specification framework for Kotlin. It allows you to easily define specifications in a clear, understandable, human readable way.

There’s yet no static analyzers for Kotlin. Java has: FindBugs, PMD, Checkstyle, Sonarqube, Error Prone, FB infer. Kotlin has kotlinc and IntelliJ itself comes with static analysis engine called the Inspector. Findbugs works with Kotlin but detects some issues that are already covered by the programming language itself and are impossible in Kotlin.

To use Kotlin or not?

After writing some classes with Kotlin and testing converting existing Java classes to Kotlin it makes the code leaner and easier to read especially with data classes like DTOs. Less (boilerplate) code is better. You can call Java code from Kotlin and Kotlin code can be used from Java rather smoothly as well although there are some things to remember.

So, to use Kotlin or not? It looks a good statically-typed alternative to Java if you want to expand your horizons. It’s pragmatic evolution to Java that respects the need for good Java integration and doesn’t introduce anything that’s terribly hard to understand and includes a whole bunch of features you might like. The downsides what I’ve come across are that tooling support is kind of limited, meaning in practice only IntelliJ IDEA. Also documentation isn’t always up to date or updated when the language evolves and that’s also an issue when searching for examples and issues. But hey, everything is fun with Kotlin :)

Web analytics with Piwik: keeping control over your own data

Web analytics is one the essential tools for a website and including measuring web traffic and getting information about the number of visitors it can be also used as a tool to assess and improve the effectiveness of a website. The most common way to collect data is to use on-site web analytics, measure a visitor’s behavior once on your website, with page tagging technology like on Google Analytics which is widely used web analytics service. But what would you use if you want to keep control over your own data?

You don’t have to look far as the only open source web analytics application is Piwik which aims to be the ultimate open alternative to Google Analytics. Here’s a short overview to Piwik Analytics and how to get started with it.

“Web analytics is the measurement, collection, analysis and reporting of web data for purposes of understanding and optimizing web usage.” – Wikipedia

Piwik Open Analytics Platform

Piwik is web analytics application which tracks online visits to one or more websites and displays reports on these visits for analysis. In short it aims to be the ultimate open source alternative to Google Analytics. The code is GPL v3 licensed and available in GitHub. In technical side Piwik is written in PHP, uses MySQL database and you can host it by yourself. And if you don’t want to setup or host Piwik yourself you can also get commercial services.

Piwik provides the usual features you would expect from a web analytics application. You get reports regarding the geographic location of visits, the source of visits, the technical capabilities of visitors, what the visitors did and the time of visits. Piwik also provides features for analysis of the data it accumulates such as saving notes to data, goals for actions, transitions for seeing how visitors navigate, overlaying analytics data on top of a website and displaying how metrics change over time. The easiest way to see what it has to offer is to check the Piwik online demo.

Feature highlights

You might ask how Piwik differs from other web analytics applications such as Google Analytics? One principle advantage of using Piwik is that you are in control. You can host Piwik on your own server and the data is tracked inside your MySQL database: you’ve full control over your data. Software as a service analytics applications on the other hand, have full access to the data users collect. Data privacy is essential for public sector and enterprises who can’t or don’t want to share it for example with Google. You ensure that your visitors behavior on your website is not shared with advertising companies.

Other interesting feature is that it provides advanced privacy options: ability to anonymize IP addresses, purge tracking data regularly (but not report data), opt-out support and Do Not Track support. Your website visitors can decide if they want to be tracked.

You can also do scheduled reports which are sent by e-mail, import data from web server logs, use the API for accessing reports and administrative functions and Piwik also has mobile app to access the analytics data. Piwik is also customizable with plugins and you can integrate it with WordPress and other applications.

Piwik’s User Interface

Piwik has clean and simple user interface as seen in the following screenshots (taken from the online demo).

Piwik main view
Piwik main view

Piwik visitors overview
Piwik visitors overview

Setting up Piwik

Setting up Piwik is easy and there’s good documention available for running Piwik web analytics. All you need is web server like Nginx, PHP 5.5 and MySQL or MariaDB. You can setup it manually but the most easiest way to start with it is to use the provided Docker image and docker-compose. The docker-compose file setups four containers (MySQL, Piwik, Nginx and Cron) and with compose you can start it up. The Piwik image is available from official docker-library.

The alternative is to do your own Docker image for Piwik and related services. In my opinion it makes sense to have just two containers: one for Piwik related web stuff and other for MySQL. The Piwik container runs Piwik, Nginx and Cron script with e.g. supervisor. The official image uses Debian (from PHP) but Piwik runs nicely also on Alpine Linux. One thing to tinker with when using Docker is to get MySQL access to Piwik’s assets for LOAD DATA INFILE which will greatly speed Piwik’s archiving process.

If you’re setting up Piwik manually you can watch a video of installation and after that a video of configuring settings. After you’re done with the 5 minute installation you get the JavaScript tag which you add to the bottom of each page of your website. If you’re using React there’s Piwik analytics component for React Router. Piwik will then record the activity across your website within your database.

And that’s about all there is to starting with Piwik. Simple setup with Docker or doing it manually, adding the JavaScript tag, configuring some options if needed and then just wait for the data from visitors.


Piwik is good and feature rich alternative for web analytics application. Setting it up isn’t as straightforward as using some hosted service as Google Analytics but that’s the way self-hosted services always are. If you need web analytics and want to keep control of your own data and don’t mind hosting it yourseld and paying for the server then Piwik is a good choice.

Weekly notes 9

Summer is here and mountain biking trails are calling but keeping up with what happens in the field never stops. This week Apple had their worldwide developers conference which filled up social media although didn’t present anything remarkable. In the other news there was good collection of slides for Java developers, ebook for DevOps and HyperDev looks interesting for quickly bang out JavaScript.

Weekly notes, issue 9, 17.6.2016

Java: stay updated, reactive and in the cloud

13 Decks Java developers must see to stay updated
Selection of nice slideshows for Java developers. Best practices, microservices, debugging, Elasticsearch, SQL.

Java SE 8 best practices
Java 8 best practices by Stephen Colebourne’s is good read. The slides cover all the basic uses, such as lambdas, exceptions, streams and interfaces. (from the “13 Decks Java developers” post)

Microservices + Oracle: A Bright Future
Good slides of what are microservices. Considerations, prerequisites, patterns, technologies and Oracle’s plans. (from the “13 Decks Java developers” post)

Notes on Reactive Programming, Part I: The Reactive Landscape and Part II: Writing Some Code
A solid intro to the reactive programming. And no, it’s no coincidence that this is first. A reactive system is an entirely different beast, and such a good fit for a small set of scenarios. (from Java Web Weekly, Issue 128)

Netflix OSS, Spring Cloud, or Kubernetes? How About All of Them!
The Netflix ecosystem of tools is based on practical usage at scale, so it’s always super useful to go deep into understanding their tools. (from Java Web Weekly, Issue 128)

Takeouts from WWDC 2016

Digging into the dev documentation for APFS, Apple’s new file system

Interesting low level stuff in Mac OS Sierra. APFS takes over HFS+, has native encryption, snapshots (Time Machine done right) and is case-sensitive. Hacker News comments are worth reading.

The 13 biggest announcements from Apple WWDC 2016
WWDC 2016 was about software and incremental changes. Siri is opening up to app developers, iOS is growing up, iOS gets Apple TV remote app and Apple introduces single sign-on system.

Continuous learning

DevOpsSec: Securing Software through Continuous Delivery
DevOpsSec free ebook is worth reading if you’re interested securing software through continuous delivery. Uses case studies from Etsy, Netflix, and the London Multi-Asset Exchange to illustrate the steps leading organizations have taken to secure their DevOps processes.

Microservice Pitfalls & AntiPatterns, Part 1
An anti-pattern is just like a pattern, except that instead of a solution it gives something that looks superficially like a solution but isn’t one. A pitfall is something that was never a good idea, even from the start. (from The Microservice Weekly #31)

Tools of the trade

Introducing HyperDev
HyperDev looks to be an interesting new product at Fog Creek Software (known from e.g. Trello). It’s developer playground for building full-stack web-apps fast. “The fastest way to bang out JavaScript code on Node.js and get it running on the internet.” as Joel Spolsky describes it.

V8, modern JavaScript, and beyond – Google I/O 2016
Debugging Node.js apps with Chrome Developer Tools is soon enabled by coming v8_inspector support.

Something different

Why do we have allergies?
Allergies such as peanut allergy and hay fever make millions of us miserable, but scientists aren’t even sure why they exist.

Docker containers and using Alpine Linux for minimal base images

After using Docker for a while, you quickly realize that you spend a lot of time downloading or distributing images. This is not necessarily a bad thing for some but for others that scale their infrastructure are required to store a copy of every image that’s running on each Docker host. One solution to make your images lean is to use Alpine Linux which is a security-oriented, lightweight Linux distribution.

Lately I’ve been working with our Docker images for Java and Node.js microservices and when our stack consist of over twenty services, one thing to consider is how we build our docker images and what distributions to use. Building images upon Debian based distributions like Ubuntu works nicely but it gives packages and services which we don’t need. And that’s why developers are aiming to create the thinnest most usable image possible either by stripping conventional distributions, or using minimal distributions like Alpine Linux.

Choosing your Linux distribution

What’s a good choice of Linux distribution to be used with Docker containers? There was a good discussion in Hacker News about small Docker images, which had good points in the comment section to consider when choosing container operating system.

For some, size is a tiny concern, and far more important concerns are, for example:

  • All the packages in the base system are well maintained and updated with security fixes.
  • It’s still maintained a few years from now.
  • It handles all the special corner cases with Docker.

In the end the choice depends on your needs and how you want to run your services. Some like to use the quite large Phusion Ubuntu base image which is modified for Docker-friendliness, whereas others like to keep things simple and minimal with Alpine Linux.

Divide and conquer?

One question to ask yourself is: do you need full operating system? If you dump an OS in a container you are treating it like a lightweight virtual machine and that might be fine in some cases. If you however restrict it to exactly what you need and its runtime dependencies plus absolutely nothing more then suddenly it’s something else entirely – it’s process isolation, or better yet, it’s portable process isolation.

Other thing to think about is if you should combine multiple processes in single container. For example if you care about logging you shouldn’t use a logger daemon or logrotate in a container, but you probably want to store them externally – in a volume or mounted host directory. SSH server in container could be useful for diagnosing problems in production, but if you have to log in to a container running in production – you’re doing something wrong (and there’s docker exec anyways). And for cron, run it in a separate container and give access to the exact things your cronjob needs.

There are a couple of different schools of thought about how to use docker containers: as a way to distribute and run a single process, or as a lighter form of a virtual machine. It depends on what you’re doing with docker and how you manage your containers/applications. It makes sense to combine some services, but on the other hand you should still separate everything. It’s preferred to isolate every single process and explicitly telling it how to communicate with other processes. It’s sane from many perspectives: security, maintainability, flexibility and speed. But again, where you draw the line is almost always a personal, aesthetic choice. In my opinion it could make sense to combine nginx and php-fpm in a single container.

Minimal approach

Lately, there has been some movement towards minimal distributions like Alpine Linux, and it has got a lot of positive attention from the Docker community. Alpine Linux is a security-oriented, lightweight Linux distribution based on musl libc and busybox using a grsecurity/PaX patched Linux kernel and OpenRC as its init system. In its x86_64 ISO flavor, it weighs in at an 82MB and a container requires no more than 8 MB. Alpine provides a wealth of possible packages via its apk package manager. As it uses musl, you may run into some issues with environments expecting glibc-like behaviour (for example Kubernetes or with compiling some npm modules), but for most use cases it should work just fine. And with minimal base images it’s more convenient to divide your processes to many small containers.

Some advantages for using Alpine Linux are:

  • Speed in which the image is downloaded, installed and running on your Docker host
  • Security is improved as the image has a smaller footprint thus making the attack surface also smaller
  • Faster migration between hosts which is especially helpful in high availability and disaster recovery configurations.
  • Your system admin won’t complain as much as you will use less disk space

For my purposes, I need to run Spring Boot and Node.js applications on Docker containers, and they were easily switched from Debian based images to Alpine Linux without any changes. There are official Docker images for OpenJDK/OpenJRE on Alpine and Dockerfiles for running Oracle Java on Alpine. Although there isn’t an official Node.js image built on Alpine, you can easily make your own Dockerfile or use community provided files. When official Java Docker image is 642 MB, Alpine Linux with OpenJDK 8 is 150 MB and with Oracle JDK 382 MB (can be stripped down to 172 MB). With official Node.js image it’s 651 MB (or if using slim 211 MB) and with Alpine Linux that’s 36 MB. That’s a quite a reduction in size.

Examples of using minimal container based on Alpine Linux:

For Node.js:

FROM alpine:edge
RUN apk update && apk upgrade \
    && apk add nodejs="$NODE_ALPINE_VERSION"

For Java applications with OpenJDK:

FROM alpine:edge
RUN { \
      echo '#!/bin/sh'; \
      echo 'set -e'; \
      echo; \
      echo 'dirname "$(dirname "$(readlink -f "$(which javac || which java)")")"'; \
   } > /usr/local/bin/docker-java-home \
   && chmod +x /usr/local/bin/docker-java-home
ENV JAVA_HOME /usr/lib/jvm/java-1.8-openjdk
RUN set -x \
    && apk update && apk upgrade \
    && apk add --no-cache bash \
    && apk add --no-cache \
      openjdk8="$JAVA_ALPINE_VERSION" \
    && [ "$JAVA_HOME" = "$(docker-java-home)" ]

If you want to read more about running services on Alpine Linux, check Atlassian’s Nicola Paolucci’s nice article about experiences of running Java apps on Alpine.

Go small or go home?

So, should you use Alpine Linux for running your application on Docker? As also Docker official images are moving to Alpine Linux then it seems to make perfect sense from both a performance and security perspectives to switch to Alpine. And if you don’t want to take the leap from Debian or Ubuntu or want support from the downstream vendor you should consider stripping it from unneeded files to make it smaller.

Weekly notes 8

The Spring has been quite busy at work but Summer is just around the corner and that means either holidays or having some time to learn new things and see how things could be make better. My weekly notes has turned out to be monthly notes but that’s how things sometimes work out. But back to the issue which covers topics about continuous learning, best practices in development, looks into building blocks in Netflix’s stack and how to get started with ELK stack. And for the Summer project there’s Stanford’s Swift and iOS 9 course. Having done my iOS app with Swift it seems to be nice language.

Weekly notes, issue 8, 19.5.2016

Learning new things

Developing iOS 9 Apps With Swift from Stanford
Stanford iOS course is updated for Swift and iOS 9 and is good resource for learning iOS, Swift, or just to refresh yourself on best practices when developing for the platform. (Indie iOS focus weekly, issue 66)

Keep on learning and keep it simple

The single biggest mistake programmers make every day
Nice writeup of basic principles in programming. In short: Keep It Stupid Simple. Make it work, make it right, make it fast. Do One Thing.

Being A Developer After 40
Software development is always changing which this article tells nicely and gives good advice for the young at heart how to reach the glorious age of 40 as a happy software developer. tl;dr; Forget the hype, Choose your galaxy wisely, Learn about software history, Keep on learning, Teach, Workplaces suck, Know your worth, Send the elevator down, LLVM, Follow your gut, APIs are king, Fight complexity,

5 Tips To Improve Your JS with ES6
A well recorded hour long remote talk covering not only some handy ES6 tips, but how to work with ES6 generally and some of the tools available. (from JavaScript Weekly, issue 274)

Microservices, best practices and Java

Microservices are about applying a group of Best Practices
Moving an existing codebase to a microservice architecture is no small feat. And that’s not even taking into account the non-technical challenges. We definitely need more nuanced strategies based on actual production experience with microservices to help drive these architectural decisions. (from Java Web Weekly 123)

jDays 2016: Java EE Microservices Platforms
A lot of people preach that you can’t build microservices with Java EE but Steve Millidge’s talk about Java EE Microservices Platforms tells us that Payara Micro and Wildfly Swarm are fast and have a small memory footprint and that it does not require any code changes to port the application from one to other. (from Java Web Weekly 18/16)

The Netflix Stack: Part 1, Part 2 and Part 3
Microservices architecture in software development is what you should nowadays do but the question is how? The Netflix Stack article series covers some open source libraries you can use to build your architecture. Part 1 covers Eureka for service discovery and Part 2 is about Hystrix, latency and fault tolerance library. Part 3 is about creating rest clients for all of your services. The blog posts are an overview of what you can find in the accompanying repository.

Java app monitoring with ELK: part 1: Logstash and Logback and part 2: ElasticSearch
These blog posts tells you about the ELK stack (ElastichSearch, Logtash, Kibana) which is useful tool for logging visualization and analysis. (from Java Web Weekly 116)


10 SQL tricks that you didn’t think were possible
Lukas Eder tells you 10 SQL tricks that many of you might not have thought were possible. The article is a summary of his extremely fast-paced, ridiculously childish-humoured talk. “SQL is the original microservice”.

Tools of the trade

“A a simple starting point for a better Bash user experience out of the box.” These settings do make Bash easier and more useful. (from Weekend Reading)

Stranger Danger: Addressing the Security Risk in NPM Dependencies
Presentation from the O’Reilly Fluent Conference by Snyk co-founders which covers recently found exploit, and shows you how to use Snyk in your development workflow.

Something different

Interesting simulation with JavaScript how the web looks like to people with dyslexia. In the comments person with dyslexia tells that it’s easier to read when the text shifts. So, would dyslexia mode be good for website UX :) (from Weekend Reading)

Avoiding JVM delays caused by random number generation

The library used for random number generation in Oracle’s JVM relies on /dev/random by default for UNIX platforms. This can potentially block the WebLogic Server process because on some operating systems /dev/random waits for a certain amount of “noise” to be generated on the host machine before returning a result.

Although /dev/random is more secure, it’s recommended to use /dev/urandom if the default JVM configuration delays WebLogic Server startup. To determine if your operating system exhibits this behaviour, try displaying a portion of the file from a shell prompt: head -n 1 /dev/random

If the command returns immediately, you can use /dev/random as the default generator for JVM. If the command does not return immediately, use these steps to configure the JVM to use /dev/urandom:

  1. Open the $JAVA_HOME/jre/lib/security/java.security file in a text editor.
  2. Change the line “securerandom.source=file:/dev/random” to read: securerandom.source=file:/dev/./urandom
  3. Save your change and exit the text editor.

And because there’s a bug in JDK when you use /dev/urandom you have to set it up as /dev/./urandom

You can also set up system property “java.security.egd” which will override the securerandom.source setting.

Weekly notes 7

Easter and couple of days of free time is good for taking a break from the routines or finally have some time to develop your personal pet projects. At least my Highkara news reader for iOS needs some UI tests for screenshots and maybe I get to finish my imgur app for tvOS. But before that here’s the weekly notes.

This week we get overview to OWASP projects, see how Stack Overflow is built, learn to design for the Apple TV and get to run WebLogic on Docker container. Finally we discover how Spotify Discover Weekly playlists work.

Issue 7, 2016-03-24


Quick developer’s guide to OWASP projects
Interesting poster-type developer’s guide to OWASP projects. Learn how to secure your web apps against common web vulnerabilities.

How it’s built

Stack Overflow: The Architecture – 2016 Edition
If you’re wondering how’s Stack Overflow built and what’s the load check this article. Interesting. Running on Windows using IIS, ASP.Net, .Net, SQL Server and supported by CentOS and Redis, Elasticsearch.

Why I Left Gulp and Grunt for npm Scripts
Cory House explains how Gulp and Grunt are unnecessary abstractions, whereas npm scripts are plenty powerful and often easier to live with. It’s easier to debug as there’s no extra layer of abstraction, there’s no dependence on plugin authors to update, original tool is better and clearer documented. (from Web Design Weekly #219)

iOS and tvOS development

An in-app debugging and exploration tool for iOS
Excellent tool for iOS developer which helps you for example to simulate 3D Touch in the Simulator. Also in Xcode 7.3 you can now simulate 3D Touch without external tools if your trackpad has Force Touch.

Designing for the Apple TV
Michael Flarup writes some tips for getting design right when working with the Apple TV. He covers all of the basics but also some interesting points like making sure you meet the expectations of a TV based platform in terms of displaying and taking advantage of video based content. (from iOS dev Weekly #239)

This is pttrns for tvOS. Not a huge amount of data in yet but what’s there is worth a look. (from iOS dev Weekly #240)

Enterprise Java

WebLogic on Docker Containers Series: Part 1, Part 2 and Part 3
If you are using WebLogic as your application server, you should have a look at Bruno Borges series about running WebLogic on Docker. First post gets you started and shows how to create a basic Docker image with WebLogic and one with a configured WebLogic domain. The second post takes a more detailed look at the creation of the images, and the third one focusses on the domain configuration. (from Java Weekly 8/16)

Something different

I Documented Two Years of Travel By Painting In My Moleskine Notebook
Lovely hand-crafted art collection created by a traveler during her visits to different places around the world. An alternative to taking thousands of photos that no one will look at afterwards anyway and a beautiful, more emotional representation of lovely places. (from WDRL 126)

How Spotify Discover Weekly playlists work? and Recommending music on Spotify with deep learning
If you’re wondering how Spotify finds the tracks to your Discover Weekly list, read these two articles.

Container orchestration with CoreOS at Devops Finland meetup

Development and Operations, DevOps, is one of the important things when going beyond agile. It’s boosting the agile way of working and can be seen as an incremental way to improve our development practices. And what couldn’t be a good place to improve than learning at meetups how others are doing things. This time DevOps Finland meetup was about container orchestration with CoreOS and it was held at Oppex’s lounge in central Helsinki. The talks gave a nice dive into CoreOS, covering both beginner and seasoned expert points of view. Here’s my short notes about the presentations.

CoreOS intro for beginners, by beginners

The first talk was practically an interactive Core OS tutorial by Antti Vähäkotamäki and Frans Ojala. Their 99 slides showed how to get started with CoreOS on Vagrant step by step and what difficulties they experienced. Nothing special.

CoreOS in production, lessons learned

The more interesting talk about CoreOS was “CoreOS in production, lessons learned” by Vlad Bondarenko from Oppex where he told about their software stack and how they’re running it. In short, they’re running on baremetal with CoreOS Nginx for reverse proxy, Node.js for UI and API and RethinkDB and SolrCloud clusters. Deployment is made with Ansible and makefiles and Ship.it is used for Node.js. Service discovery is DNS based with docker-etcd-registrator component and they’ve also written their own DNS server. For Node.js config management with etcd they’ve made etcd-simple-config component. With Docker they use standard images with volumes and inject own data to the container.

CoreOS seemed to work quite well for them with easy cluster management, running multiple versions of 3rd party and own software and having zero downtime updates or rollbacks. But there were some cons also like maturity (bugs) and scripting systemd.

Kontena, CoreOS war stories

The last talk was about CoreOS war stories in Kontena by Jari Kolehmainen. The slides tell the story of how they use CoreOS on Kontena and what are the pain points. In story short it comes to configuration management and issues related to etcd.

For bootstrapping they use CloudInit which is de-facto way to initialize cloud instances and Integrated to CoreOS. The hard parts with etcd are discovery, security (tls certificates), using central services vs. workers and maintenance (you don’t do it). Now they run etcd inside a container, bind it only to localhost and overlay network (Weave Net) and master coordinates etcd discovery. With automatic updates they use the best-effort strategy: If etcd is running, locksmith coordinates the reboots; Otherwise just reboot when update is available.

Presentation’s summary was that the “OS” part is currently best option for containers and etcd is a must, but a little hard to handle. For the orchestrator they suggest that pick one which hides all the complexities. And automate all the things.

Problems with installing Oracle DB 12c EE, ORA-12547: TNS: lost contact

For development purposes I wanted to install Oracle Database 12c Enterprise Edition to Vagrant box so that I could play with it. It should’ve gone quite straight forwardly but in my case things got complicated although I had Oracle Linux and the pre-requirements fulfilled. Everything went fine until it was time to run the DBCA and create the database.

The DBCA gave “ORA-12547: TNS: lost contact” error which is quite common. Google gave me couple of resources to debug the issue. Oracle DBA Blog explained common issues which cause ORA-12547 and solutions to fix it.

One of the suggested solutions was to check to ensure that the following two files are not 0 bytes:

ls -lt $ORACLE_HOME/bin/oracle
ls -lt $ORACLE_HOME/rdbms/lib/config.o

And true, my oracle binary was 0 bytes

-rwsr-s--x 1 oracle oinstall 0 Jul  7  2014 /u01/app/oracle/product/12.1.0/dbhome_1/bin/oracle

To fix the binary you need to relink it and to do that rename the following file:

$ cd $ORACLE_HOME/rdbms/lib
$ mv config.o config.o.bad

Then, shutdown the database and listener and then “relink all”

$ relink all

If just things were that easy. Unfortunately relinking ended on error:

[oracle@oradb12c lib]$ relink all
/u01/app/oracle/product/12.1.0/dbhome_1/bin/relink: line 168: 13794 Segmentation fault      $ORACLE_HOME/perl/bin/perl $ORACLE_HOME/install/modmakedeps.pl $ORACLE_HOME $ORACLE_HOME/inventory/make/makeorder.xml > $CURR_MAKEORDER
writing relink log to: /u01/app/oracle/product/12.1.0/dbhome_1/install/relink.log

After googling some more I found similar problem and solution: Relink the executables by running make install.

cd $ORACLE_HOME/rdbms/lib
make -f ins_rdbms.mk install
cd $ORACLE_HOME/network/lib
make -f ins_net_server.mk install
If needed you can also relink other executables:
<pre lang="shell">
make -kf ins_sqlplus.mk install (in $ORACLE_HOME/sqlplus/lib)
make -kf ins_reports60w.mk install (on CCMgr server)
make -kf ins_forms60w.install (on Forms/Web server)

But of course it didn’t work out of the box and failed to error:

/bin/ld: cannot find -ljavavm12
collect2: error: ld returned 1 exit status
make: *** [/u01/app/oracle/product/12.1.0/dbhome_1/rdbms/lib/oracle] Error 1

The solution is to copy the libjavavm12.a to under $ORACLE_HOME lib as explained:

cp $ORACLE_HOME/javavm/jdk/jdk6/lib/libjavavm12.a $ORACLE_HOME/lib/

Run the make install commands from above again and you should’ve working oracle binary:

-rwsr-s--x 1 oracle oinstall 323649826 Feb 17 16:27 /u01/app/oracle/product/12.1.0/dbhome_1/bin/oracle

After this I ran the relink again which worked and also the install of the database worked fine.

relink all

Start the listener:

lsnrctl start LISTENER

Create the database:

dbca -silent -responseFile $ORACLE_BASE/installation/dbca.rsp

The problems I encountered while installing Oracle Database 12c Enterprise Edition to Oracle Linux 7 although in Vagrant and with Ansible were surprising as you would think that on certified platform it should just work. If I would’ve been using CentOS or Ubuntu it would’ve been totally different issue.

You can see the Ansible tasks I did to get Oracle DB 12c EE installed on Oracle Linux 7 in my vagrant-experiments GitHub repo.

Oracle DB 12c EE Ansible Tasks
Oracle DB 12c EE Ansible Tasks