Generating documentation as code with mermaid and PlantUML

Writing documentation is always a task which isn’t much liked and especially with diagrams and flowcharts there’s the problem of which tools to use. One crafty tool is Draw.io with web and desktop editors but what to use if you want to write documentation as a code and see the changes clearly in text format and maintain source-controlled diagrams? One of the tools for drawing diagrams with human readable text are mermaid and PlantUML.

mermaid

“Generation of diagrams and flowcharts from text in a similar manner as markdown.”

mermaid is a simple markdown-like script language for generating charts from text via javascript. You can try it in live editor.

mermaid sequence diagram

You can write mermaid diagrams in text editor but it’s better to use some editor with plugins to preview your work. Markdown Preview Enhanced for Atom and VS Code can render mermaid and PlantUML. There’s also dedicated preview plugins for VS Code and Atom.

To preview mermaid definition in VS Code with Markdown Preview Enhanced press Cmd-P to open Command palette and select Markdown Preview Enhanced: Open Preview.

mermaid in VS Code with Markdown Preview Enhanced

To preview mermaid definition in VS Code with Mermaid Preview press Cmd-P to open Command palette and select Preview Mermaid Diagram.

mermaid in VS Code with Mermaid preview
mermaid in VS Code with Mermaid preview

Generating PNG images from mermaid definitions

To use mermaid diagrams it’s useful to export them to PNGs. You can use mermaid.cli tool which takes a mermaid definition file as input and generates svg/png/pdf file as output.

Install mermaid.cli locally:

npm install mermaid.cli

Generate PNG:

./node_modules/.bin/mmdc -i input.mmd -o output.png

If you have plenty of defition files you can use the following script to generate PNGs:

#!/usr/bin/env bash
 
for mmd in ./docs/*.mmd
do
    filename="${mmd##*/}"
    echo "Generating $mmd"
    ./node_modules/.bin/mmdc -i $mmd -o ${filename%%.*}.png
done

Alternatively you can use node_modules/mermaid/bin/mermaid.js $mmd where mmd is the mermaid file.

PlantUML diagrams

PlantUML is used to draw UML diagrams, using a simple and human readable text description.

PlantUML is used to draw UML diagrams, using a simple and human readable text description. Diagrams are defined using a simple and intuitive language (pdf) and images can be generated in PNG, in SVG or in LaTeX format.

You can use PlantUML to write e.g. sequence diagrams, usecase diagrams, class diagrams, component diagrams, state diagrams and deployment diagrams.

PlantUML example diagram:

PlantUML diagram

Simple way to create and view PlantUML diagrams is to use Visual Studio Code and Markdown Preview Enhanced plugin which renders both PlantUML and mermaid diagrams. Alternative option is to use plantuml plugin.

To preview PlantUML diagram in VS Code with Markdown Preview Enhanced press Cmd-P to open Command palette and select Markdown Preview Enhanced: Open Preview.

PlantUML in VS Code with Markdown Preview Enhanced

There’s an online demo server which you can use to view PlantUML diagrams. The whole diagram is compressed into the URL itself and diagram data is stored in PNG metadata, so you can fetch it even from a downloaded image. For example this link opens the PlantUML Server with a simple Authentication activity diagram.

Running PlantUML server locally

Although you can render PlantUML diagrams online it’s better for usability and security reasons to install a local server. And this approach is important if you plan to generate diagrams with sensitive information. The easiest path is to run PlantUML Server Docker container and configure localhost as server.

docker run -d -p 8080:8080 plantuml/plantuml-server:jetty

In VS Code config, open user setting, and configure like:

"plantuml.server": "http://localhost:8080",
"plantuml.render": "PlantUMLServer",

Now to preview diagram in VS Code press alt-D to start PlantUML preview.

PlantUML preview in VS Code and local server

You can also generate diagrams from the command line. First download PlantUML compiled Jar and run the following command which will look for @startXXX into file1, file2 and file3. For each diagram, a .png file will be created.

java -jar plantuml.jar file1 file2 file3

The plantuml.jar needs Graphviz for dot (graph description language) and on macOS you can install it from Homebrew: brew install graphviz.

For processing a whole directory, you can use the following command which will search for @startXXX and @endXXX in .c, .h, .cpp, .txt, .pu, .tex, .html, .htm or .java files of the given directories:

java -jar plantuml.jar "directory1" "directory2"

Maintain source-controlled diagrams as a code

Documentation and drawing diagrams can be simple and maintaining source-controlled diagrams with tools like PlantUML and mermaid is achievable. These tools are not like the behemoth of Sparx Enterprise Architect but provide light and easy way to draw different diagrams for software development. You don’t have to draw lines and position labels manually as they are magically added where they fit and you even get as crude boxes and squares as thousands of dollars more expensive tools. Now the question is which tool to choose: PlantUML or mermaid?

Extracting JSON value from command line with jq and Python

Developing modern web applications you often come to around checking REST API responses and parsing JSON values. You can do it with a combination of Unix tools like sed, cut and awk but if you’re allowed to install extra tools or use Python then things get easier. This post shows you couple of options for extracting JSON values with Unix tools.

There are a number of tools specifically designed for the purpose of manipulating JSON from the command line, and will be a lot easier and more reliable than doing it with awk. One of those tools is jq as shown Stack Overflow. You can install it in macOS from Homebrew: brew install jq.

$ curl -s 'https://api.github.com/users/walokra' | jq -r '.name'

If you’re limited to tools that are likely installed on your system such as Python, using the json module gives you the benefit of a proper JSON parser and avoiding any extra dependencies.

Python:

$ curl -s 'https://api.github.com/users/walokra' | \
    python -c "import sys, json; print(json.load(sys.stdin)['name'])"

Stack Overflow answers to the question of “Parsing JSON with Unix tools” shows you other options with standard tools like sed, cut and Awk and more exotic options with Perl, Node.js and PHP.

Setting up Bower and Gulp in Windows

Doing things manually is fine once but if you can automate things it’s better. With little tools you can speed up development and reduce recursive mundane tasks such as starting a project or setting up boilerplate code. I recently came across Bower which is a package manager for the web. With Bower you can fetch and install packages from all over, and it takes care of finding, downloading, and saving the stuff you’re looking for. The other interesting tool to help you get going is Gulp which enables you to automate and enhance your workflow. Let’s see how to put things together on Windows, nothing special just steps to get you started.

Gulp tasks

Install Git

Bower needs Git to work so first install Git if you don’t have it. I chose Git for Windows which gives you BASH emulation used to run Git from the command line, graphical user interface for using Git and Shell integration.

Just click through the installation.

Install Node.js

Bower depends on Node.js and NPM so you need to get Node.js. Just download the installation package from Node.js site and click through it.

Install Bower

After you have Node.js installed we can install Bower with npm. You might need to restart your Windows to get all the path variables setup so Npm can find them.

Open up the Git Bash or Command Prompt and Bower is installed globally by running the following command.

$ npm install -g bower

Once you have Bower installed you then can install packages and dependencies using these commands:

# Using a local or remote package
bower install <package>
 
# Using a specific version of a package
bower install <package>#<version>
 
# Search packages
$ bower search <package>

By default packages will be put in the bower_components directory which can be changed if you prefer. If you want your packages downloaded into js/libs you can achieve this by creating a .bowerrc file

.bowerrc

{
    "directory": "js/libs"
}

You can also create a bower.json file which allows you to define the packages needed along with dependencies and then simply run bower install to download packages. In our example we setup a simple Backbone.js application which uses Bootstrap.

bower.json

{
    "name": "Foobar",
    "version": "0.1.0",
    "dependencies": {
          "jquery": "~2.0.3",
          "underscore": "~1.5.0",
          "bootstrap": "~3.3.2",
          "backbone": "~1.1.2"
    }
}

Our bower.json describes that we want some JavaScript libraries and as we have defined the version with ~ it can have bigger minor versions, e.g. jquery version can be between 2.0.3 < 2.1.0. Read more about semantic versioner for npm.

Now after creating that file inside the app directory you can run the following command:

$ bower install

After that you should see all your JavaScript packages under bower_components folder.

Install Gulp

To automate and enhance your workflow you can use Gulp for example to copy the files where you want them. There are nice recipes to show how to benefit of Gulp.

Install Gulp globally with npm:

$ npm install --global gulp

Install Gulp also in your project devDependencies:

$ npm install --save-dev gulp

Now we can setup our Gulp dependencies which pull from npm. Create a new package.json file in your project root and just add an empty object, {} and save it.

Next we install gulp-bower plugin which we can use to install Bower packages.

$ npm install --save-dev gulp-bower

This will install all the needed dependencies in a node_modules folder and also automatically update our package.json file with these dependencies.

Finally we need to setup the gulpfile.js which defines our tasks we want to perform. First we define what we installed in npm step above and create a config object to hold various settings. The bowerDir is just the path to the bower_components. Finally we add task for running bower and default task. Our bower tasks basically runs bower install but by including in the gulpfile other contributors only have to run gulp bower and have them all setup and ready.

gulp.js

var gulp = require('gulp'),
    bower = require('gulp-bower');
 
var config = {
     bowerDir: './bower_components'
}
 
gulp.task('bower', function() {
    return bower()
        .pipe(gulp.dest(config.bowerDir))
});
 
$ gulp.task('default', ['bower']);

The default task runs the bower task and all the user has to do to setup the needed packages is to run

$ gulp

In our case running gulp just runs our bower task which downloads the JavaScript packages we need. Pretty simple.

Gulp is powerful tool and has many use cases but also needs some to get all things working like you want and even then you might need to make compromises. One crafty task for Gulp and Bower is to customize your Bootstrap theme. Also Mark Goodyear has written good article about Getting started with gulp which shows some typical use cases.

Essential IntelliJ IDEA keyboard shortcuts

Recently I switched from using Eclipse to IntelliJ IDEA as our Java EE application’s front-end was done with JavaScript and the support for front-end technologies in Eclipse is more or less non-existent. The switch for long time Eclipse user wasn’t easy as IDEA works a bit differently but the change was worth it. The biggest difference in daily work with IDE is the shortcuts which are quite different in IDEA. In theory you can use Eclipse keymap for shortcuts but it just doesn’t work like it should and in practice you have to learn the IDEA way. There are many posts in the Internet about keyboard shortcuts in IDEA but there’s always place for more :) So, here’s my list of shortcuts to keep in your finger memory.

Learn keymap with Key Promoter

To learn your way around IntelliJ IDEA’s keyboard shortcuts there’s nice “Key Promoter” plugin to train yourself. It prompts whenever you use the mouse when you could’ve used the keyboard instead (similar to Eclipse’s Mousefeed).

To install the plugin:

  1. Ctrl+Alt+S to pull up the Settings screen
  2. Filter on “plugin”. Click “Plugins”, then “Browse Repositories” at the bottom
  3. Filter on “key promoter”
  4. Double click to install
  5. Essential IntelliJ IDEA keyboard shortcuts

    IntelliJ IDEA keymap

    You may be tempted to just go with the Eclipse keymap but it’s better to learn the IDEA way although it’s quite irritating at start. You also should change some default IDEA keyboard shortcuts to better ones like “closing editor window” with Ctrl+F4 which is too cumbersome compared to the de facto Ctrl+W. And changing “comment current line or selection” with Ctrl+/ which is impossible with Finnish keyboards to Ctrl+7.

    If you want to know how Eclipse shortcuts map to IDEA there’s nice post about IntelliJ IDEA shortcuts for Eclipse users and I added some in my list.

    Recent Viewed or edited Files: CTLR + E / CTRL + SHIFT + E
    Shows you a popup with all the recent files that you have opened or actually changed in the IDE. If you start typing, you can filter the files.

    Go to Class or file: CTRL + N and CTRL + Shift + N
    Allows you to search by name for a Java file in your project. If you combine it with SHIFT, it searches any file. Adding ALT on top of that it searches for symbols. (Eclipse: Ctrl+Shift+T and Ctrl+Shift+R)

    Find and Replace in Path: CTRL + SHIFT + F / CTRL + SHIFT + R
    Allows you to find in files or replace in files. (Eclipse: Ctrl+H)

    Incremental Find: F3 / CTRL + L
    When using CTLR-F to find in current file the F3 lets you to loop through the results. (Eclipse: Ctrl+K)

    Delete line: CTRL + Y
    Delete current line under cursor. Yank it. (Eclipse: Ctrl+D)

    Goto line: CTRL + G
    Go to given line number. (Eclipse: Ctrl-L)

    Syntax Aware Selection: CTRL + W
    Allows you to select code with context. Awesome when you need to select large blocks or just specific parts of a piece of code.

    Complete Statement: CTRL + SHIFT + ENTER
    This will try to complete your current statement. How? By adding curly braces, or semicolon and line change.

    Smart Type Completion: CTRL + SHIFT + SPACE
    Like auto complete (CTRL + SPACE) but if you add a SHIFT you get the smart completion. This means that the IDE will try to match expected types that suit the current context and filter all the other options.

    Reformat source code and optimize imports: CTRL + ALT + L
    Allows you to reformat source code to meet the requirements of your code style. Lays out spacing, indents, keywords etc. Reformatting can apply to the selected text, entire file, or entire project.

    Quick Fix: Alt+Enter
    (Eclipse: Ctrl+1)
    Gives you a list of intentions applicable to the code at the caret.

    Paste one of the previous values from clipboard: CTRL + SHIFT + V
    Shows you a dialog to select previous value from the clipboard to be pasted.

    Comment or uncomment line or block: Ctrl+7 / Ctrl+Shift+7
    Allows you to comment or uncomment the current line or selected block of source code. This is originally Ctrl + / (Slash) which is impossible with Finnish keyboard layouts.

    Show Diff (in Changes): CTRL + D
    In Changes tab compares the file with latest repository version.

    Highlight Usages: CTRL + SHIFT + F7
    Place the cursor in a element and after pressing the cursor the IDE will highlight all the occurrences of the selected element.

    Go to Declaration: CTRL + B
    If you place the cursor in a class, method or variable and use the shortcut you will immediately jump to the declaration of the element.

    Navigate Between Methods: ALT + UP/DOWN Arrows
    Jump between methods.

    Refactoring String Fragments: CTRL + ALT + V
    Refactor hardcoded string into variable/field/constant. Select the section of the String you want to extract, and use the normal “Extract…” shortcuts to extract it into a variable.

    Other useful keyboard shortcuts

    There are many useful keyboard shortcuts and you can print them from Help > Default Keymap Reference. Here are some more shortcuts which are also handy.

    Update application: CTRL + F10
    Current file structure: CTRL + F12
    Bookmarks: F11 and SHIFT + F11
    Generate Getters & Setters (in editor): ALT + INSERT
    Create New _* (in project navigator): ALT + INSERT
    Refactor – Rename: SHIFT + F6
    Open Settings CTRL + Alt + S
    Duplicate line: CTRL + D
    Move line: CTRL + ALT + UP / DOWN
    Find Command: CTRL + SHIFT + A
    Show usages in a pop-up list: CTRL + Alt + F7
    Extract Variable/Method/Constant/Field: CTRL + ALT + V/M/C/F
    Quick JavaDoc Popup: CTRL + Q
    Tab switcher: CTRL + TAB
    Jump to Project Navigator: ALT + 1
    Jump back to last tool window/panel: F12
    Jump to beginning/end of block (e.g., method start/end): CTRL + [ and CTRL + ]
    Toggle uppercase/lowercase of selection: CTRL + SHIFT + U
    Toggle collapse/expand: CTRL + .
    Toggle full screen editor (hide other tool windows): CTRL + SHIFT + F12

    Not a keyboard shortcut exactly but the “iter” smart template is great. If you want to iterate though something using a for loop type “iter” then TAB to use the live template. It will figure out the most likely variable you want to iterate over and generate a for loop for it. In Eclipse it worked more logically with just typing for and then autocomplete.

Monitor and profile application with Java Mission Control

Monitoring Java applications is can be done with different tools and with JDK you get one good tool for it: Java Mission Control. Java Mission Control and Java Flight Recorder together create a complete tool chain to continuously collect low level and detailed runtime information enabling after-the-fact incident analysis. Starting with Oracle JDK 7 Update 40 (7u40) Java Mission Control (JMC) bundled with the HotSpot JVM. Let’s take a short look what those tools are.

“Java Flight Recorder is a profiling and event collection framework built into the Oracle JDK. It allows Java administrators and developers to gather detailed low level information about how the Java Virtual Machine (JVM) and the Java application are behaving. Java Mission Control is an advanced set of tools that enables efficient and detailed analysis of the extensive of data collected by Java Flight Recorder. The tool chain enables developers and administrators to collect and analyze data from Java applications running locally or deployed in production environments.”Oracle Java Mission Control.

Java Mission Control and Java Flight Recorder are commercial features, which are available as part of the Oracle Java SE Advanced product. They are freely available for download for development and evaluation purposes, as per the terms in the Oracle Binary Code License Agreement, but require an Oracle Java SE Advanced license for production use.

Using Java Mission Control

Oracle Java Mission Control is a tool suite for managing, monitoring, profiling, and troubleshooting your Java applications and it consists of the JMX Console and the Java Flight Recorder. To get a good overview how you can use Java Mission Control check Java Mission Control demo video. The JMC Client is built to run on Eclipse and it’s based on the features of the old JRockit Mission Control.

The JMX Console enables you to monitor and manage your Java application and the JVM at runtime but the main and most important feature is the Flight Recorder. Java Flight Recorder (JFR) records the behavior of the JVM at runtime and you can analyze the recording offline using the Java Flight Recorder tool. They say that the overall profiling overhead for your applications stays at less than 2%, usually much less.

Starting with Oracle JDK 7 Update 40 (7u40) it’s bundled with the HotSpot JVM and although you can connect it to older JDK’s like application running on JDK 6 the newer ones show more information and have more features. So no real fun with legacy applications. The Flight Recorder needs at least JDK 7 Update 40.

Start JMC from the Windows command prompt:
"c:\Program Files\Java\jdk1.8.0_20\bin\jmc.exe"

Java Mission Control can be connected to local or remote Java Application. Start your application with following Virtual Machine flags which enables the JMX remote without authentication and Mission Control:

-Dcom.sun.management.jmxremote 
-Dcom.sun.management.jmxremote.port=8999 
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false 
-XX:+UnlockCommercialFeatures 
-XX:+FlightRecorder

If you’re using WebLogic then the JMX Remote settings are following:

-Djavax.management.builder.initial=weblogic.management.jmx.mbeanserver.WLSMBeanServerBuilder
-Dcom.sun.management.jmxremote.port=8999 -Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.authenticate=false 
-XX:+UnlockCommercialFeatures 
-XX:+FlightRecorder

JMC does not include or run with a security manager, so a user must ensure to run the JMC client in a secure environment.

After connecting JMC with your Java application it opens a familiar Eclipse based user interface. JMX Console has couple of tabs on the bottom which shows overview, MBeans, Memory and Thread information.

JMC: JMX Console Overview

JMC: JMX Console MBeans
JMC: JMX Console Memory

JMC: JMX Console Threads

The more useful tool is the Java Flight Recorder (JFR) for profiling your application. In Java Mission Control JVM Browser right click on the Java Virtual Machine you wish to start a Flight Recording.

Leave all the default settings and select the “Profiling – on server” template for your Event Settings. Just hit finish at this point. You can also click Next to go to the higher level event settings which are groupings of named settings in the template. You can select how often you want JFR to sample methods by changing the Method Sampling setting.

The recording will be downloaded automatically and displayed in Mission Control. Click the tab group for Code to start visualizing your Method Profiling Sample events. Switch to the method profiling tab to find a top list of the hottest methods in your application.

Too bad I don’t have nice recording to show but here’s couple of screenshots. Better overview of how to use Flight Recording can be seen from the Java Mission Control demo video.

JMC: Flight Recording

JMC: Flight Recording