Generating documentation as code with mermaid and PlantUML

Writing documentation is always a task which isn’t much liked and especially with diagrams and flowcharts there’s the problem of which tools to use. One crafty tool is Draw.io with web and desktop editors but what to use if you want to write documentation as a code and see the changes clearly in text format and maintain source-controlled diagrams? One of the tools for drawing diagrams with human readable text are mermaid and PlantUML.

mermaid

“Generation of diagrams and flowcharts from text in a similar manner as markdown.”

mermaid is a simple markdown-like script language for generating charts from text via javascript. You can try it in live editor.

mermaid sequence diagram

You can write mermaid diagrams in text editor but it’s better to use some editor with plugins to preview your work. Markdown Preview Enhanced for Atom and VS Code can render mermaid and PlantUML. There’s also dedicated preview plugins for VS Code and Atom.

To preview mermaid definition in VS Code with Markdown Preview Enhanced press Cmd-P to open Command palette and select Markdown Preview Enhanced: Open Preview.

mermaid in VS Code with Markdown Preview Enhanced

To preview mermaid definition in VS Code with Mermaid Preview press Cmd-P to open Command palette and select Preview Mermaid Diagram.

mermaid in VS Code with Mermaid preview
mermaid in VS Code with Mermaid preview

Generating PNG images from mermaid definitions

To use mermaid diagrams it’s useful to export them to PNGs. You can use mermaid.cli tool which takes a mermaid definition file as input and generates svg/png/pdf file as output.

Install mermaid.cli locally:

npm install mermaid.cli

Generate PNG:

./node_modules/.bin/mmdc -i input.mmd -o output.png

If you have plenty of defition files you can use the following script to generate PNGs:

#!/usr/bin/env bash
 
for mmd in ./docs/*.mmd
do
    filename="${mmd##*/}"
    echo "Generating $mmd"
    ./node_modules/.bin/mmdc -i $mmd -o ${filename%%.*}.png
done

Alternatively you can use node_modules/mermaid/bin/mermaid.js $mmd where mmd is the mermaid file.

PlantUML diagrams

PlantUML is used to draw UML diagrams, using a simple and human readable text description.

PlantUML is used to draw UML diagrams, using a simple and human readable text description. Diagrams are defined using a simple and intuitive language (pdf) and images can be generated in PNG, in SVG or in LaTeX format.

You can use PlantUML to write e.g. sequence diagrams, usecase diagrams, class diagrams, component diagrams, state diagrams and deployment diagrams.

PlantUML example diagram:

PlantUML diagram

Simple way to create and view PlantUML diagrams is to use Visual Studio Code and Markdown Preview Enhanced plugin which renders both PlantUML and mermaid diagrams. Alternative option is to use plantuml plugin.

To preview PlantUML diagram in VS Code with Markdown Preview Enhanced press Cmd-P to open Command palette and select Markdown Preview Enhanced: Open Preview.

PlantUML in VS Code with Markdown Preview Enhanced

There’s an online demo server which you can use to view PlantUML diagrams. The whole diagram is compressed into the URL itself and diagram data is stored in PNG metadata, so you can fetch it even from a downloaded image. For example this link opens the PlantUML Server with a simple Authentication activity diagram.

Running PlantUML server locally

Although you can render PlantUML diagrams online it’s better for usability and security reasons to install a local server. And this approach is important if you plan to generate diagrams with sensitive information. The easiest path is to run PlantUML Server Docker container and configure localhost as server.

docker run -d -p 8080:8080 plantuml/plantuml-server:jetty

In VS Code config, open user setting, and configure like:

"plantuml.server": "http://localhost:8080",
"plantuml.render": "PlantUMLServer",

Now to preview diagram in VS Code press alt-D to start PlantUML preview.

PlantUML preview in VS Code and local server

You can also generate diagrams from the command line. First download PlantUML compiled Jar and run the following command which will look for @startXXX into file1, file2 and file3. For each diagram, a .png file will be created.

java -jar plantuml.jar file1 file2 file3

The plantuml.jar needs Graphviz for dot (graph description language) and on macOS you can install it from Homebrew: brew install graphviz.

For processing a whole directory, you can use the following command which will search for @startXXX and @endXXX in .c, .h, .cpp, .txt, .pu, .tex, .html, .htm or .java files of the given directories:

java -jar plantuml.jar "directory1" "directory2"

Maintain source-controlled diagrams as a code

Documentation and drawing diagrams can be simple and maintaining source-controlled diagrams with tools like PlantUML and mermaid is achievable. These tools are not like the behemoth of Sparx Enterprise Architect but provide light and easy way to draw different diagrams for software development. You don’t have to draw lines and position labels manually as they are magically added where they fit and you even get as crude boxes and squares as thousands of dollars more expensive tools. Now the question is which tool to choose: PlantUML or mermaid?

Monthly notes 31

The first part of Summer has been great and holiday season is near. Here’s monthly notes for June with topics of microservices, kubernetes, design patterns and stories of how Shopify and Airbnb build their services. Also some tools like Kap. Happy reading.

Issue 31, 28.6.2018

Microservices

7 tips for effective microservices
“Have a request-id/correlation-id for every request, Maintain backward compatibility of interfaces, Have a centralized logging system, Implement idempotency and retries, Be aware of language constraints, Have a single service to manage the system state, Strike a balance between in-memory-data and db persistence” (from The Microservice Weekly)

Kubernetes

AWS Workshop for Kubernetes
“Self-paced workshop designed for Development and Operations teams who would like to leverage Kubernetes on Amazon Web Services (AWS).”

Kubernetes best practices: terminating with grace
“This episode of “Kubernetes Best Practices,” let’s take a look at how you can help Kubernetes do its job more efficiently and reduce the downtime your applications experience.”

Kubernetes Chaos Engineering: Lessons Learned — Part 1

Kubernetes and containers for enterprise developers
“O’Reilly Media Podcast talks with JP Phillips, platform engineer at IBM Cloud.”

iOS

xcprojectlint: A security blanket for Xcode project files
Would you like to automate some consistency in your Xcode project files with checks for settings defined at the project level (rather than in an xcconfig), missing files and empty file groups? This tool does exactly that, and more. Also, I like the way it’s described: “Provides a security blanket, ensuring neither your co-workers, nor git screw up your Xcode project file.” (from iOS Dev Weekly 353)

This app hacked the iPhone’s dual camera system, and you’ve never seen anything like it
Interesting: portrait mode collects 2D depth data along with the image itself. This app uses depth data to change the lightning source of photos after the fact. (from Weekend Reading)

Tools

Capture your screen
An open-source screen recorder built with web technology. Crafty for quick gif/mp4/webm/apng to issues, slack or other views.

Mozilla SSL Configuration Generator

How others are doing things

Shopify Infrastructure with Niko Kurtti
“Shopify has built its own platform-as-a-service on top of Kubernetes called Cloudbuddies. Niko Kurtti is a production engineer at Shopify joins the Software Engineering Daily show to describe Shopify’s infrastructure – how they run so many stores, how they distribute those stores across their infrastructure, and the motivation for building their own internal platform on top of Kubernetes.”

Building Services at Airbnb, Part 1
The first in a series on scaling service development, this article looks at the core structure, the Service IDL, underpinning the new Services Oriented Architecture at Airbnb.

Building Services at Airbnb, Part 2
The second in a series on scaling service development, this article looks at some of the key tooling that supports the new Services Oriented Architecture at Airbnb.

Design

Dieter Rams 10 Principles of Good Design
“But what is good design?” It’s around structure, function and aesthetics. “Good design is as little design as possible” (from @sidebario)

Design Patterns on CodePen

Awesome design patterns
A curated list of software and architecture related design patterns. Software design pattern – A general, reusable solution to a commonly occurring problem within a given context in software design. It is a description or template for how to solve a problem that can be used in many different situations.

Something Different

Cool Backgrounds
Collection of tools to create compelling, colorful images for blogs, social media, and websites. Beyond backgrounds, the images generated can be used as 🖥 desktop wallpapers or cropped for 📱 mobile wallpapers.

OWASP Helsinki chapter meeting 34: Secure API

OWASP Helsinki Chapter held a meeting number 34 last week at Eficode with topics of
“Perfectly secure API” and “Best friends: API security & API management”. The event gave good overview to the topics covered and was quite packed with people. Eficode’s premises were modern and there was snacks and beverages. And also a sauna. Here is a short recap of the talks.

OWASP Helsinki Chapter Meeting 34

Perfectly secure API

Matti Suominen from Nixu talked about perfectly secure API and things related to get there. Can API be secure? On gut feeling APIs seems to be rubbish and have problems. He covered the topic from three view points: security, risks and defense. Good starting point is to read OWASP resources like ASVS, Top 10 and Security cheat sheet. Also implement security centrally, involve business in design and DIY never works out.

Best friends: API security & API management

Antti Virtanen from Solita talked about API security and API management and how we’ve traveled from dark ages to modern times. You can do API security with tools like Amazon AWS API Gateway but the main point was to step further with API management. Use some already made products like Apigee and open source alternative Tyk.io. Slides are available in Slideshare.

Snacks and beverages

Refreshments were basic and different

Monthly notes 30

Summer is approaching and even in Finland the weather is sunny and warm. I’ve been busy as the Enduro-MTB racing season has started and most weekends are spent at the race track. But here’s monthly notes for May with topics of state of the Web, how geolocation in browsers work, and something about tools. Happy reading.

Issue 30, 30.5.2018

Web

The State of the Web at Google I/O 2018
Service Worker, Progressive Web Apps (PWAs), WebAssembly, Lighthouse, AMP, Web Packaging, Polymer, Angular. (from @igrigorik)

Ever stop to think about geolocation in your desktop browser?
tl;dr; location is triangulated by location services like Mozilla & Google from scan of nearby Wi-Fi access points’ signal strength and their known locations. List is collected when people walk with a phone with GPS and Wi-Fi on which polls networks. (from @walokra)

Is GraphQL The Future?
If you are not sold on GraphQL then this post might tip you over the edge. Alan Johnson does a great job in explaining the awesomeness that GraphQL has to offer. (from Web Design Weekly 321)

Tools for making HTTP requests

Imsomnia
Powerful HTTP and GraphQL tool belt. Debug APIs like a human, not a robot. Finally, a REST client you’ll love.

RESTed – Simple HTTP Requests
RESTed allows developers to quickly format and make HTTP requests and view the response. For Mac.

Paw
Paw is a full-featured HTTP client that lets you test and describe the APIs you build or consume. It has a beautiful native macOS interface to compose requests, inspect server responses, generate client code and export API definitions.

Security

US cell carriers are selling access to real-time phone location data
Intriguing thoughts: “Access to your real-time phone location data is sold to companies and public has zero idea how much personal location data is available. It is done throughout the industrialized world to varying degrees.” e.g. stocks are traded based on where peoole go. (from @walokra)

JavaScript

`npm audit`: identify and fix insecure dependencies
“npm audit is a new command that performs a moment-in-time security review of your project’s dependency tree. Audit reports contain information about security vulnerabilities in your dependencies and can help you fix a vulnerability by providing simple-to-run npm commands and recommendations for further troubleshooting.” (from JavaScript Daily)

JavaScript Algorithms and Data Structures
A wide variety of algorithms (e.g. permutations, Levenshtein distance, binary search) and data structures (e.g. linked lists, trees, stacks) implemented in JavaScript with explanations and links to further reading. (from JavaScript Weekly 387)

Thinking

Full Cycle Developers at Netflix — Operate What You Build
A look at how Netflix believes in ‘operating what you build’. (from Web Operations Weekly 167)

Something different

Unchained: A story of love, loss, and blockchain
> It was a smart contract that stipulated sexual fidelity and parental responsibilities. Tokens from their joint earnings paid the AI judges and IoT sensor oracles that monitored contract violations. On mornings like this, you really needed commitment that was mathematically provable, not just an empty promise at the altar.

Two days of React Finland 2018: Day two with React and React Native

React Finland 2018 conference was held last week and I had the opportunity to attend it and listen what’s hot in the React world. The conference started with workshops and after that there was two days of talks of React, React Native, React VR and all things that go with developing web applications with them. The two conference days were packed with great talks and new information. This is the second part of my recap of the talks and my notes which I posted to Twitter. Check out also the first part of my notes from the first day’s talks.

React Finland 2018, Day 2

How React changed everything — Ken Wheeler

Second day started with keynote by Ken Wheeler. He examined how React changed the front end landscape as we know it and started it with nice time travel to the 90s with i.a. Flash, JavaScript and AngularJS. Most importantly the talk took a look at the core idea of React, and why it transcends language or rendering target and posit on what that means going forward. And last we heard about what React async: suspense and time slicing.

“Best part of React is the community”

How React changed everything

Get started with Reason — Nik Graf

The keynote also touched Reason ML and Nik Graf went into details kicking off with the basics and going into how to leverage features like variant types and pattern matching to make impossible states impossible.

Get started with Reason ML

Making Unreasonable States Impossible — Patrick Stapfer

Based on “Get started with Reason” Patrick Stapfer’s talk went deeper into the world of variant types and pattern matching and put them into a practical context. The talk was nice learning by doing TicTacToe live coding. It showed how Reason ML helps you design solid APIs, which are impossible to misuse by consumers. We also got more insights into practical ReasonReact code. Presentation is available on the Internet.

Conclusion about ReasonReact:

  • More rigid design
  • More KISS (keep it simple, stupid) than DRY (don’t repeat yourself)
  • Forces edge-cases to be handled
Learning Reason by doing TicTacToe

Reactive State Machines and Statecharts — David Khourshid

David Khourshid’s talk about state machines and statecharts was interesting. Functional + reactive approach to state machines can make it much easier to understand, visualize, implement and automatically create tests for complex user interfaces and flows. Model the code and automatically generate exhaustive tests for every possible permutation of the code. Things mentioned: React automata, xstate. Slides are available on the Internet.

“Model once, implement anywhere” – David Khourshid

The talk was surprisingly interesting especially for use cases as anything to make testing better is good. This might be something to look into.

ReactVR — Shay Keinan

After theory heavy presentations we got into more visual stuff: React VR. Shay Keinan presented the core concepts behind VR, showed different demonstrations, and how to get started with React VR and how to add new features from the Three.js library. React VR: Three.js + React Native = 360 and VR content. On the VR device side it was mentioned that Oculus Go, HTC Vive Focus are the big step to Virtual Reality.

“Virtual Reality’s possibilities are endless. Compares to lucid dreaming.” – Shay Keinan

WebVR enables web developers to create frictionless, immersive experiences and we got to see Solar demo and Three VR demo which were lit 🔥.

React VR

World Class experience with React Native — Michał Chudziak

I’ve shortly experimented with React Native so it was nice to listen Michał Chudziak’s talk how to set up a friendly React Native development environment with the best DX, spot bugs in early stage and deliver continuous builds to QA. Again Redux was dropped in favour of apollo-link-state.

Work close to your team – Napoleon Hill

What makes a good Developer eXperience?

  • stability
  • function
  • clarity
  • easiness

GraphQL was mentioned to be the holy grail of frontend development and perfect with React Native. Tools for better developer experience: Haul, CircleCI, Fastlane, ESLint, Flow, Jest, Danger, Detox. Other tips were i.a to use native IDEs (XCode, Android Studio) as it helps debugging. XCode Instruments helps debug performance (check iTunes for video) and there’s also Android Profiler.

World Class experience with React Native

React Finland App – Lessons learned — Toni Ristola

Every conference has to have an app and React Finland of course did a React Native app. Toni Ristola lightning talked about lessons learned. Technologies used with React Native was Ignite, GraphQL and Apollo Client 👌 App’s source code is available on GitHub.

Lessons learned:

  • Have a designer in the team
  • Reserve enough time — doing and testing a good app takes time
  • Test with enough devices — publish alpha early
React Finland App – Lessons learned

React Native Ignite — Gant Laborde

80% of mobile app development is the same old song which can be cut short with Ignite CLI. Using Ignite, you can jump into React Native development with a popular combination of technologies, OR brew your own. Gant Laborde talked about the new Bowser version which makes things even better with Storybook, Typescript, Solidarity, mobx-state-tree and lint-staged. Slides can be found on the Internet.

Ignite
Ignite

How to use React, webpack and other buzzwords if there is no need — Varya Stepanova

Varya Stepanova’s lightning talk suggested to start a side-project other than ToDo app to study new development approaches and showed what it can be in React. The example was how to generate a multilingual static website using Metalsmith, React and other modern technologies and tools which she uses to build her personal blog. Slides can be found on the Internet.

Doing meaningful side-projects is a great idea to study new things and I’ve used that for i.a. learning Swift with Highkara newsreader, did couple of apps for Sailfish OS and played with GraphQL and microservices while developing app with largish vehicle dataset.

After party

Summary

Two days full of talks of React, React Native, React VR and all the things that go with developing web applications with them was great experience. Days were packed with great talks, new information and everything went smoothly. The conference was nicely organized, food was good and participants got soft hoodies to go with the Allas Sea Pool ticket. The talks were all great but especially “World Class experience with React Native” and “React Native Ignite” gave new inspiration to write some app. Also “ReactVR” seemed interesting although I think Augmented Reality will be bigger thing than Virtual Reality. It was nice to hear from “The New Best Practices” talk that there really is no new best practices as the old ones still work. Just use them!

Something to try and even to take into production will be Immer, styled components and Next.js. One thing which is easy to implement is to start using lint-staged although we are linting all the things already.

One of the conference organizers and speaker, Juho Vepsäläinen, wrote Lessons Learned from the conference and many of the points he mentions are to the point. The food was nice but “there wasn’t anything substantial for the afternoon break”. There wasn’t anything to eat after lunch but luckily I had own snacks. Vepsäläinen also mentions that “there was sometimes too much time between the presentations” but I think the longer breaks between some presentations were nice for having a quick stroll outside and have some fresh air. The venue was quite warm and the air wasn’t so good in the afternoon.

The Afterparty at Sea Life Helsinki was interesting choice and it worked nicely although there wasn’t so many people there. The aquarium was fishy experience and provided also some other content than refreshments. Too bad I hadn’t have time to go and check the Allas Sea Pool which we got a free ticket. Maybe next time.

Thanks to the conference crew for such a good event and of course to my fellow Goforeans which attended it and had a great time!

Two days of React Finland 2018: Day one topics of React

React Finland 2018 conference was held last week and I had the opportunity to attend it and listen what’s hot in the React world. The conference started with workshops and after that there was two days of talks of React, React Native, React VR and all things that go with developing web applications with them. The two conference days were packed with great talks and new information. Here’s the first part of my notes from the talks which I posted to Twitter. Read also the second part with more of React Native.

React Finland 2018: Day 1

React Finland combined the Finnish React community with international flavor from Jani Eväkallio to Ken Wheeler and other leading talents of the community. The event was the first of its kind in Finland and consisted of a workshop day and two days of talks around the topic. It was nice that the event was single track so you didn’t need to choose between interesting talks.

At work I’ve been developing with React couple of years and tried my hands with React Native so the topics were familiar. The conference provided crafty new knowledge to learn from and maybe even put to production. Overall the conference was great experience and everything went smoothly. Nice work from the React Finland conference team! And of course thanks to Gofore which sponsored the conference and got me a ticket.

I tweeted my notes from almost every presentation and here’s a recap of the talks. I heard that the videos from the conference will be available shortly.

The New Best Practices — Jani Eväkallio

First day’s keynote was by Jani Eväkallio who talked about “The New Best Practices”. As the talk description wrote “When React was first introduced, it was ridiculed for going against established web development best practices as we knew them. Five years later, React is the gold standard for how we create user interfaces. Along the way, we’ve discovered a new set of tools, design patterns and programming techniques.”

The new best practices were:

  • Build big things from small things
  • Write code for humans first: flow, Typescript, storybook
  • Stay close to the language:
    • helps i.a. linters
  • Always prefer simplicity
  • Don’t break things:
    • Facebook makes React API changes easy to upgrade, depreciation well in advance, migration, documentation. it’s a flow, not versions. Use codemod.
  • Keep an open mind

You ask “what new best practices”? Yep, that’s the thing. We don’t need new best practices as the same concepts like Model-View-Controller and separation of concerns are still valid. We should use best practices which have been proved good before as they also work nicely with React philosophy. Eväkallio also talked why React will be around for a long time. It’s because components and interoperable components are an innovation primitive.

The New Best Practices

Declarative state and side effects — Christian Alfoni

After the keynote it was time to get more practical and Christian Alfoni talked about how we can get help writing our business logic in a declarative manner and see what benefits it gives us. He talked about lessons learned refactoring Codesandbox.io from Redux to Cerebral and about Cerebral which provides a declarative state and side effects management for popular JavaScript frameworks. Talks slides are available on the Internet.

Alfoni also pointed to Turning the database inside out with Apache Samza. Also that Cerebral had time travel before Dan Abramov presented Live React in his talk Hot Reloading with Time Travel at react-europe 2015

Immer: Immutability made easy — Michel Weststrate

Immutable data structures are a good thing and Michel Weststrate showed Immer which is a tiny package that allows you to work with immutable data structures with unprecedented ease. Managing the state of React app is a huge deal with Redux and any help is welcome. “Immer doesn’t require learning new data structures or update APIs, but instead creates a temporarily shadow tree which can be modified using the standard JavaScript APIs. The shadow tree will be used to generate your next immutable state tree.”

The talk showed how to write your reducers in a much more readable way, with half the code and without requiring additional large libraries. The talk slides are available on the Internet.

Get Rich Quick With React Context — Patrick Hund

“Get Rich Quick With React Context” lightning talk by Patrick Hund didn’t tell how good job opportunities you have when doing React But how with React 16.3 the context API has been completely revamped and demonstrated a good use case: Putting ad placements on your web page to get rich quick! Other use cases are localizations. Check out the slides which will tell you how easy it is to use context now and how to migrate your old context code to the new API.

There’s always a better way to handle localization — Eemeli Aro

“There’s always a better way to handle localization” lightning talk by Eemeli Aro told about how localization is a ridiculously difficult problem in the general case, but in the specific you can get away with really simple solutions, especially if you understand the compromises you’re making.

I must have been dozing as all I got was there are also other options to store localizations than JSON like YAML and JavaScript property format especially when dealing with non-developers like translators. The talk was quite general and on abstract level and mentioned solutions to localization were react-intl, react-i18next and react-message-context.

Styled Components, SSR, and Theming — Kasia Jastrzębska

Web applications need to be styled and Kasia Jastrzębska talked about CSS-in-JS with styled-components by going through the new API, performance improvements, server side rendering with Next.js. She also showed the theming manager available with v2 of styled-components. Talk slides are available on the Internet.

Takeaways from this talk was that CSS in React app can be written as you always have or by using CSS-in-JS solutions. There are several benefits of using styled-components but I’m still thinking how styles get scattered all over components.

Universal React Apps Using Next.js — Sia Karamalegos

53% of mobile site visits are abandoned if pages take longer than 3 seconds to load.
DoubleClick by Google, 2016

Every user’s hardware is different and processing speed can hinder user experience on client-side rendered React applications and so Sia Karamalegos talked how server-side rendering and code-splitting can drastically improve user experience. By minimizing the work that the client has to do. Performance and shipping your code matters. The talk showed how to easily build an universal React apps using the Next.js framework and walked through the concepts and code examples. Talk slides are available on the Internet.

There are lots of old (mobile) devices which especially benefit from Server Side Rendering. Next.js is a minimalistic framework for universal, server-rendered (or statically pre-rendered) React applications which enables faster page loads. Pages are server-rendered by default for initial load, you can enable prefetching future routes and there’s automatic code splitting. It’s also customizable so you can use own Babel and Webpack configurations and customize the server API with e.g. Express. And if you don’t want to use a server Next.js can also build static web apps that you can then host on Github pages or AWS S3.

Universal React apps using Next.js

State Management in React Apps with Apollo Client — Sara Vieira

Apollo Client was one of the most mentioned framework in the conference along with Reason ML and Sara Vieira gave energetic talk how to use it for state management in React Apps. If you haven’t come across Apollo Client it’s caching GraphQL client and helps you to manage data coming from the server. Virieira showed how to manage local state with apollo-link-state.

The talk was fast paced and I somewhat missed the why part but at least it’s easy to setup: yarn add apollo-boost graphql react-apollo. Have to see slides and demo later. Maybe the talk can be wrapped up to: “GQL all the things” and “I don’t like Redux” :D

State Management with Apollo

Detox: A year in. Building it, Testing with it — Rotem Mizrachi-Meidan

Detox testing framework for React Native talk by Rotem Mizrachi-Meidan was the other talk I dozed along. Mizrachi-Meidan talked what developing and using Detox in production has taught and how Detox works and what makes it deterministic. The talk showed how mobile apps could be tested. There’s a video of earlier talk on the Internet.

Detox

Make linting great again! — Andrey Okonetchnikov

One thing in software development which always gets developers to argue over stupid things is code formatting and linting. Andrey Okonetchnikov talked how “with a wrong workflow linting can be really a pain and will slow you and your team down but with a proper setup it can save you hours of manual work reformatting the code and reducing the code-review overhead.”

The talk was a quick introduction how 🚫💩 lint-staged a node.js library can improve developer experience. Small tool coupled with tools that analyze and improve the code like ESLint, Stylelint, Prettier and Jest can make a big difference.

Missed talks

There was also two talks I missed: “Understanding the differences is accepting” by Sven Sauleau and “Why I YAML” by Eemeli Aro. Sauleau showed “interesting” twists of Javascript language.

Read also the second part with more of React Native.

Monthly notes 29

This month’s notes are about front-end technologies: Sneak peek beyound React 16 and videos from Vue and Angular conferences. Also CSS Blocks + OptiCSS is great and for us in EU it’s nice that Fargate is finally available in Ireland. Check also list of important podcasts for software engineers.

Issue 29, 29.4.2018

Security

Computer security principles
One should keep in mind that there’s no such thing as perfect security. To put it another way, a 100% hack-safe systems do not exist. It’s all about the resources attacker(s) have, whether it is money, brain power, or equipment. Security standards and best practices changes quickly and therefore a system built five years ago is not inevitably conforming to current standards. So let’s look at some proactive measures that can be done to harden a system or code.

Cloud

AWS DevDay Munich 2018: Deep Dive into AWS Fargate (video)
“The session covers the state of containerized application development and deployment trends, new container capabilities on AWS that are available now, options for running containerized applications on AWS, and how AWS customers successfully run container workloads in production.” And for us in the EU Fargate is now available in Ireland.

Deploying FARGATE services using CloudFormation
“TL;DR — Deploying Fargate services is not as straight forward as you may think, especially if you’re used to the current EC2 configuration and are now trying to migrate running services. I had to go through a couple of days and few dozens of CloudFormation deployment iterations to figure out my missing / wrong settings before I made it through my first running Fargate container.”

CloudFormation Templates for AWS Fargate deployments
Sample CloudFormation templates for how to run Docker containers in AWS Fargate with various networking configurations.

Front-End

CSS at Scale: LinkedIn’s New Open Source Projects Take on Stylesheet Performance
“TL;DR: CSS Blocks + OptiCSS = :fire: So you get to write component-scoped CSS but end up with globally scoped, browser-friendly and compressed CSS classes (think atomic CSS). CSS Blocks does its magic with statically analyzing your markup and updating it with the new classes. It runs the OptiCSS as well, so you get tree-shaking and dead-code elimination also. Not 100% of the terms here, but basically unused code gets wiped.”

Sneak Peek: Beyond React 16
Intriguing ~30min talk with demos of what the future of React might look like showing off the new capabilities that async rendering unlocks for your components. Time Slicing lets you render and update large React component trees without blocking the user interactions. Suspense lets you render a component tree “in background” while components are fetching data, and display them only after the whole tree is ready. (from Twitter)

Talks from VueConf US 2018
The first ever VueConf US took place in New Orleans on March 26-28, 2018.

Talks from ng-conf 2018
Collection of all lectures that were presented during the conference. Each session includes a concise description and relevant slides.

To listen

Awesome list of Important Podcasts for software engineers
List of podcasts which are helpful for software engineers/programmers. Can’t pick any favorite from that list as I’m mostly listening to Finnish podcasts like ATK-hetki and Webbidevaus.fi.

Something different

“Never mad over what I can’t change. Never stressed over what I can’t control.”
– from Rhay1991

Designer Challenges Himself To Create Logos With Hidden Meanings For A Year, And Result Is Amazing
Clever.

Build Monitor with Raspberry Pi and Touch Screen

Information is a great tool in software development and it’s useful to have easy access to it. The more obvious you make your problems, the harder you make them to ignore. The more attention they get, the quicker they get solved. One thing developers like to monitor in software development is continuous integration status and metrics from running services. And what better way to achieve visibility and visualize to those metrics than building an information radiator.

I didn’t want to invent the wheel again so I got Raspberry Pi 3 Model B with accessories and 7″ touch screen to base my project. Using a Raspberry Pi as an information radiator isn’t a new idea and the Internet is full of examples of different adaptations with screens, lights, bells and whistles. For the start we just visualized our Jenkins builds and Grafana dashboard but later on we will propably do a custom dashboard.

Setting up the base

The information radiator is easy to get running as you only need a computer which preferably runs Linux. You can use an old laptop and attach it to external screen or if you’re like me and want to tinker you can get e.g. Raspberry Pi 3 and couple it with small external screen for portability. Nice and low cost solution which gets you some hacker value. I got the Rpi from our local hardware store and unfortunately the Model B+ was just released on the same day. The extra 15% power, 5 GHz Wifi and less heat and throttling would’ve been nice.

Raspberry Pi 3 Model B and accessories

I got the Raspberry Pi starting package with the official case, power supply, HDMI cable and a MicroSD card with preloaded NOOBS. So I just needed to connect the cables, put SD Card in and click to install Raspbian. Other interesting operating systems would’ve been Fedberry which is Fedora ‘Minimal, XFCE and LXQt’ Remixes.

For the screen I used 7″ IPS 5 point touch screen for Raspberry Pi with 1024×600 resolution and HDMI from joy-it codename RB-LCD-7-2. Initially I thought I could install the whole system with this display but as it turned out Rpi doesn’t understand it out of the box. It just showed some white noise and interference . Luckily some one had already solved this and I got the right config after I had installed Raspbian with real monitor.

Joy-it touch screen with default settings

Edit your /boot/config.txt:

# uncomment to force a specific HDMI mode (this will force VGA)
hdmi_group=2
hdmi_mode=87
 
# Add line:
hdmi_cvt=1024 600 60 3 0 0 0

And reboot your Raspberry Pi after those changes.

You should also run $ sudo raspi-config to setup for example WiFi country to allow channels 12 and 13 and your current Timezone.

I also updated Raspbian which bumps it to rpi-4.14.y linux tree:

$ sudo apt-get update
$ sudo apt-get upgrade
$ sudo apt-get dist-upgrade
$ sudo rpi-update

To connect to Rpi with SSH enable it with raspi-config > Interfacing Options or just:

$ sudo systemctl enable ssh
$ sudo systemctl start ssh

For the note, by default the user pi has password raspberry. You should change it but if you want to remove the nagging of default password do the following:

$ sudo apt-get -y purge pprompt
$ sudo rm /etc/profile.d/sshpwd.sh
$ sudo rm /etc/xdg/lxsession/LXDE-pi/sshpwd.sh

Problems with WiFi connection

I set up the Raspberry Pi at our local office and at home and there were no problems with WiFi connection. But when I brought it to customer premises the WiFi connection was weak and practically couldn’t move a bit. My MacBook worked fine but it was connected to 5 GHz network which isn’t an option with my Rpi 3 Model B. The WiFi on Rpi 3 was using channel 11 on 802.11i with WPA2 as shown with iwlist wlan0 scan.

There is a thread on Raspberry Pi forum about Very poor wifi performance which suggest to set up WiFi internalisation correctly to allow channels 12 and 13. At one point the issue was that only channels 1-11 are available on the Rpi 3 but checking out the ‘next’ branch of firmware/kernel (sudo BRANCH=next rpi-update) apparently fixed channels 12/13. I was on kernel 4.9.80 so it wasn’t a problem for me. The other suggested problem is with Atheros chipset based router which doesn’t like Broadcom WiFi on Rpi 3.

For some disabling power management solves the connection issues. For RPi built-in Broadcom (Cypress) WiFi there’s no control for power management and it’s disabled by the kernel. In iw / iwlist / iwconfig you see bug with “Power Management:on”.

But nevertheless testing switching it off made my WiFi connection better but it’s strength didn’t of course change.

$ sudo iwconfig wlan0 power off

To make it permanent you can add something like this in your interfaces file:

$ sudo touch /etc/network/if-up.d/wlan0
$ sudo chmod +x /etc/network/if-up.d/wlan0
$ sudo echo -e '#!/bin/bash\niwconfig wlan0 power off' > /etc/network/if-up.d/wlan0

Accessing Raspberry Pi remotely

The information radiator is usually connected to a TV with no keyboard or mouse attached so accessing it remotely is useful. You can use x11vnc which allows you to VNC into a headless Pi with a VNC client like Apple Remote Desktop, RealVNC’s vncviewer or homebrew’s tiger-vnc.

$ sudo apt-get install ttf-mscorefonts-installer
$ sudo apt-get install x11vnc

To start x11vnc automatically create new or edit existing ~/.xsessionrc file:

$ cat ~/.xsessionrc
/usr/bin/x11vnc -noxrecord -noxdamage -forever -bg -rfbport 5900

Getting interesting things on the screen

To test our setup and quickly show some data I just added a Build Monitor view in Jenkins and other view with Dashboard view. I also configured the Rpi to automatically start Chromium browser in kiosk mode after reboots and directed it to Jenkins website so there would be no need for interactions to get things on the screen. To show several sources of data and get things running quickly without customized information radiator we used Revolver – Tabs Chromium extension to rotate between multiple browser tabs: one showed Jenkins Build Monitor other Grafana Dashboard and third Twitter feed.

To automatically start the chromium-browser after Raspbian desktop starts, edit the following lxsession file:

$ cp /etc/xdg/lxsession/LXDE-pi/autostart /home/pi/.config/lxsession/LXDE-pi/autostart
$ vim /home/pi/.config/lxsession/LXDE-pi/autostart
 
#@xscreensaver -no-splash  # comment this line out to disable screensaver
# Disable Xsession from blanking
@xset s off
@xset -dpms
@xset s noblank
 
@sh ./autostart.sh
# load chromium after boot and point to the localhost webserver in full screen mode
@chromium-browser --kiosk --no-default-browser-check --no-first-run --disable-infobars "http://localhost/"

Chromium has a feature to show “Restore pages” nagging popup when not grafefully shutdown and you can try the following Stack Overflow suggestion. What was also suggested was doing “chmod 001 ~/.config/chromium/Default/Preferences” but it results to another nagging window.

$ cat ./autostart.sh
#!/bin/sh
sed -i 's/"exited_cleanly":false/"exited_cleanly":true/' ~/.config/chromium/'Local State'
sed -i 's/"exited_cleanly":false/"exited_cleanly":true/; s/"exit_type":"[^"]\+"/"exit_type":"Normal"/' ~/.config/chromium/Default/Preferences

You could use “–restore-last-session” or “–incognito” parameter which also works but it has several disadvantages, such as disabling the cache and login information. Or maybe I should just use Firefox.

Raspberry Pi and Jenkins Build Monitor

It might be also useful to set Chromium to restart every night. When running Chromium for longer periods it may fill Rpi’s memory with garbage and after it must be hard rebooted.

Turning the monitor on and off automatically

When running Rpi as a wall monitor it’s useful to save energy and extend the life of your monitor by turning the monitor on/off on a daily schedule. You can do this by running a cron script. Get this script and put it in /home/pi/rpi-hdmi.sh and make it executable: chmod +x /home/pi/rpi-hdmi.sh. Call the script at the desired time with cron entry:

$ crontab -e
 
# Turn HDMI Off (22:00)
0 17 * * * /home/pi/rpi-hdmi.sh off
 
# Turn HDMI On (7:00)
30 7 * * * /home/pi/rpi-hdmi.sh on

If you’ve problems with the above script try this which is the original but change “curr_vt=`fgconsole`” to be “curr_vt=`sudo fgconsole`” as fgconsole needs sudo privileges and otherwise you get an error “Couldn’t get a file descriptor referring to the console”.

From simple dashboard to real information Radiator

Showing just Jenkins Build Monitor or Grafana dashboards is simple but to get more information of things running you could show things like the success rate of the builds, build health, latest open pull request and project Twitter Messages. One nice example of information radiator is Panic’s Status board.

There are different ways to create customizable dashboard and one way is to use Dashing which is a dashboard framework and to get headstart you can see Project Dashboard https://github.com/martin-naumann/project-cockpit is which shows “build health” for the latest build, the latest open pull request in Github, the success rate of your builds, some free-form text and your project’s or company’s logo. It uses the Jenkins API to get the ration of successful / non-successful builds as well as the latest build state. It also calls the Github API to get the latest pull request to display the name and picture of the author and the title of the pull request on the dashboard.

For more leisure use you can set up the Raspberry Pi as a wall display to show information like calendar, weather, photos and RSS feeds. One option is to use Dakboard which is a web interface used to display information and is quite configurable with different services. At first Dakboard seems nice but is quite limited on what data it can show and some useful features are premium. Another open source option is MagicMirror² which seems to be more modular and extensible (as you can create your own modules) but needs more tinkering.

Monthly notes 28

Winter refuses to make way for the Spring and March in Southern Finland has been quite cold despite warm and rainy week which melted away some of the already scarce snow. So, skiing mainly on artificial snow and mountainbiking on icy paths which is nice. But this also leaves time to read what has happened on software development field. This month it’s about tools and working methods.

Issue 28: 17.3.2018

Tools of the trade

Must have extensions for VS Code (according to me)
tl;dr; Auto import, Beautif, Clipboard History, Git History, Code Runner, Docker, Material Icon Theme, Path Intellisense. (from @ThePracticalDev)

Reclaim your abandonware
Super cool post about how to get the abandoned mac Twitter client to support 280 character tweets by modifying its assembly. (from @b0rk)

Keep calm and code on: Productivity tools for developers
Suggestion of tools for developers for different tasks. I didn’t agree all of them so my suggestions are in brackets. tl;dr; 1. actiTIME (or toggl) 2. Cold Turkey (or other pomodoro) 3. Strict Workflow 4. Habitica 5. Oh My Zsh (or Bash-it) 6. The Silver Searcher 7. UltraEdit (or Atom, VS Code etc.) 8. Homebrew 9. GitHub Changelog Generator. (from @ThePracticalDev)

Git aliases
If you use Git command-line a lot, you will probably grow your own list of Git aliases sooner or later. After simple standard aliases (ci -> commit, co -> checkout) you might want to see some advanced tricks you may find useful.

CTFR
Get subdomains of an HTTPS website abusing Certificate Transparency logs. (from @KitPloit). Apparently also curl "https://crt.sh/?q=%.starbucks.com&output=json" -sS | jq .name_value | uniq | tee output works.

Front-end

Front-End Performance Checklist 2018
Unbiased and objective front-end performance checklist for 2018 — an overview of the issues you might need to consider to ensure that your response times are fast, user interaction is smooth and your sites don’t drain user’s bandwidth. (from @igrigorik)

Working methdods

Tim Ottinger: what once was thought impossibility is now commonplace in software development
TDD, pairing, mobbing, evolutionary design, self organizing, lean startup, commenting code, interpreted languages, beta, noestimates. “You have to ask, what impossible thing is going to be done next? We change how we think, and new vistas open up.”

Branching Is Easy. So? Git-flow Is Not Agile.
I wrote this blog post ages ago and I still stand firmly behind it trunk-based development 4ever. (from @skamille)

Getting Things Done – A Programmer Productivity Guide
“Everybody has some sort of system—even not having a system and trying to remember everything is technically a system. I wanted to share mine because it seems to work pretty well.” (from @ThePracticalDev)

Herding cats is easy compared to managing developers (video)
A short and sharp 10 minute guide to managing developers by Dom Millar at NDC Conference Sydney 2017.

Something different

10 x weekend brunches in Helsinki
Il Birricifio, Ipi Kulmakuppila, Sandro, Gastro Café Kallio, Yes Yes Yes!, Loop, Moko Market, Sue Ellen, Paulig Kulma and Krog Roba. My addition to the list is Rupla. (from @VisitHelsinki)

Monthly notes 27

For cold winter evenings here’s something to read. Monthly notes for February are about relearning and thinking.

Issue 27: 23.2.2018

Relearning

Computer Science and why it’s necessary even for web developers
“Computer Science and why it’s necessary even for web developers I know that in some countries a degree in CS is expensive or unattainable, and that some companies do unnecessary algorithm interviews. This thread is not about degrees or interviews, it’s about CS itself.”

Free Intro to Web Development slides (with demos)
Slides of the Web Dev Intro labs for the “6.813 User Interface Design and Implementation” at MIT
(from Twitter)

The Four Rules of Simple Design (in order of importance)

  • Passes the tests
  • Reveals intention
  • No duplication
  • Fewest elements

And, yes, “fewest elements” is last, which means you only minimize classes and methods if everything else satisfied

Tools

shuttle
When openssh port forwarding doesn’t cut it, use sshuttle: “Transparent proxy meets VPN meets ssh.”

Microservices

The Death of Microservice Madness in 2018
There are many cases where great efforts have been made to adopt microservice patterns without necessarily understanding how the costs and benefits will apply to the specifics of the problem at hand. The post describes in detail what microservices are, why the pattern is so appealing, and also some of the key challenges that they present.

Should that be a Microservice? Keep These Six Factors in Mind
These days, you can’t swing a dry erase marker without hitting someone talking about microservices but few have spent any appreciable time asking if a given application should be a microservice. tl;dr; “1. Multiple Rates of Change; 2. Independent Life Cycles; 3. Independent Scalability; 4. Isolated Failure; 5. Simplify Interactions with External Dependencies; 6. The Freedom to Choose the Right Tech for the Job”.

JavaScript

A Guide to Web Performance Optimization with Webpack
This guide walks through how to effectively optimize site resources using webpack. This can help users load and interact with your sites more quickly. (from JavaScript Weekly 373)

Security

face-verify.js: Monitoring who is physically looking at a website for additional security
Demo project showing how Machine Box tech can be integrated into JavaScript applications. Facebox takes an image and tells you how many faces it sees, as well as who those faces belong to provided you have shown it a single example previously. You can use this capability to build additional security into web apps so you can see how many people are watching the screen and who they are. Using the webcam with some JavaScript and Facebox, you can periodically check to ensure only authorised people can see the information that users consider sensitive.

Mac Privacy: Sandboxed Mac apps can record your screen at any time without you knowing
TL;DR Any Mac app can take screenshots of your Mac silently, and use basic OCR software to read all text on the screen. (from Weekend Reading)

To think about

Nick Stenning on Twitter
“Flat organisational structures do not exist. There are only organisations with visible structure and organisations with invisible structure”. (from Weekend Reading)

Developers On Call
Quite self-explanatory ideas for how to manage on-call rotations without burn out but maybe it’s not always that way. The linked Twitter thread is worth reading. (from Weekend Reading)

Something different

2017: The Year in Charts
These are the charts and themes that tell the story of 2017. I. The Year Volatility Died; II. Records Are Made to Be Broken; III. The World is Flattening; IV. Still Easy After All These Years; V. A Good Old-Fashioned Mania; VI. King Dollar Dethroned; VII. Wrapping Up: 1991-99 Redux?