The first part of Summer has been great and holiday season is near. Here’s monthly notes for June with topics of microservices, kubernetes, design patterns and stories of how Shopify and Airbnb build their services. Also some tools like Kap. Happy reading.
Issue 31, 28.6.2018
Microservices
7 tips for effective microservices
“Have a request-id/correlation-id for every request, Maintain backward compatibility of interfaces, Have a centralized logging system, Implement idempotency and retries, Be aware of language constraints, Have a single service to manage the system state, Strike a balance between in-memory-data and db persistence” (from The Microservice Weekly)
Kubernetes
AWS Workshop for Kubernetes
“Self-paced workshop designed for Development and Operations teams who would like to leverage Kubernetes on Amazon Web Services (AWS).”
Kubernetes best practices: terminating with grace
“This episode of “Kubernetes Best Practices,” let’s take a look at how you can help Kubernetes do its job more efficiently and reduce the downtime your applications experience.”
xcprojectlint: A security blanket for Xcode project files
Would you like to automate some consistency in your Xcode project files with checks for settings defined at the project level (rather than in an xcconfig), missing files and empty file groups? This tool does exactly that, and more. Also, I like the way it’s described: “Provides a security blanket, ensuring neither your co-workers, nor git screw up your Xcode project file.” (from iOS Dev Weekly 353)
Shopify Infrastructure with Niko Kurtti
“Shopify has built its own platform-as-a-service on top of Kubernetes called Cloudbuddies. Niko Kurtti is a production engineer at Shopify joins the Software Engineering Daily show to describe Shopify’s infrastructure – how they run so many stores, how they distribute those stores across their infrastructure, and the motivation for building their own internal platform on top of Kubernetes.”
Building Services at Airbnb, Part 1
The first in a series on scaling service development, this article looks at the core structure, the Service IDL, underpinning the new Services Oriented Architecture at Airbnb.
Building Services at Airbnb, Part 2
The second in a series on scaling service development, this article looks at some of the key tooling that supports the new Services Oriented Architecture at Airbnb.
Awesome design patterns
A curated list of software and architecture related design patterns. Software design pattern – A general, reusable solution to a commonly occurring problem within a given context in software design. It is a description or template for how to solve a problem that can be used in many different situations.
Something Different
Cool Backgrounds
Collection of tools to create compelling, colorful images for blogs, social media, and websites. Beyond backgrounds, the images generated can be used as ? desktop wallpapers or cropped for ? mobile wallpapers.
OWASP Helsinki Chapter held a meeting number 34 last week at Eficode with topics of
“Perfectly secure API” and “Best friends: API security & API management”. The event gave good overview to the topics covered and was quite packed with people. Eficode’s premises were modern and there was snacks and beverages. And also a sauna. Here is a short recap of the talks.
OWASP Helsinki Chapter Meeting 34
Perfectly secure API
Matti Suominen from Nixu talked about perfectly secure API and things related to get there. Can API be secure? On gut feeling APIs seems to be rubbish and have problems. He covered the topic from three view points: security, risks and defense. Good starting point is to read OWASP resources like ASVS, Top 10 and Security cheat sheet. Also implement security centrally, involve business in design and DIY never works out.
Antti Virtanen from Solita talked about API security and API management and how we’ve traveled from dark ages to modern times. You can do API security with tools like Amazon AWS API Gateway but the main point was to step further with API management. Use some already made products like Apigee and open source alternative Tyk.io. Slides are available in Slideshare.
From dark ages to modern ages and better API security with #API management as @anakonantti from @SolitaOy showed at #OWASPHelsinki meetup. #Apigee is one solution. Step further from API gateway, adds DX and publishing tools. Open source alternative is @tyk_io but it’s complex. pic.twitter.com/re69veD9bx
Summer is approaching and even in Finland the weather is sunny and warm. I’ve been busy as the Enduro-MTB racing season has started and most weekends are spent at the race track. But here’s monthly notes for May with topics of state of the Web, how geolocation in browsers work, and something about tools. Happy reading.
Ever stop to think about geolocation in your desktop browser?
tl;dr; location is triangulated by location services like Mozilla & Google from scan of nearby Wi-Fi access points’ signal strength and their known locations. List is collected when people walk with a phone with GPS and Wi-Fi on which polls networks. (from @walokra)
Is GraphQL The Future?
If you are not sold on GraphQL then this post might tip you over the edge. Alan Johnson does a great job in explaining the awesomeness that GraphQL has to offer. (from Web Design Weekly 321)
Tools for making HTTP requests
Imsomnia
Powerful HTTP and GraphQL tool belt. Debug APIs like a human, not a robot. Finally, a REST client you’ll love.
RESTed – Simple HTTP Requests
RESTed allows developers to quickly format and make HTTP requests and view the response. For Mac.
Paw
Paw is a full-featured HTTP client that lets you test and describe the APIs you build or consume. It has a beautiful native macOS interface to compose requests, inspect server responses, generate client code and export API definitions.
Security
US cell carriers are selling access to real-time phone location data
Intriguing thoughts: “Access to your real-time phone location data is sold to companies and public has zero idea how much personal location data is available. It is done throughout the industrialized world to varying degrees.” e.g. stocks are traded based on where peoole go. (from @walokra)
JavaScript
`npm audit`: identify and fix insecure dependencies
“npm audit is a new command that performs a moment-in-time security review of your project’s dependency tree. Audit reports contain information about security vulnerabilities in your dependencies and can help you fix a vulnerability by providing simple-to-run npm commands and recommendations for further troubleshooting.” (from JavaScript Daily)
JavaScript Algorithms and Data Structures
A wide variety of algorithms (e.g. permutations, Levenshtein distance, binary search) and data structures (e.g. linked lists, trees, stacks) implemented in JavaScript with explanations and links to further reading. (from JavaScript Weekly 387)
Unchained: A story of love, loss, and blockchain
> It was a smart contract that stipulated sexual fidelity and parental responsibilities. Tokens from their joint earnings paid the AI judges and IoT sensor oracles that monitored contract violations. On mornings like this, you really needed commitment that was mathematically provable, not just an empty promise at the altar.
React Finland 2018 conference was held last week and I had the opportunity to attend it and listen what’s hot in the React world. The conference started with workshops and after that there was two days of talks of React, React Native, React VR and all things that go with developing web applications with them. The two conference days were packed with great talks and new information. This is the second part of my recap of the talks and my notes which I posted to Twitter. Check out also the first part of my notes from the first day’s talks.
React Finland 2018, Day 2
How React changed everything — Ken Wheeler
Second day started with keynote by Ken Wheeler. He examined how React changed the front end landscape as we know it and started it with nice time travel to the 90s with i.a. Flash, JavaScript and AngularJS. Most importantly the talk took a look at the core idea of React, and why it transcends language or rendering target and posit on what that means going forward. And last we heard about what React async: suspense and time slicing.
“Best part of React is the community”
How React changed everything
Get started with Reason — Nik Graf
The keynote also touched Reason ML and Nik Graf went into details kicking off with the basics and going into how to leverage features like variant types and pattern matching to make impossible states impossible.
Get started with Reason ML
Making Unreasonable States Impossible — Patrick Stapfer
Based on “Get started with Reason” Patrick Stapfer’s talk went deeper into the world of variant types and pattern matching and put them into a practical context. The talk was nice learning by doing TicTacToe live coding. It showed how Reason ML helps you design solid APIs, which are impossible to misuse by consumers. We also got more insights into practical ReasonReact code. Presentation is available on the Internet.
Conclusion about ReasonReact:
More rigid design
More KISS (keep it simple, stupid) than DRY (don’t repeat yourself)
Forces edge-cases to be handled
Learning Reason by doing TicTacToe
Reactive State Machines and Statecharts — David Khourshid
David Khourshid’s talk about state machines and statecharts was interesting. Functional + reactive approach to state machines can make it much easier to understand, visualize, implement and automatically create tests for complex user interfaces and flows. Model the code and automatically generate exhaustive tests for every possible permutation of the code. Things mentioned: React automata, xstate. Slides are available on the Internet.
“Model once, implement anywhere” – David Khourshid
The talk was surprisingly interesting especially for use cases as anything to make testing better is good. This might be something to look into.
Compelling use case for state machines is: model code & automatically generate exhaustive tests for every possible permutation of the code. @DavidKPiano and surprisingly interesting talk of Reactive State Machines and Statechart at #ReactFinland. "Model once, implement anywhere" pic.twitter.com/v5iynBA4te
After theory heavy presentations we got into more visual stuff: React VR. Shay Keinan presented the core concepts behind VR, showed different demonstrations, and how to get started with React VR and how to add new features from the Three.js library. React VR: Three.js + React Native = 360 and VR content. On the VR device side it was mentioned that Oculus Go, HTC Vive Focus are the big step to Virtual Reality.
“Virtual Reality’s possibilities are endless. Compares to lucid dreaming.” – Shay Keinan
WebVR enables web developers to create frictionless, immersive experiences and we got to see Solar demo and Three VR demo which were lit ?.
React VR
World Class experience with React Native — Michał Chudziak
I’ve shortly experimented with React Native so it was nice to listen Michał Chudziak’s talk how to set up a friendly React Native development environment with the best DX, spot bugs in early stage and deliver continuous builds to QA. Again Redux was dropped in favour of apollo-link-state.
Work close to your team – Napoleon Hill
What makes a good Developer eXperience?
stability
function
clarity
easiness
GraphQL was mentioned to be the holy grail of frontend development and perfect with React Native. Tools for better developer experience: Haul, CircleCI, Fastlane, ESLint, Flow, Jest, Danger, Detox. Other tips were i.a to use native IDEs (XCode, Android Studio) as it helps debugging. XCode Instruments helps debug performance (check iTunes for video) and there’s also Android Profiler.
World Class experience with React Native
React Finland App – Lessons learned — Toni Ristola
Every conference has to have an app and React Finland of course did a React Native app. Toni Ristola lightning talked about lessons learned. Technologies used with React Native was Ignite, GraphQL and Apollo Client ? App’s source code is available on GitHub.
Lessons learned:
Have a designer in the team
Reserve enough time — doing and testing a good app takes time
Test with enough devices — publish alpha early
React Finland App – Lessons learned
React Native Ignite — Gant Laborde
80% of mobile app development is the same old song which can be cut short with Ignite CLI. Using Ignite, you can jump into React Native development with a popular combination of technologies, OR brew your own. Gant Laborde talked about the new Bowser version which makes things even better with Storybook, Typescript, Solidarity, mobx-state-tree and lint-staged. Slides can be found on the Internet.
Ignite
How to use React, webpack and other buzzwords if there is no need — Varya Stepanova
Varya Stepanova’s lightning talk suggested to start a side-project other than ToDo app to study new development approaches and showed what it can be in React. The example was how to generate a multilingual static website using Metalsmith, React and other modern technologies and tools which she uses to build her personal blog. Slides can be found on the Internet.
Doing meaningful side-projects is a great idea to study new things and I’ve used that for i.a. learning Swift with Highkara newsreader, did couple of apps for Sailfish OS and played with GraphQL and microservices while developing app with largish vehicle dataset.
Two days full of talks of React, React Native, React VR and all the things that go with developing web applications with them was great experience. Days were packed with great talks, new information and everything went smoothly. The conference was nicely organized, food was good and participants got soft hoodies to go with the Allas Sea Pool ticket. The talks were all great but especially “World Class experience with React Native” and “React Native Ignite” gave new inspiration to write some app. Also “ReactVR” seemed interesting although I think Augmented Reality will be bigger thing than Virtual Reality. It was nice to hear from “The New Best Practices” talk that there really is no new best practices as the old ones still work. Just use them!
Something to try and even to take into production will be Immer, styled components and Next.js. One thing which is easy to implement is to start using lint-staged although we are linting all the things already.
One of the conference organizers and speaker, Juho Vepsäläinen, wrote Lessons Learned from the conference and many of the points he mentions are to the point. The food was nice but “there wasn’t anything substantial for the afternoon break”. There wasn’t anything to eat after lunch but luckily I had own snacks. Vepsäläinen also mentions that “there was sometimes too much time between the presentations” but I think the longer breaks between some presentations were nice for having a quick stroll outside and have some fresh air. The venue was quite warm and the air wasn’t so good in the afternoon.
The Afterparty at Sea Life Helsinki was interesting choice and it worked nicely although there wasn’t so many people there. The aquarium was fishy experience and provided also some other content than refreshments. Too bad I hadn’t have time to go and check the Allas Sea Pool which we got a free ticket. Maybe next time.
Thanks to the conference crew for such a good event and of course to my fellow Goforeans which attended it and had a great time!
React Finland 2018 conference was held last week and I had the opportunity to attend it and listen what’s hot in the React world. The conference started with workshops and after that there was two days of talks of React, React Native, React VR and all things that go with developing web applications with them. The two conference days were packed with great talks and new information. Here’s the first part of my notes from the talks which I posted to Twitter. Read also the second part with more of React Native.
React Finland 2018: Day 1
React Finland combined the Finnish React community with international flavor from Jani Eväkallio to Ken Wheeler and other leading talents of the community. The event was the first of its kind in Finland and consisted of a workshop day and two days of talks around the topic. It was nice that the event was single track so you didn’t need to choose between interesting talks.
At work I’ve been developing with React couple of years and tried my hands with React Native so the topics were familiar. The conference provided crafty new knowledge to learn from and maybe even put to production. Overall the conference was great experience and everything went smoothly. Nice work from the React Finland conference team! And of course thanks to Gofore which sponsored the conference and got me a ticket.
I tweeted my notes from almost every presentation and here’s a recap of the talks. I heard that the videos from the conference will be available shortly.
The New Best Practices — Jani Eväkallio
First day’s keynote was by Jani Eväkallio who talked about “The New Best Practices”. As the talk description wrote “When React was first introduced, it was ridiculed for going against established web development best practices as we knew them. Five years later, React is the gold standard for how we create user interfaces. Along the way, we’ve discovered a new set of tools, design patterns and programming techniques.”
The new best practices were:
Build big things from small things
Write code for humans first: flow, Typescript, storybook
Stay close to the language:
helps i.a. linters
Always prefer simplicity
Don’t break things:
Facebook makes React API changes easy to upgrade, depreciation well in advance, migration, documentation. it’s a flow, not versions. Use codemod.
Keep an open mind
You ask “what new best practices”? Yep, that’s the thing. We don’t need new best practices as the same concepts like Model-View-Controller and separation of concerns are still valid. We should use best practices which have been proved good before as they also work nicely with React philosophy. Eväkallio also talked why React will be around for a long time. It’s because components and interoperable components are an innovation primitive.
The New Best Practices
Declarative state and side effects — Christian Alfoni
Immutable data structures are a good thing and Michel Weststrate showed Immer which is a tiny package that allows you to work with immutable data structures with unprecedented ease. Managing the state of React app is a huge deal with Redux and any help is welcome. “Immer doesn’t require learning new data structures or update APIs, but instead creates a temporarily shadow tree which can be modified using the standard JavaScript APIs. The shadow tree will be used to generate your next immutable state tree.”
The talk showed how to write your reducers in a much more readable way, with half the code and without requiring additional large libraries. The talk slides are available on the Internet.
“Get Rich Quick With React Context” lightning talk by Patrick Hund didn’t tell how good job opportunities you have when doing React But how with React 16.3 the context API has been completely revamped and demonstrated a good use case: Putting ad placements on your web page to get rich quick! Other use cases are localizations. Check out the slides which will tell you how easy it is to use context now and how to migrate your old context code to the new API.
There’s always a better way to handle localization — Eemeli Aro
“There’s always a better way to handle localization” lightning talk by Eemeli Aro told about how localization is a ridiculously difficult problem in the general case, but in the specific you can get away with really simple solutions, especially if you understand the compromises you’re making.
I must have been dozing as all I got was there are also other options to store localizations than JSON like YAML and JavaScript property format especially when dealing with non-developers like translators. The talk was quite general and on abstract level and mentioned solutions to localization were react-intl, react-i18next and react-message-context.
Styled Components, SSR, and Theming — Kasia Jastrzębska
Takeaways from this talk was that CSS in React app can be written as you always have or by using CSS-in-JS solutions. There are several benefits of using styled-components but I’m still thinking how styles get scattered all over components.
Universal React Apps Using Next.js — Sia Karamalegos
53% of mobile site visits are abandoned if pages take longer than 3 seconds to load. – DoubleClick by Google, 2016
Every user’s hardware is different and processing speed can hinder user experience on client-side rendered React applications and so Sia Karamalegos talked how server-side rendering and code-splitting can drastically improve user experience. By minimizing the work that the client has to do. Performance and shipping your code matters. The talk showed how to easily build an universal React apps using the Next.js framework and walked through the concepts and code examples. Talk slides are available on the Internet.
There are lots of old (mobile) devices which especially benefit from Server Side Rendering. Next.js is a minimalistic framework for universal, server-rendered (or statically pre-rendered) React applications which enables faster page loads. Pages are server-rendered by default for initial load, you can enable prefetching future routes and there’s automatic code splitting. It’s also customizable so you can use own Babel and Webpack configurations and customize the server API with e.g. Express. And if you don’t want to use a server Next.js can also build static web apps that you can then host on Github pages or AWS S3.
Universal React apps using Next.js
State Management in React Apps with Apollo Client — Sara Vieira
Apollo Client was one of the most mentioned framework in the conference along with Reason ML and Sara Vieira gave energetic talk how to use it for state management in React Apps. If you haven’t come across Apollo Client it’s caching GraphQL client and helps you to manage data coming from the server. Virieira showed how to manage local state with apollo-link-state.
The talk was fast paced and I somewhat missed the why part but at least it’s easy to setup: yarn add apollo-boost graphql react-apollo. Have to see slides and demo later. Maybe the talk can be wrapped up to: “GQL all the things” and “I don’t like Redux” :D
State Management with Apollo
Detox: A year in. Building it, Testing with it — Rotem Mizrachi-Meidan
Detox testing framework for React Native talk by Rotem Mizrachi-Meidan was the other talk I dozed along. Mizrachi-Meidan talked what developing and using Detox in production has taught and how Detox works and what makes it deterministic. The talk showed how mobile apps could be tested. There’s a video of earlier talk on the Internet.
Detox
Make linting great again! — Andrey Okonetchnikov
One thing in software development which always gets developers to argue over stupid things is code formatting and linting. Andrey Okonetchnikov talked how “with a wrong workflow linting can be really a pain and will slow you and your team down but with a proper setup it can save you hours of manual work reformatting the code and reducing the code-review overhead.”
The talk was a quick introduction how ?? lint-staged a node.js library can improve developer experience. Small tool coupled with tools that analyze and improve the code like ESLint, Stylelint, Prettier and Jest can make a big difference.
Missed talks
There was also two talks I missed: “Understanding the differences is accepting” by Sven Sauleau and “Why I YAML” by Eemeli Aro. Sauleau showed “interesting” twists of Javascript language.
This month’s notes are about front-end technologies: Sneak peek beyound React 16 and videos from Vue and Angular conferences. Also CSS Blocks + OptiCSS is great and for us in EU it’s nice that Fargate is finally available in Ireland. Check also list of important podcasts for software engineers.
Issue 29, 29.4.2018
Security
Computer security principles
One should keep in mind that there’s no such thing as perfect security. To put it another way, a 100% hack-safe systems do not exist. It’s all about the resources attacker(s) have, whether it is money, brain power, or equipment. Security standards and best practices changes quickly and therefore a system built five years ago is not inevitably conforming to current standards. So let’s look at some proactive measures that can be done to harden a system or code.
Cloud
AWS DevDay Munich 2018: Deep Dive into AWS Fargate (video)
“The session covers the state of containerized application development and deployment trends, new container capabilities on AWS that are available now, options for running containerized applications on AWS, and how AWS customers successfully run container workloads in production.” And for us in the EU Fargate is now available in Ireland.
Deploying FARGATE services using CloudFormation
“TL;DR — Deploying Fargate services is not as straight forward as you may think, especially if you’re used to the current EC2 configuration and are now trying to migrate running services. I had to go through a couple of days and few dozens of CloudFormation deployment iterations to figure out my missing / wrong settings before I made it through my first running Fargate container.”
CSS at Scale: LinkedIn’s New Open Source Projects Take on Stylesheet Performance
“TL;DR: CSS Blocks + OptiCSS = :fire: So you get to write component-scoped CSS but end up with globally scoped, browser-friendly and compressed CSS classes (think atomic CSS). CSS Blocks does its magic with statically analyzing your markup and updating it with the new classes. It runs the OptiCSS as well, so you get tree-shaking and dead-code elimination also. Not 100% of the terms here, but basically unused code gets wiped.”
Sneak Peek: Beyond React 16
Intriguing ~30min talk with demos of what the future of React might look like showing off the new capabilities that async rendering unlocks for your components. Time Slicing lets you render and update large React component trees without blocking the user interactions. Suspense lets you render a component tree “in background” while components are fetching data, and display them only after the whole tree is ready. (from Twitter)
Talks from ng-conf 2018
Collection of all lectures that were presented during the conference. Each session includes a concise description and relevant slides.
Information is a great tool in software development and it’s useful to have easy access to it. The more obvious you make your problems, the harder you make them to ignore. The more attention they get, the quicker they get solved. One thing developers like to monitor in software development is continuous integration status and metrics from running services. And what better way to achieve visibility and visualize to those metrics than building an information radiator.
I didn’t want to invent the wheel again so I got Raspberry Pi 3 Model B with accessories and 7″ touch screen to base my project. Using a Raspberry Pi as an information radiator isn’t a new idea and the Internet is full of examples of different adaptations with screens, lights, bells and whistles. For the start we just visualized our Jenkins builds and Grafana dashboard but later on we will propably do a custom dashboard.
Setting up the base
The information radiator is easy to get running as you only need a computer which preferably runs Linux. You can use an old laptop and attach it to external screen or if you’re like me and want to tinker you can get e.g. Raspberry Pi 3 and couple it with small external screen for portability. Nice and low cost solution which gets you some hacker value. I got the Rpi from our local hardware store and unfortunately the Model B+ was just released on the same day. The extra 15% power, 5 GHz Wifi and less heat and throttling would’ve been nice.
Raspberry Pi 3 Model B and accessories
I got the Raspberry Pi starting package with the official case, power supply, HDMI cable and a MicroSD card with preloaded NOOBS. So I just needed to connect the cables, put SD Card in and click to install Raspbian. Other interesting operating systems would’ve been Fedberry which is Fedora ‘Minimal, XFCE and LXQt’ Remixes.
For the screen I used 7″ IPS 5 point touch screen for Raspberry Pi with 1024×600 resolution and HDMI from joy-it codename RB-LCD-7-2. Initially I thought I could install the whole system with this display but as it turned out Rpi doesn’t understand it out of the box. It just showed some white noise and interference . Luckily some one had already solved this and I got the right config after I had installed Raspbian with real monitor.
Joy-it touch screen with default settings
Edit your /boot/config.txt:
# uncomment to force a specific HDMI mode (this will force VGA)
hdmi_group=2
hdmi_mode=87
# Add line:
hdmi_cvt=1024 600 60 3 0 0 0
# uncomment to force a specific HDMI mode (this will force VGA)
hdmi_group=2
hdmi_mode=87
# Add line:
hdmi_cvt=1024 600 60 3 0 0 0
And reboot your Raspberry Pi after those changes.
You should also run $ sudo raspi-config to setup for example WiFi country to allow channels 12 and 13 and your current Timezone.
I also updated Raspbian which bumps it to rpi-4.14.y linux tree:
For the note, by default the user pi has password raspberry. You should change it but if you want to remove the nagging of default password do the following:
I set up the Raspberry Pi at our local office and at home and there were no problems with WiFi connection. But when I brought it to customer premises the WiFi connection was weak and practically couldn’t move a bit. My MacBook worked fine but it was connected to 5 GHz network which isn’t an option with my Rpi 3 Model B. The WiFi on Rpi 3 was using channel 11 on 802.11i with WPA2 as shown with iwlist wlan0 scan.
There is a thread on Raspberry Pi forum about Very poor wifi performance which suggest to set up WiFi internalisation correctly to allow channels 12 and 13. At one point the issue was that only channels 1-11 are available on the Rpi 3 but checking out the ‘next’ branch of firmware/kernel (sudo BRANCH=next rpi-update) apparently fixed channels 12/13. I was on kernel 4.9.80 so it wasn’t a problem for me. The other suggested problem is with Atheros chipset based router which doesn’t like Broadcom WiFi on Rpi 3.
For some disabling power management solves the connection issues. For RPi built-in Broadcom (Cypress) WiFi there’s no control for power management and it’s disabled by the kernel. In iw / iwlist / iwconfig you see bug with “Power Management:on”.
But nevertheless testing switching it off made my WiFi connection better but it’s strength didn’t of course change.
$ sudo iwconfig wlan0 power off
$ sudo iwconfig wlan0 power off
To make it permanent you can add something like this in your interfaces file:
The information radiator is usually connected to a TV with no keyboard or mouse attached so accessing it remotely is useful. You can use x11vnc which allows you to VNC into a headless Pi with a VNC client like Apple Remote Desktop, RealVNC’s vncviewer or homebrew’s tiger-vnc.
To test our setup and quickly show some data I just added a Build Monitor view in Jenkins and other view with Dashboard view. I also configured the Rpi to automatically start Chromium browser in kiosk mode after reboots and directed it to Jenkins website so there would be no need for interactions to get things on the screen. To show several sources of data and get things running quickly without customized information radiator we used Revolver – Tabs Chromium extension to rotate between multiple browser tabs: one showed Jenkins Build Monitor other Grafana Dashboard and third Twitter feed.
To automatically start the chromium-browser after Raspbian desktop starts, edit the following lxsession file:
$ cp /etc/xdg/lxsession/LXDE-pi/autostart /home/pi/.config/lxsession/LXDE-pi/autostart
$ vim /home/pi/.config/lxsession/LXDE-pi/autostart
#@xscreensaver -no-splash # comment this line out to disable screensaver
# Disable Xsession from blanking
@xset s off
@xset -dpms
@xset s noblank
@sh ./autostart.sh
# load chromium after boot and point to the localhost webserver in full screen mode
@chromium-browser --kiosk --no-default-browser-check --no-first-run --disable-infobars "http://localhost/"
$ cp /etc/xdg/lxsession/LXDE-pi/autostart /home/pi/.config/lxsession/LXDE-pi/autostart
$ vim /home/pi/.config/lxsession/LXDE-pi/autostart
#@xscreensaver -no-splash # comment this line out to disable screensaver
# Disable Xsession from blanking
@xset s off
@xset -dpms
@xset s noblank
@sh ./autostart.sh
# load chromium after boot and point to the localhost webserver in full screen mode
@chromium-browser --kiosk --no-default-browser-check --no-first-run --disable-infobars "http://localhost/"
Chromium has a feature to show “Restore pages” nagging popup when not grafefully shutdown and you can try the following Stack Overflow suggestion. What was also suggested was doing “chmod 001 ~/.config/chromium/Default/Preferences” but it results to another nagging window.
$ cat ./autostart.sh
#!/bin/sh
sed -i 's/"exited_cleanly":false/"exited_cleanly":true/' ~/.config/chromium/'Local State'
sed -i 's/"exited_cleanly":false/"exited_cleanly":true/; s/"exit_type":"[^"]\+"/"exit_type":"Normal"/' ~/.config/chromium/Default/Preferences
$ cat ./autostart.sh
#!/bin/sh
sed -i 's/"exited_cleanly":false/"exited_cleanly":true/' ~/.config/chromium/'Local State'
sed -i 's/"exited_cleanly":false/"exited_cleanly":true/; s/"exit_type":"[^"]\+"/"exit_type":"Normal"/' ~/.config/chromium/Default/Preferences
You could use “–restore-last-session” or “–incognito” parameter which also works but it has several disadvantages, such as disabling the cache and login information. Or maybe I should just use Firefox.
Raspberry Pi and Jenkins Build Monitor
It might be also useful to set Chromium to restart every night. When running Chromium for longer periods it may fill Rpi’s memory with garbage and after it must be hard rebooted.
Turning the monitor on and off automatically
When running Rpi as a wall monitor it’s useful to save energy and extend the life of your monitor by turning the monitor on/off on a daily schedule. You can do this by running a cron script. Get this script and put it in /home/pi/rpi-hdmi.sh and make it executable: chmod +x /home/pi/rpi-hdmi.sh. Call the script at the desired time with cron entry:
$ crontab -e
# Turn HDMI Off (22:00)
0 17 * * * /home/pi/rpi-hdmi.sh off
# Turn HDMI On (7:00)
30 7 * * * /home/pi/rpi-hdmi.sh on
$ crontab -e
# Turn HDMI Off (22:00)
0 17 * * * /home/pi/rpi-hdmi.sh off
# Turn HDMI On (7:00)
30 7 * * * /home/pi/rpi-hdmi.sh on
If you’ve problems with the above script try this which is the original but change “curr_vt=`fgconsole`” to be “curr_vt=`sudo fgconsole`” as fgconsole needs sudo privileges and otherwise you get an error “Couldn’t get a file descriptor referring to the console”.
From simple dashboard to real information Radiator
Showing just Jenkins Build Monitor or Grafana dashboards is simple but to get more information of things running you could show things like the success rate of the builds, build health, latest open pull request and project Twitter Messages. One nice example of information radiator is Panic’s Status board.
There are different ways to create customizable dashboard and one way is to use Dashing which is a dashboard framework and to get headstart you can see Project Dashboard https://github.com/martin-naumann/project-cockpit is which shows “build health” for the latest build, the latest open pull request in Github, the success rate of your builds, some free-form text and your project’s or company’s logo. It uses the Jenkins API to get the ration of successful / non-successful builds as well as the latest build state. It also calls the Github API to get the latest pull request to display the name and picture of the author and the title of the pull request on the dashboard.
For more leisure use you can set up the Raspberry Pi as a wall display to show information like calendar, weather, photos and RSS feeds. One option is to use Dakboard which is a web interface used to display information and is quite configurable with different services. At first Dakboard seems nice but is quite limited on what data it can show and some useful features are premium. Another open source option is MagicMirror² which seems to be more modular and extensible (as you can create your own modules) but needs more tinkering.
Winter refuses to make way for the Spring and March in Southern Finland has been quite cold despite warm and rainy week which melted away some of the already scarce snow. So, skiing mainly on artificial snow and mountainbiking on icy paths which is nice. But this also leaves time to read what has happened on software development field. This month it’s about tools and working methods.
Reclaim your abandonware
Super cool post about how to get the abandoned mac Twitter client to support 280 character tweets by modifying its assembly. (from @b0rk)
Keep calm and code on: Productivity tools for developers
Suggestion of tools for developers for different tasks. I didn’t agree all of them so my suggestions are in brackets. tl;dr; 1. actiTIME (or toggl) 2. Cold Turkey (or other pomodoro) 3. Strict Workflow 4. Habitica 5. Oh My Zsh (or Bash-it) 6. The Silver Searcher 7. UltraEdit (or Atom, VS Code etc.) 8. Homebrew 9. GitHub Changelog Generator. (from @ThePracticalDev)
Git aliases
If you use Git command-line a lot, you will probably grow your own list of Git aliases sooner or later. After simple standard aliases (ci -> commit, co -> checkout) you might want to see some advanced tricks you may find useful.
CTFR
Get subdomains of an HTTPS website abusing Certificate Transparency logs. (from @KitPloit). Apparently also curl "https://crt.sh/?q=%.starbucks.com&output=json" -sS | jq .name_value | uniq | tee output works.
Front-end
Front-End Performance Checklist 2018
Unbiased and objective front-end performance checklist for 2018 — an overview of the issues you might need to consider to ensure that your response times are fast, user interaction is smooth and your sites don’t drain user’s bandwidth. (from @igrigorik)
10 x weekend brunches in Helsinki
Il Birricifio, Ipi Kulmakuppila, Sandro, Gastro Café Kallio, Yes Yes Yes!, Loop, Moko Market, Sue Ellen, Paulig Kulma and Krog Roba. My addition to the list is Rupla. (from @VisitHelsinki)
For cold winter evenings here’s something to read. Monthly notes for February are about relearning and thinking.
Issue 27: 23.2.2018
Relearning
Computer Science and why it’s necessary even for web developers
“Computer Science and why it’s necessary even for web developers I know that in some countries a degree in CS is expensive or unattainable, and that some companies do unnecessary algorithm interviews. This thread is not about degrees or interviews, it’s about CS itself.”
And, yes, “fewest elements” is last, which means you only minimize classes and methods if everything else satisfied
Tools
shuttle
When openssh port forwarding doesn’t cut it, use sshuttle: “Transparent proxy meets VPN meets ssh.”
Microservices
The Death of Microservice Madness in 2018
There are many cases where great efforts have been made to adopt microservice patterns without necessarily understanding how the costs and benefits will apply to the specifics of the problem at hand. The post describes in detail what microservices are, why the pattern is so appealing, and also some of the key challenges that they present.
Should that be a Microservice? Keep These Six Factors in Mind
These days, you can’t swing a dry erase marker without hitting someone talking about microservices but few have spent any appreciable time asking if a given application should be a microservice. tl;dr; “1. Multiple Rates of Change; 2. Independent Life Cycles; 3. Independent Scalability; 4. Isolated Failure; 5. Simplify Interactions with External Dependencies; 6. The Freedom to Choose the Right Tech for the Job”.
face-verify.js: Monitoring who is physically looking at a website for additional security
Demo project showing how Machine Box tech can be integrated into JavaScript applications. Facebox takes an image and tells you how many faces it sees, as well as who those faces belong to provided you have shown it a single example previously. You can use this capability to build additional security into web apps so you can see how many people are watching the screen and who they are. Using the webcam with some JavaScript and Facebox, you can periodically check to ensure only authorised people can see the information that users consider sensitive.
Nick Stenning on Twitter
“Flat organisational structures do not exist. There are only organisations with visible structure and organisations with invisible structure”. (from Weekend Reading)
Developers On Call
Quite self-explanatory ideas for how to manage on-call rotations without burn out but maybe it’s not always that way. The linked Twitter thread is worth reading. (from Weekend Reading)
Something different
2017: The Year in Charts
These are the charts and themes that tell the story of 2017. I. The Year Volatility Died; II. Records Are Made to Be Broken; III. The World is Flattening; IV. Still Easy After All These Years; V. A Good Old-Fashioned Mania; VI. King Dollar Dethroned; VII. Wrapping Up: 1991-99 Redux?
Developing modern web applications you often come to around checking REST API responses and parsing JSON values. You can do it with a combination of Unix tools like sed, cut and awk but if you’re allowed to install extra tools or use Python then things get easier. This post shows you couple of options for extracting JSON values with Unix tools.
There are a number of tools specifically designed for the purpose of manipulating JSON from the command line, and will be a lot easier and more reliable than doing it with awk. One of those tools is jq as shown Stack Overflow. You can install it in macOS from Homebrew: brew install jq.
If you’re limited to tools that are likely installed on your system such as Python, using the json module gives you the benefit of a proper JSON parser and avoiding any extra dependencies.
Stack Overflow answers to the question of “Parsing JSON with Unix tools” shows you other options with standard tools like sed, cut and Awk and more exotic options with Perl, Node.js and PHP.
