Holiday season is soon here and it's good to take a short break from work and maybe learn or code some new things while relaxing and enjoying the winter time outside. Here's the monthly notes for December. Happy holidays!
Tips of ppl who want to learn
ReaktorNow Development Discussion campaign shared some insights in the field of software engineering. "Always keep learning and expanding your skills, and remember to step out of your comfort zone."
A novice’s guide to learning to code with CS50
Taking Down an Insider Threat
Excellent story about pentesting from the inside. And of great digital forensics and incident response team and meticulously implemented security practices.
Everything about distributed systems is terrible
Hillel Wayne 38 minutes talk at Code Mesh LDN 18 titled "Everything about distributed systems is terrible" talks about TLA+, formal specification system designed by Leslie Lamport. The claim is that you can find bugs in your (distributed) system by model checking that could be practically impossible to find with testing or in production.
Software development is one of the professions where you have to keep your knowledge up to date and follow what happens in the field. Staying current in the field and expanding your horizons can be achieved with different ways and one good way I have used is to follow different news sources, newsletters, listening podcasts and attending meetups. Here is my opinionated selection of resources to learn, share ideas, newsletters, meetups and other things for software developers. Meetups and some things are Finnish related.
There are some good sites to follow what happens in technology. They provide community powered links and discussions.
Podcasts provide nice resource for gathering experiences and new information how things can be done and what's happening and coming up in software development. I commute daily about an hour and time flies when you find good episodes to listen. Here's my selection of podcast relating to software development.
Webbidevaus: "Puheradiota webbikehityksestä suomeksi! Juontajina Antti Mattila ja Riku Rouvila."
Turvakäräjät: "Ajankohtaista kyberiä, kansantajuisesti. Käräjillä mukana Antti Kurittu, Laura Kankaala ja Juho Jauhiainen."
Herrasmieshakkerit: "Käsittelee tietoturvaan ja vääjäämättömään kybertuhoon liittyviä asioita ja ilmiöitä. Oppaana Mikko Hyppönen ja Tomi Tuominen."
Koodia pinnan alla: "Suomenkielinen podcast pinnan alla tapahtuvasta ohjelmistoteknologian magiasta. Puikoissa Markus Hjort ja Yrjö Kari-Koskinen."
ATK-hetki: "Vesa Vänskä ja Antti Akonniemi keskustelevat teknologiasta, bisneksestä ja itsensä kehittämisestä."
Normal information overload is easily achieved so it’s beneficial to use for example curated newsletters for the subjects which intersects the stack you’re using and topics you’re interested at.
The power of newsletter lies in the fact that it can deliver condensed and digestible content which is harder to achieve with other good news sources like feed subscriptions and Twitter. Well curated newsletter to targeted audience is a pleasure to read and even if you forgot to check your newsletter folder, you can always get back to them later.
Java Performance Tuning News: A monthly newsletter focusing on Java performance issues, including the latest tips, articles, and news about Java Performance. Curated by Jack Shirazi and Kirk Pepperdine.
DB Weekly: A weekly round-up of database technology news and articles covering new developments, SQL, NoSQL, document databases, graph databases, and more.
HTML and CSS
HTML5Weekly: Weekly HTML5 and Web Platform technology roundup. Curated by Peter Cooper.
CSS Weekly: Roundup of css articles, tutorials, experiments and tools. Curated by Zoran Jambor.
December is just around the corner but before that here's monthly notes for November. More about leadership and stories, something about software development.
Issue 35, 13.11.2018
CSS and Network Performance
Bash-it is a collection of community Bash commands and scripts for Bash 3.2+. (And a shameless ripoff of oh-my-zsh?). Includes autocompletion, themes, aliases, custom functions, a few stolen pieces from Steve Losh, and more.
Managing with the Brain in Mind
"Treat people fairly, draw people together to solve problems, promote entrepreneurship and autonomy, foster certainty wherever possible, and find ways to raise the perceived status of everyone". Good read about SCARF. (from @walokra)
On Being A Senior Engineer
What makes for a good senior engineer? tl;dr; Be mature engineer. Good read for everyone regardless of the line of business.
Seek out constructive criticism of their designs.
Understand the non-technical areas of how they are perceived.
Do not shy away from making estimates, and are always trying to get better at it.
Have an innate sense of anticipation, even if they don’t know they do.
Understand that not all of their projects are filled with rockstar-on-stage work.
Lift the skills and expertise of those around them.
Make their trade-offs explicit when making judgements and decisions.
Don’t practice CYAE (“Cover Your Ass Engineering”)
You work to live, not live to work
Remember, your job is not your life. You work to live, not live to work. Work on what makes you happy and not burn yourself out. Thread has good tips to recognize it and take control. (from @jevakallio)
Have you ever wondered how to become a bug bounty hunter or wanted to organize a bug bounty program? OWASP Helsinki chapter meeting number 35 told all about bug bounty programs from hacker and organizer point of views. The event was held 6.11.2018 at Second Nature Security (2NS) premises in Keilaniemi. Here's my short notes.
Notes from OWASP Helsinki chapter meeting #35
"Hunting for bounties in a web browser" by Juho Nurminen from 2NS started the event talks and told about how to approach the issue and showed some findings in details. For the usual of understanding the technology and focusing on what you know, it's beneficial to read up prior art. Is it repeatable bug? Reproduce it in other context. The talk presented cve-2018-6033 (extension code can execute downloaded files), cve-2018-6039 (XSS in DevTools, privileged API can be overwritten) and cve-2011-2800 (data leak across origins). tl;dr; pwn things, submit crbug.com, profit.
"#OWASPHelsinki 35 started by @jupenur hunting bounties in web browsers. Understand the tech (web, js, extensions, plugin API, devtools, NaCI, WebAssembly, etc.). Focus on what you know. Read up prior art. Nice examples of bugs found. @OWASPHelsinki meetup hosted by @2NS_fi." - @walokra
In "How to become a bug bounty hunter" Iiro Uusitalo from Solita talked about bug bounty platforms and tips to be succesful. In short: POC or GTFO, recon, stay on scope, automate all the things, focus, report, wait, profit, join the community.
"How to become a bug bounty hunter, told by @iiuusit at @OWASPHelsinki meetup. Tips: poc or gtfo, recon, stay on scope, automate all the things, focus, report, wait, profit, join the community. #OWASPHelsinki" - @walokra
"Running a successful bug bounty program" by Thomas Malmberg from Hackrfi bug bounty program covered the topic from the "random dude from the other side of the table" point of view. "What really matters is finding bugs" but there's a lot of things to manage. It comes to managing expectations of hackers and program owners. And remembering that hackers work for you (program owners) but they are not your employees.
"What really matters is finding bugs.” @tsmalmbe from @hackrfi told how to run a successful bug bounty program at @OWASPHelsinki meetup. Managing expectations of hackers and program owners. Remember: hackers work for you; hackers are not your employees. #OWASPHelsinki" - @walokra
The evening ended with a panel & discussion about bug bounty with Juho, Iiro and Thomas. There was lots of interesting questions asked and here's some of them in short.
Hardware bug bounties, how to do if device not publicly available?
On premises hack days -> not so successful, too little time, concentrate on low hanging fruits.
How to choose [bug bounty] program?
Wide scope -> low hanging fruits.
What kind of reports of findings
OWASP Top 10 covers almost everything.
Everyone is scared of finding remote code execution.
Business impact findings.
Recon: who we are, what we do -> what has big business impact. Also where's the legacy code?
Impact of how hacker and product owner sees findings? Owner will set the impact, how it should happen at both ends? how to define the final impact corresponding the value?
Always estimate, run some CVSS estimator.
Use Google's approach.
Fairness and trust. Programs task is to create trust.
Awfraid of reporting found bugs when there's no bug bounty program?
Program has rules which covers legal matters. Read the rules, ask.
Top 3 negative things?
Program runner went public, lots of bugs, hackers pwned whole system.
Program runner: call on Friday night, database lost. bug bounty program to blame.
Bug bounty programs role, client and customer: public programs. -> ncss, cert-fi.
Pentesting vs. bug bounty?
You shouldn't do bug bounty if you don't have enough security maturity. Too many reports at start (duplicates, cost much, etc.), then nothing if you don't pay.
Low hanging fruits are not interesting for good hackers
Pentesting last 30 days and result is report covering certain things.
Bug bounty concentrates on specific aspect.
Bug bounty and threat model? When program open, easier for black market to find vulnerabilities?
Threat model for users? Depends on product / service you are providing.
0-day on some Finnish site selling on USA black market -> not much interest.
Pentesting should be done first.
How to improve process?
Educating the bottom of the pyramid. Hammer and nails.
Public programs generate lots of noice vs. private
Bug bounty in 5 years?
More automated things, scripts to detectivive things, AI
Bug hunter side: more professional all around the pyramid, more spam
GraphQL Finland 2018 conference was held last week (18-19.10.2018) at Paasitorni and the first of its kind event in Finland brought a day of workshops and a day of talks around GraphQL. The event was organized by the same people as React Finland and it showed, in good ways. The talks were interesting, venue was appropriate, atmosphere was cosy and after party was bookie. Here's my notes from the event.
All of the talks were live streamed and they're available on Youtube. I was lucky to get a ticket to the event and be able to enjoy the talks live. In overall most of talks were easy to comprehend although I only had some experience with GraphQL through experiments and what I had learnt couple of months ago at React Finland 2018 conference (my notes from day 1 and day 2).
"GraphQL is an open source data query and manipulation language, and a runtime for fulfilling queries with existing data. It was developed internally by Facebook in 2012 before being publicly released in 2015. It provides a more efficient, powerful and flexible alternative to REST and ad-hoc web service architectures. It allows clients to define the structure of the data required, and exactly the same structure of the data is returned from the server, therefore preventing excessively large amounts of data from being returned. - Wikipedia"
Adopting GraphQL in Large Codebases - Adam Miskiewicz
The event started with Adam Miskiewicz's story from Airbnb and incrementally adopting GraphQL. It's simple to start using GraphQL in your project but adding it incrementally and carefully in huge codebases powering large distributed systems is not quite as straightforward. The talk dived into how Airbnb is tackling this challenge, what they've learned so far, and how they plan to continue evolving their GraphQL infrastructure in the future. Towards GraphQL Native!
Going offline first with GraphQL — Kadi Kraman Kadi Kraman from Formidable Labs talked about going offline first with GraphQL. She did a nice interactive demo with React Native and Apollo 2. Users expect your mobile app to work offline and the tooling in GraphQL makes it reasonably straightforward to get your React Native app working offline. Slides
"Do this as you go and offline comes almost as a side-effect"
Life is hard and so is learning GraphQL — Carolyn Stransky
Life is hard, without documentation. Carolyn Stransky presented her story of ups and downs when learning GraphQL and documentation's role in it. The problem with GraphQL is that - because there's no "vanilla" GraphQL - there’s no central hub for all of the information and tooling necessary to learn. It's underutilized and scattered throughout our community. The talk touched on how to better enable GraphQL docs for learning and comprehension and slides pointed to good resources.
Some of the GraphQL Finland talks were quite deep in the content and as most of the talks were around 15 minutes, the pace was quite demanding. At the event I concentrated on topics which seemed most relevant and saved the rest for later. The sponsor's lounge by Gofore and Digia provided nice relaxing space to get your thoughts together. Here are the topics I saved for later.
End-to-end type-safety with GraphQL — Johannes Schickling
Talk dived deep into one of the most powerful features of GraphQL - its type-system. GraphQL can be used to enable end-to-end type-safety across any language, making your application architecture more resilient and easier to evolve.
Reason and GraphQL — Nik Graf
Using Reason's type inference you can create GraphQL servers with 100% type coverage. And Reason shines even more so on the client. Send one quick introspection request and you get full autocompletion on your schema right in the browser.
Autumn is well on it's way and winds are bringing rains and clouds to the sky. Autumn also means that meetups are awaken and interesting stories from the field are presented. Here's monthly notes for September. Start with writing readable code, continue to build React app with TypeScript, read how hacker puzzles can be solved and improve your designs with tactics instead of talent. Also use smarted command line tools and listen a Kubernetes security journey.
Issue 34, 29.9.2018
10 practices for writing readable code
Writing readable code may seem subjective but there are core elements within all code which make it readable. Follow these 10 practices. Although I don't quite agree with removing comments ?
"As engineers, we can, and should, and will do better. Have better tools, build better apps, faster, more predictable, more reliable, using fewer resources". But on the other hand people won’t pay for efficiency. They buy solutions to their problems. (from @walokra)
Fullstack Express-React App With TypeScript
Have you thought about starting a React app with TypeScript and integrating it with Travis CI and Heroku? Read this definitive guide and check the source of a starter kit for a full stack express-react app. (from @walokra)
Solving the Disobey 2018 puzzle
Great writeup of solving the Disobey.fi 2019 hacker ticket puzzle. Shows you some tools and techniques you can use to progress with these kind of puzzles. Contains spoilers, so steer clear if you want the fulfilment and bliss that comes from solving it. (from @walokra)
This talk is about you [React Native Developer] (video)
Life of a React Native developer? Jani Eväkallio talks about you at React Native EU 2018. When building software products we're focused on "how" but should ask also "what" and "why". Not just be happy when tickets move from left to right side of the screen. (from @walokra)
7 Practical Tips for Cheating at Design
"Improving your designs with tactics instead of talent." Every web developer inevitably runs into situations where they need to make visual design decisions, whether they like it or not. There are a ton of tricks you can use to level up your work that don’t require a background in graphic design. Here are seven simple ideas you can use to improve your designs today.
Tools of the trade
Command line is powerful tool but the common tools can be improved. Remy Sharp wrote his current list of improved CLI tools.
Open source, end-to-end distributed tracing to monitor and troubleshoot transactions in complex distributed systems.
Red Hat Forum Finland 2018 was held 11.9.2018 at Finlandia-talo and it's mainline was "Ideas worth exploring. Come with questions. Leave with ideas." The event was divided to keynote and to four breakout sessions. The four breakout sessions were: 1. Automation – Ansible 2. Journey to Cloud-Native Applications with OpenShift 3. Business & Solution track 4. Half day Executive discussions and round tables. I chose to get hands-on with OpenShift but also Ansible would've been interesting. Here's my notes from the event.
Red Hat Forum Finland 2018: Ideas worth exploring
Red Hat Forum 2018 Helsinki started with keynote session by Michel Isnard from Red Hat and in "Digital transformation & the open organization" he talked about open source and how Red Hat embraces it. "Open source is collaborative curiosity, a culture with a desire to connect and the technologies to do it. Yet what draws our attention isn't the technology alone; it’s what we can do with it. It gives us the platform for imagination, a focal point to collectively push for new possibilities."
Be courageous, be open and innovate in the open.
Next there was customer reference by Markku Reinikainen from SOS International. He told us about their open innovation platform and how they have modernized their applications and moved to the mobile world.
Journey to Cloud-Native Applications with OpenShift
The main content of the Red Hat Forum event were the breakout sessions. I chose the full day hands-on workshop which showed how to modernize an existing legacy monolithic application by applying microservice architecture principles, using modern lightweight runtimes like WildFly Swarm (Thorntail.io) and Spring Boot, and deploying to container-based infrastructure using OpenShift Container Platform. The material and slides are available on GitHub.
The lab was split into four scenarios, going through the process of understanding how a developer can most effectively use Red Hat technologies in deploying a monolith to OpenShift, wrapping it with a CI/CD pipeline, developing microservices to start replacing functionality in the monolith, and integrating it all together to form the beginnings of a complete modernization of an existing app. The last scenario was about using Istio to prevent and detect issues in a distributed system.
The session started with Red Hat Application Migration Toolkit (RHAMT) and migrating (lift & shift) Java EE monolith app on WebLogic to run on JBoss EAP and OpenShift in the cloud. Crafty tool which fixed poor and non-standard choices done in legacy app.
The breakout session had also a talk from Red Hat partner. "Shift to a Cloud-First Core" talk by Capgemini told how they are approaching OpenShift projects. Different options, some are easier depending of legacy technologies. Retain, retire, migrate: lift & shift, new layers, new apps.
OpenShift hands-on session continued with developer introduction which was about live synchronization and changes, deploying to different environments, Jenkins Pipeline, Continuous Delivery and approval steps.
Third and fourth scenarios were about strangling the monolith with transforming it to microservices architecture with and without Spring Boot. Splitting up monolith to domain specific applications and connecting them. Lots of things that goes over the hill and seems magic if you're not familiar with them. You just click click click, done, profit. Some technologies used were Spring Boot and Spring Cloud, Snowdrop, Feign and Hystrix.
The last and most interesting part of the hands-on session was Istio and resilient apps and due time schedule Red Hat guy clicked and talked it through. It gave good overview to visualization, monitoring, metrics, fault injection, traffic shifting, circuit breaking, rate limiting and tracing. Time was limited so much things left to be read.
All the OpenShift scenarios used Katacoda which made the hands-on experience with just a few clicks. Crafty tool for this kind of sessions and although you just clicked through with relative fast pace. For example "Developer Introduction to OpenShift" estimated time 45-60 minutes and the lab had 23 minutes. The limited time made the hands-on experience somewhat superficial but you got the point what the possibilities are and how OpenShift works.
And last Red Hat talked about OpenShift and their services regarding application modernization. Modernization of legacy applications is in high demand and there are different paths to achieve that.
One point regarding monoliths vs. microservices was that as Martin Fowler wrotes in Monolith First.: "you shouldn't start a new project with microservices, even if you're sure your application will be big enough to make it worthwhile."
Red Hat Forum Finland 2018 was nice event and the content was interesting. The hands-on session was fast paced but you got the point and ideas worth exploring. Will look into Istio. The WiFi network had some problems but got better when more access points were added. After the official program there was some networking and drinks. Some food other than hemp snacks and vegetable chips would've been nice but Woolshed provided in that regard. Thanks for Red Hat for organizing the event and good talks.
The meetup started with Splitting React codebases for increased development speed by Hugo Kiiski from Smartly.io. He told how their Video Editor component is separated from main frontend. Code is in monorepo managed by Lerna. More tools going to be splitted. The recording of the presentation can be seen on Vimeo. (Twitter)
Use GraphQL! by Mikhail Novikov showed a quick intro to GraphQL, covered the current state of its adoption and described several ways of how to move to GraphQL. GraphQL "fills the gap between client and server developer needs and values. Matching server capabilities with client requirements." GraphQL clients to use are i.a. Apollo and Relay. See the slides for more information. The recording of the presentation can be seen on Vimeo. (Twitter)
As I mentioned the event has hosted by Smartly.io and their office in Postitalo was cosy and had nice demo room for the meetup. Also the food and beverages were nice althought the hamburger patty was a bit too raw.
Summer has turned to Autumn and it begins to show in the weather. Sun is setting earlier and soon it's dark almost from dawn to dusk, rain clouds are gathering in the sky with cold winds. Good time to stay inside and read some articles and learn new things. Here's the monthly notes for August.
Issue 33, 28.8.2018
Elements of Artificial Intelligence free online course
"Do you wonder what AI really means? Are you thinking about the kind of impact AI might have on your job or life? Do you want to understand how AI will develop and affect us in the coming years? Then this is the course for you!"
Microservices and cloud
Docker Pattern: The Build Container
Let’s say that you’re developing a microservice in a compiled language or an interpreted language that requires some additional “build” steps to package and lint your application code. This is a useful docker pattern for the "build" container.
Experiences with running PostgreSQL on Kubernetes
Gravitational CTO, Sasha Klizhentas, tells about his experience running PostgreSQL on Kubernetes. The challenges involved, open source and commercial tools that can help and other alternatives to managing stateful applications on Kubernetes.
Google Cloud Platform - The Good, Bad, and Ugly (It's Mostly Good)
Deps developer tells his thoughts about Google Cloud Platform and splits them into good, meh, bad, ugly, and opportunities for improvement. He compares and contrasts with Amazon Web Services (AWS), the other hosting provider that he has the most experience with, and GCP's biggest competitor.
Goodbye Microservices: From 100s of problem children to 1 superstar
Segment's story of going to microservices architecture and back. "When deciding between microservices or a monolith, there are different factors to consider with each. In some parts of our infrastructure, microservices work well but our server-side destinations were a perfect example of how this popular trend can actually hurt productivity and performance. It turns out, the solution for us was a monolith."
Have you ever needed to generate a random number in code?
Have you ever needed to generate a random number in code? whether it's for rolling a dice, or shuffling a set, this tweet thread is here for you! There's no reason that it should be easy or obvious, very experienced programmers repeat common mistakes. I did, before I learned ... from (@colmmacc)
Tools of the trade
Semantic Commit Messages
See how a minor change to your commit message style can make you a better programmer. Format: <type>(<scope>): <subject>. <scope> is optional.
The Psychology of Money "Let me tell you the story of two investors, neither of whom knew each other, but whose paths crossed in an interesting way."
Issue 32, 23.7.2018
Defining Component APIs in React
Collects some of the best practices for working with React. "The following is a collection of thoughts, opinions, and advice for defining component APIs that are meant to be more flexible, composable, and easier to understand. None of these are hard-and-fast rules, but they’ve helped guide the way I think about organizing and creating components." (from Weekend reading)
TIL: node-jsmin (port of Crockford's JSMin) was dropped from a lot of places as modified MIT license with "The Software shall be used for Good, not Evil" is not compliant with definition of open source software which doesn't permit any restriction on how software may be used. (from @walokra)
Introducing Jib — build Java Docker images better
"Jib, an open-source Java containerizer from Google that lets Java developers build containers using the Java tools they know. Jib is a fast and simple container image builder that handles all the steps of packaging your application into a container image. It does not require you to write a Dockerfile or have docker installed, and it is directly integrated into Maven and Gradle."
Brutalist Web Design
TL;DR; Content is readable on ~all screens & devices. Only hyperlinks & buttons respond to clicks. Hyperlinks are underlined, buttons are buttons. Back button works. View content by scrolling. Decoration when needed and no unrelated content. Performance is a feature. (from @walokra)
Little known trick: the <script> tag in html runs the code inside, and also hides it using css display:none. But I can change that to display:block, so that I can show sample code to the reader and also run it on the page to generate diagrams. (need to test across browsers). This also applies to <style> tags, where you can also use contentEditable to create a live editable css of the page you are on. (from @ Amit Patel)
Terminal-based web browser renders everything a modern browser can (HTML5, CSS3, JS, video, even WebGL). Use case: run the browser in a data center with fast internet, and access it over SSH from a device that has slow/limited internet. (from Weekend reading)
"petition to make "paste and match formatting" the default paste option"
Riot Games Approach to Anti-Cheat
Riot Games published an article about their anti-cheating methods – nothing really fancy or new but, in the Hacker News thread there was an interesting comment by a cheat writer:
"The current Mac game client for League Of Legends contains full debug symbols and it doesn't have Packman (the packer described in this article), which makes it quite easy to look through the symbols. Inside you can find all of the anti-cheat-related network packets. Now, I personally expect anti-cheat to snoop around my system when I'm doing something shady like scanning its memory. However, if I was a normal user of the game, I would be a bit concerned to know that it might be sending my recently used file names, drive names, system driver names, currently running processes, processor information, system state, and even entire binary files that it automatically deems as "suspicious", to their servers."
@aral and maya kosoff: "X is a service that enables you to control articles presented to your wife on the websites she usually visits, in order to influence her on a subconscious level to initiate sex. The best bit? It's "just" adtech. It's retargeting. It's how Google makes money." Also suggested use cases are "get your kid a dog" or "stop drinking" which eems to open up a whole new acquaintance micromarketing concept. Makes you think how you're influenced and by whom.
"Imgur's fake adherence to GDPR is exactly the kind of transgression that should trigger those multi-million euro fines. There are literally HUNDREDS and HUNDREDS of shady services getting your data. Only bulk link is to ALLOW ALL, which is also default. Tons you can't opt-out. ?"
"The StemCAPtain replaces the stem cap, aka top cap, piece of a threadless 1" or 1 1/8" headset with different functional accessories. In addition to the simple and elegant analog clock, we offer a thermometer, bottle opener, picture frame, compass, GPS mount, and USB charger"