Automate your dependency management using update tool

Software often consists of not just your own code but also is dependent of third party libraries and other software which has their own update cycle and new versions are released now and then with fixes to vulnerabilities and with new features. Now the question is what is your dependency management strategy and how do you automate it?

Fortunately automated dependency updates for multiple languages is a solved problem as there are several update tools to help you: Renovate, Dependabot (GitHub), Greenkeeper ($), Depfu ($) and Dependencies.io ($) to name some alternatives. In this blog post I will concentrate on using Renovate and integrate it with GitLab CI.

Renovate your dependencies

Renovate is open source tool which works with most git hosting platforms (public or self-hosted) and it's possible to host Renovate Bot yourself. It’s installable via npm/yarn or Docker Hub.

In short, the idea and workflow of dependency update tools are following:

  1. Checks for updates: pulls down your dependency files and looks for any outdated or insecure requirements.
  2. Opens pull requests: If any of your dependencies are out-of-date, tool opens individual pull requests to update each one.
  3. Review and merge: You check that your tests pass, scan the included changelog and release notes, then hit merge with confidence.

Now you just run the depedency update tool on regular basis on your continuous integration, watch how the pull requests fly and you get to keep your dependencies secure and up-to-date.

The manual chore of checking for updates, looking for changelogs, making changes, running tests, writing pull requests and more is now moved to reviewing pull requests with better confidence of what has changed.

Self-hosted in GitLab CI

Renovate Bot is a node.js application so you’ve couple of alternative ways to run it on your CI/CD environment. You can use a node docker image which installs and runs renovate, or you can use Renovate Bot's own Docker image as I chose to do. We are using docker-in-docker approach of running the Renovate docker container. That means you can start Docker containers from within an other Docker container.

First create an  account for the bot on the Gitlab instance (the best choice) or use your own account. Then generate a personal access token with the api scope for renovate to access the repositories and create the branches and merge requests containing the dependency updates.

Then create a repository for the configuration and renovate will use that repo’s CI pipelines. Paste your Gitlab token under CI / CD > Variables as a new variable and give it the name RENOVATE_TOKEN. Set it to protected and masked to hide the token from the CI logs and to only use it for Pipelines starting on protected branches (your master branch is protected by default).

You'll also need a Github access token with the repos scope for renovate to read sources and changelogs of dependencies hosted on Github. It’s not important what Github account is used as it's just needed because Github's rate-limiting would block your bot making unauthenticated requests. Paste it as an other variable with the name GITHUB_COM_TOKEN.

To configure Renovate we need to add three files to our repository:

config.js for configuring renovate:

module.exports = {
  platform: ‘gitlab’,
  endpoint: ‘https://gitlab.com/api/v4/',
  assignees: [‘your-username’],
  baseBranches: [‘master’],
  labels: ['renovate', 'dependencies', 'automated'],
  onboarding: true,
  onboardingConfig: {
    extends: ['config:base'],
  },
};

repositories.txt for repositories we want to check:

openpatch/ui-core
openpatch/template

.gitlab-ci.yml to run renovate:

default:
  image: docker:19
  services:
    - docker:19-dind

# Because our GitLab runner doesn’t have TLS certs mounted and runs on K8s
variables:
  DOCKER_HOST: tcp://localhost:2375
  DOCKER_DRIVER: overlay2
  DOCKER_TLS_CERTDIR: ‘'

renovate:
  stage: build
  script:
    - docker run -e RENOVATE_TOKEN="$RENOVATE_TOKEN" -e GITHUB_COM_TOKEN="$GITHUB_COM_TOKEN" -v $PWD/config.js:/usr/src/app/config.js renovate/renovate:13 $(cat repositories.txt | xargs)
  only:
    - master

Now everything is finished and when you run the pipeline renovate will check the repositories in repositories.txt and create merge request if a dependency needs to be updated.

The first merge request to repository is Configure Renovate which helps you to understand and configure settings before regular Merge Requests begin.

As a last step create a Pipeline Schedule to run the pipeline every x hours or x day or whatever you like. You can do this in the bot's config project / repository under CI / CD > Schedules by creating a new schedule and chosing the frequency to run your bot.

You can also reduce noise by Package Grouping and Automerging. Here’s an example of grouping eslint themed packages and automerging them if tests pass.

Project’s renovate.json:

{
    "packageRules": [
        {
          "packagePatterns": [ "eslint" ],
          "groupName": "eslint",
          "automerge": true,
          "automergeType": "branch"
        }
      ]
}

Summary

Congratulations! You’ve now automated the dependency updating with GitLab CI. Just keep waiting for the merge requests and see if your test suites are successful.  If you are really trusting your test suite, you can even let renovate auto-merge the request, if the pipeline succeeds.

Short notes on tech 11/2021

Week 11 of 2021

Work life

Handbook for Remote employees
Read how Remote works as a company and take notes.

How to Deal with Difficult People on Software Projects

Your Thinking Rate Is Fixed
"You can’t force yourself to think faster. If you try, you’re likely to end up making much worse decisions. Here’s how to improve the actual quality of your decisions instead of chasing hacks to speed them up." (from Hackernewsletter)

Generalists vs specialists - who has a greater chance of success?
(from Hackernewsletter)

Learning

The Front-End Developer Learning Roadmap

AWS Cloud Development Kit Workshop
Learn CDK.

Software development

Best practices for REST API design
"In this article, we'll look at how to design REST APIs to be easy to understand for anyone consuming them, future-proof, and secure and fast since they serve data to clients that may be confidential." (from Hackernewsletter)

You really should use dev containers
tl;dr; "use development containers, indicating the act of running and compiling code inside a container, not on the host machine OS."

The case for and against Amazon Cognito

Information Security

How often should I rotate my ssh keys?
"More often than never! As often as you can. And make sure you retire old keys when you’re done with them." (from Hackernewsletter)

Tools of the trade

Helppo
"Instant admin UI for your database" Supports PostgreSQL and MySQL.

TSDX
Zero-config CLI for TypeScript package development

Colorful VSCode titlebars for better productivity
tl;dr;
"workbench.colorCustomizations": {
"titleBar.activeBackground": "#553955" // change this color!
}

Short notes on tech 10/2021

Week 10/2021

iOS development

Analyse, Design & Code the ClubHouse app with SwiftUI
This video from Franck Ndame on recreating the Clubhouse UI from scratch is incredibly well-produced and enjoyable to watch. Follow it up by checking out the Figma design and source code. (from iOS Dev Weekly)

Cloud

A Quick Look at GKE Autopilot (in 15 minutes)
"I was curious to take a look at GKE Autopilot, so if you don't have time to play with it, I did it for you. (from Cloud Security Reading List)"

Software development

Ethical Design Guide
Resources for creating ethical products that don't cause harm. (from Weekend Reading List)

Why is it so hard to see code from 5 minutes ago?
"Do you undo recent changes to remember what the code looked like a few minutes ago? Why can’t IDEs do that for us?" Waiting for "Yestercode" to be a VS Code extension. (from Weekend Reading List)

Modules, monoliths, and microservices
(from Hacker News comments)

Worklife

Things your manager might not know
If you never managed people before, it's not obvious how to manage your manager. Julia explains how to do that. (from Weekend Reading List)

How to be more productive without forcing yourself

Something different

Internet Archive Infrastructure
tl;dr; Lots of hardware.

Short notes on tech 9/2021

Week 9/2021

Tools of the trade

Uizard
"The design tool for everybody. Scan a paper scribble and you get a wireframe. Upload images and it will extract color and style to generate a matching theme." (from Weekend Reading)

Camo
Camo makes you look great on Zoom by using your iPhone as a webcam. You can control lighting, cropping, focus, Bokeh effect, stream 1080p with no stutter (needs cable), use front-facing, telephoto or wide lens. (from Weekend Reading)

Free for developers
List of free and free tier resources.

Cloud

Kubernetes README
"A collection of useful resources to read to learn more about Kubernetes." (from Cloud Security List)

Work

Unpacking Interview Questions
"Series sharing some of the questions the writer uses when he interviews for technical roles. He’ll unpack the question, when to ask it, and how to evaluate answers."

Something different

Scientists break through the wall of sleep to the untapped world of dreams
"Researchers at Paller’s lab at Northwestern University in Illinois, along with researchers in France, Germany and the Netherlands, have independently demonstrated two-way communication with people as they are lucidly dreaming during REM (rapid eye movement) sleep." (from Weekend Reading)