Reset Hasura migrations and squash files

Using GraphQL for creating REST APIs is nowadays popular and there are different tools you can use. One of them is Hasura which is an open-source engine that gives you realtime GraphQL APIs on new or existing Postgres databases. Hasura is quite easy to work with but if your GraphQL schemas change a lot it creates plentiful of migration files. This has some unwanted consequences (for example slowing down the hasura migrate apply or even blocking it). Here’s some notes how to reset the state and create new migrations from the state that is on the server.

Note: From Hasura 1.0.0 onwards squashing is easier with hasura migrate squash command. It's still in preview. But before Hasura 1.0.0 version you have to squash migrations manually and this blog post explains how. The results are the same: squashing multiple migrations into a single one.

Hasura documentation provides a good guide how to squash migrations but in practice there are couple of other things you may need to address. So let’s combine the steps Hasura gives and some extra steps.

Reset Hasura migrations

First make a backup branch:

  1. $ git checkout master
  2. Create a backup branch:
    $ git checkout -b backup/migrations-before-resetting-20XX-XX-XX
  3. Update the backup branch to origin:
    $ git push origin backup/migrations-before-resetting-20XX-XX-XX

We are assuming you've local Hasura running on Docker with something like the following docker-compose.yml

version: "3.6"
services:
  postgres:
    image: postgres:11-alpine
    restart: always
    ports:
      - "5432:5432"
    volumes:
      - db_data:/var/lib/postgresql/data
    command: postgres -c max_locks_per_transaction=2000
  graphql-engine:
    image: hasura/graphql-engine:v1.0.0-beta.6
    ports:
      - "8080:8080"
    depends_on:
      - "postgres"
    restart: always
    environment:
      HASURA_GRAPHQL_DATABASE_URL: postgres://postgres:@postgres:5432/postgres
      HASURA_GRAPHQL_ENABLE_CONSOLE: "true" # set to "false" to disable console
      HASURA_GRAPHQL_ADMIN_SECRET: changeme
      HASURA_GRAPHQL_ENABLED_LOG_TYPES: startup, http-log, webhook-log, websocket-log, query-log
volumes:
  db_data:

Create local instance of Hasura with up to date migrations:

  1. $ docker-compose down -v
  2. $ docker-compose up
  3. $ hasura migrate apply --endpoint=http://localhost:8080 --admin-secret=changeme

Reset migrations to master:

  1. git checkout master
  2. git checkout -b reset-hasura-migrations
  3. rm -rf migrations/*

Reset the migration history on server. On hasura SQL console, http://localhost:8080/console:

TRUNCATE hdb_catalog.schema_migrations;

Setup fresh migrations by taking the schema and metadata from the server. By default init only takes public schema if others not mentioned with the --schema "your schema" parameter. Note down the version for later use.

  1. Create migration file:
    $ hasura migrate create "init" --from-server
  2. Mark the migration as applied on this server:
    $ hasura migrate apply --version "" --skip-execution
  3. Verify status of migrations, should show only one migration with Present status:
    $ hasura migrate status
  4. You have brand new migrations now!

Resetting migrations on other environments

  1. Checkout the reset branch on local machine:
    $ git checkout -b reset-hasura-migrations
  2. Reset the migration history on remote server. On Hasura SQL console:
    TRUNCATE hdb_catalog.schema_migrations;
  3. Apply migration status to remote server:
    $ hasura migrate apply --version "<version>" --skip-execution

Local environment Hasura status

For other developers please refer these instructions in order to get the backend into same state.

Option 1: Keep old data

  1. Checkout the backup branch on local machine:
    $ git checkout backup/migrations-before-resetting-20XX-XX-XX
  2. Reset the migration history on local server. On Hasura SQL console:
    TRUNCATE hdb_catalog.schema_migrations;
  3. Apply migration status to local server:
    $ hasura migrate apply --version "<version>" --skip-execution

Option 2: Remove all and start from beginning

  1. Clean up the old docker volumes:
    $ docker-compose down -v
  2. Start up services:
    $ docker-compose up
  3. Checkout master:
    $ git checkout master
  4. Apply migrations:
    $ hasura migrate apply --endpoint=http://localhost:8080 --admin-secret=changeme

Possible extra steps

Now your Hasura migrations and database tables are in one migration init file but sometimes things don’t work out when applying it to empty database. We are using Hasura audit-trigger and had to reorder the SQL clauses done by the migrate init and add some missing parts.

  1. Move schema creations after audit clauses
  2. Move audit.audit_table(target_table regclass) to last audit clause and copy it from audit.sql
  3. Add pg_trgm extension as done previously (fixes "operator does not exist: text <%!t(MISSING)ext" in public.search_customers_by_name)
  4. Drop session constraints / index before creating new
  5. Create session table only if not exists

Notes from GraphQL Finland 2018

GraphQL Finland 2018 conference was held last week (18-19.10.2018) at Paasitorni and the first of its kind event in Finland brought a day of workshops and a day of talks around GraphQL. The event was organized by the same people as React Finland and it showed, in good ways. The talks were interesting, venue was appropriate, atmosphere was cosy and after party was bookie. Here's my notes from the event.

All of the talks were live streamed and they're available on Youtube. I was lucky to get a ticket to the event and be able to enjoy the talks live. In overall most of talks were easy to comprehend although I only had some experience with GraphQL through experiments and what I had learnt couple of months ago at React Finland 2018 conference (my notes from day 1 and day 2).

"GraphQL is an open source data query and manipulation language, and a runtime for fulfilling queries with existing data. It was developed internally by Facebook in 2012 before being publicly released in 2015. It provides a more efficient, powerful and flexible alternative to REST and ad-hoc web service architectures. It allows clients to define the structure of the data required, and exactly the same structure of the data is returned from the server, therefore preventing excessively large amounts of data from being returned. - Wikipedia"

Read also the organizer's summary of the event and check out the photos.

Life is hard, GraphQL is easy
Life is hard, GraphQL is easy

Notes from the talks

(titles are links on Youtube to particular talk)

Adopting GraphQL in Large Codebases - Adam Miskiewicz
The event started with Adam Miskiewicz's story from Airbnb and incrementally adopting GraphQL. It's simple to start using GraphQL in your project but adding it incrementally and carefully in huge codebases powering large distributed systems is not quite as straightforward. The talk dived into how Airbnb is tackling this challenge, what they've learned so far, and how they plan to continue evolving their GraphQL infrastructure in the future. Towards GraphQL Native!

Going offline first with GraphQL — Kadi Kraman
Kadi Kraman from Formidable Labs talked about going offline first with GraphQL. She did a nice interactive demo with React Native and Apollo 2. Users expect your mobile app to work offline and the tooling in GraphQL makes it reasonably straightforward to get your React Native app working offline. Slides

"Do this as you go and offline comes almost as a side-effect"

Life is hard and so is learning GraphQL — Carolyn Stransky
Life is hard, without documentation. Carolyn Stransky presented her story of ups and downs when learning GraphQL and documentation's role in it. The problem with GraphQL is that - because there's no "vanilla" GraphQL - there’s no central hub for all of the information and tooling necessary to learn. It's underutilized and scattered throughout our community. The talk touched on how to better enable GraphQL docs for learning and comprehension and slides pointed to good resources.

Database-first GraphQL Development — Benjie Gillam
Benjie Gillam from PostGraphile teached how database-centric approach to GraphQL API development can give your engineers more time to focus on the important parts of your application. Adhere to GraphQL best practices, embrace the power of PostgreSQL, and avoid common pitfalls. Interesting slides.

graphql-php — Christoffer Niska
Christoffer Niska gave some good tips for software development: Don’t over-abstract, test everything, use static type checking, follow best practices, don’t prematurely optimize.

(Un)expected use of GraphQL — Helen Zhukova
(Un)expected use of GraphQL talk by Helen Zhukova showed the benefit of single code base on client and server side. Partly live coded with i.a. CodeSandbox. The any DB in this case was MongoDB.

Mysterious closing keynote — Dan Schafer
The mysterious closing keynote was Dan Schafer talking about GraphQL history, present and future. "Strive for single sources of truth". Still lots of things to do in the ecosystem. Slides.

Talks from the deep end

Some of the GraphQL Finland talks were quite deep in the content and as most of the talks were around 15 minutes, the pace was quite demanding. At the event I concentrated on topics which seemed most relevant and saved the rest for later. The sponsor's lounge by Gofore and Digia provided nice relaxing space to get your thoughts together. Here are the topics I saved for later.

All work no play
All work no play (photo: Juho Vepsäläinen)

Makes you a dull boy
Makes you a dull boy (photo: Juho Vepsäläinen)

Emerging uses of GraphQL for rapid application development — Ellie Day
Many schemas simply mirror their RESTful counterparts, limiting their overall usefulness. The talk covers emerging patterns that enable rapid GraphQL-powered application development.

Building efficient, waterfall-free GraphQL applications — Jimmy Jia
Jia showed how in larger projects it’s easy to run into situations where requests wait on each other regardless of using GraphQL, to the detriment of user experience. He explored how these problems arise, and looked at strategies for mitigating these problems.

Building tools for GraphQL — Glenn Reyes
Using the right tooling helps development and Reyes talked about what can we do to improve DX and supercharge our GraphQL development.

End-to-end type-safety with GraphQL — Johannes Schickling
Talk dived deep into one of the most powerful features of GraphQL - its type-system. GraphQL can be used to enable end-to-end type-safety across any language, making your application architecture more resilient and easier to evolve.

Real-time GraphQL with Event Sourcing — Andrew Rhyne
Thirty-thousand foot overview of event sourcing and how naturally it fits with GraphQL and subscriptions. And what it enables at Gatsby.

Hidden gems of GraphQL query — Ivan Goncharov
GraphQL has a few useful features that are typically overlooked. The talk contained spoilers from GraphQL Specification.

Reason and GraphQL — Nik Graf
Using Reason's type inference you can create GraphQL servers with 100% type coverage. And Reason shines even more so on the client. Send one quick introspection request and you get full autocompletion on your schema right in the browser.

Writing a realtime GraphQL backend as a compiler in Haskell — Tanmai Gopal
The talk went over the key architectural decisions, programming constructs and the unforeseen implementation pitfalls Hasura ran into while building a high-performance GraphQL backend. It also covered tools from the Haskell ecosystem that made this possible.

Where art thou, my error? — Eloy Durán
Error metadata.

Afterwords

The last chance to practice your Finnish was the Afterparty ? at Gofore office!