Notes of Best Practices for writing Playwright tests

Playwright “enables reliable end-to-end testing for modern web apps” and it has good documentation also for Best Practices which helps you to make sure you are writing tests that are more resilient. If you’ve done automated end-to-end tests with Cypress or other tool you probably already know the basics of how to construct robust tests just utilizing the ways how Playwright does things. I’ve also written notes of Best Practices for Cypress.

Here are my notes of Playwright Best Practices documentation with some additions. This article assumes you know and have Playwright running.

In short:

• Make tests as isolated as possible
• Set state programmatically, don’t use the UI to build up state.
• Avoid testing third-party dependencies, only test what you control.
• Use locators to find elements on the webpage
• Use web first assertions such as toBeVisible()

Organizing tests

• Write specs in isolation, avoid coupling
• Organize tests in a way that is logical and easy to follow.

The folder structure for tests might look like.

├ pageobjects 
├── login.js
├── dev-page.js
├── books.js
├ auth
├── login.spec.js
├── logout.spec.js
├ author
├── author.get.spec.js
├── author.post.spec.js
├── author.delete.spec.js
├── author.put.spec.js
├ a11y
├── a11y.spec.js
├── a11y.lighhouse.spec.js
└ auth.setup.js

Selecting Elements

• Use built in locators
• Locators come with auto waiting and retry-ability.
• To make tests resilient, prioritize user-facing attributes and explicit contracts (text content, accessibility roles and label).
• Locators can be chained to narrow down the search to a particular part of the page.
• You can also filter locators by text or by another locator.

Examples:

page.getByRole('button', { name: 'submit' });

const product = page.getByRole('listitem').filter({ hasText: 'Product 2' });

await page.getByRole('listitem')
        .filter({ hasText: 'Product 2' })
        .getByRole('button', { name: 'Add to cart' })
        .click();

This is one part where Playwright differs from e.g. Cypress which recommends to use data-* attributes to provide context to your selectors and isolate them from CSS or JS changes.

What I read is that the recommendation is explained as “your tests mimic how your users find elements, but they will also be more robust as user-facing attributes generally change less than ids, class names or other implementation details.” I’m not so sure about that on the last part.

You can use data-* attributes and get locators from them with getByTestId, e.g.

<button data-testid="directions">Itinéraire</button>

await page.getByTestId('directions').click();

Make tests as isolated as possible

• Each test should run independently with its own local storage, session storage, data, cookies etc.
• Test isolation improves reproducibility, makes debugging easier and prevents cascading test failures.

Use web first assertions

• By using web first assertions Playwright will wait until the expected condition is met.

await expect(page.getByText('welcome')).toBeVisible();

Use Page Object Model (POM)

• Helps to avoid duplication, improve maintainability and simplify interactions between pages in multiple tests.

Page Object Model is a common pattern and it conveys more intent and encourages behaviour over raw mechanics. Playwright have included this example in their docs to give you an idea on how to implement it.

Mastering Playwright: Best Practices for Web Automation with the Page Object Model provides good examples and explanation of using POM. Also this GitHub issue comment has some info.

Controlling State

• Set state directly / programmatically
• Reuse the signed-in state in the tests with setup project.

The recommended approach for tests without server-side state is to authenticate once in the setup project, save the authentication state, and then reuse it to bootstrap each test already authenticated.

Create auth.setup.ts that will prepare authenticated browser state for all other tests. The playwright documentation shows multiple scenarios and one of them is Authenticate with API request which I prefer.

In auth.setup.js

import { test as setup, expect } from '@playwright/test'

const authFile = 'playwright/.auth/user.json';

setup('authenticate', async ({ page }) => {
  const response = await page.request.post(`${API_URL}/auth`, {
    data: {
      email: MAIL,
      password: PASSWORD
    }
  })

  await page.context().storageState({ path: authFile })
})

Controlling the state with Playwright isn’t in my opinion as easy as with Cypress as access to window.localStorage isn’t straightforward. And API for changing localStorage isn’t coming.

Visiting external sites

• Avoid testing third-party dependencies
• Only test what you control.
• Don’t try to test links to external sites or third party servers that you do not control.
• Use the Playwright Network API and guarantee the response needed.

Playwright documentation has good examples of this, e.g.

await page.route('**/api/fetch_data', route => route.fulfill({
  status: 200,
  body: testData,
}));
await page.goto('https://example.com');

Set global baseURL

• Set a baseURL in your configuration file.

Adding a baseURL in your configuration allows you to omit passing the baseURL to actions like await page.goto('/').

In playwright.config.js:

use: {
    baseURL: 'https://your-awesome.app',
},

Helpful practices

• Use fixtures
• Use global setup and teardown
• See example setup from Playwright Boilerplate project which i.a. uses Page Object Model and has no selectors in .spec files.

Developing and running tests

Generate locators

Use codegen to generate locators

• Playwright has a test generator that can generate tests and pick locators for you. It will look at your page and figure out the best locator, prioritizing role, text and test id locators.

Use the VS Code extension to generate locators

• Use the VS Code Extension to generate locators as well as record a test.

Debugging

• Debug your tests live in VSCode
• Debug your tests with the Playwright inspector: npx playwright test --debug
• For CI failures, use the Playwright trace viewer instead of videos and screenshots.
• See traces in report: npx playwright show-report

Test across all browsers

• Testing across all browsers ensures your app works for all users. In your config file you can set up projects adding the name and which browser or device to use.

Lint your tests

• Use @typescript-eslint/no-floating-promises ESLint rule to make sure there are no missing awaits before the asynchronous calls to the Playwright API.
• If you’re using JavaScript, then use eslint-plugin-no-floating-promise.

Use parallelism

• Playwright runs tests in parallel by default. Tests in a single file are run in order, in the same worker process. If you have many independent tests in a single file, you might want to run them in parallel

test.describe.configure({ mode: 'parallel' });

Using parallelism requires that you’ve written your test isolated and they are independent. So think about how you structure your tests and use global setup and teardown.

Use Soft assertions

• You can also use soft assertions. These do not immediately terminate the test execution, but rather compile and display a list of failed assertions once the test ended.

// Make a few checks that will not stop the test when failed...
await expect.soft(page.getByTestId('status')).toHaveText('Success');

// ... and continue the test to check more things.
await page.getByRole('link', { name: 'next page' }).click();

Running in CI

• Playwright can also be setup on the CI environment of your choice.

For example running Playwright on Bitbucket Cloud pipeline:

In bitbucket-pipelines.yml

image: atlassian/default-image:3

pipelines:
  custom:
    run-e2e:
      - step: &run_e2e
          name: Run e2e tests
          services:
            - docker
          caches:
            - docker
          script:
            - npm ci
            - npx playwright install --with-deps
            - npx playwright test
          artifacts:
            - "test-results/**"
  pull-requests:
    "feature/**":
      - step: *run_e2e
  branches:
    master:
      - step: *run_e2e

Debugging on CI

• For CI failures, use the Playwright trace viewer instead of videos and screenshots. The trace viewer gives you a full trace of your tests as a local Progressive Web App (PWA) that can easily be shared.

In playwright.config.js:

use: {
    trace: 'on-first-retry',
},

Record success and failure screenshots and videos

• Playwright Test can record videos for your tests, controlled by the video option in your Playwright config. By default videos are off.

In playwright.config.js:

use: {
    screenshot: "only-on-failure",
    video: 'on-first-retry',
},

Note! Using the Playwright trace viewer is recommended instead of videos and screenshots.

Summary

• Make tests as isolated as possible
• Use locators to find elements on the webpage
• Use web first assertions such as toBeVisible()
• Use Page Object Model (POM)
• Set state programmatically, don’t use the UI to build up state.
• Avoid testing third-party dependencies, only test what you control.
• For CI failures, use the Playwright trace viewer instead of videos and screenshots