GitHub Actions for a test suite

By the end of this tutorial your tests run by themselves on every push and every pull request, and a failing change cannot be merged by accident. We use the free ShopLite practice app, which comes with a few passing Playwright tests, so you can copy the files into a new GitHub repository and follow along. The workflow below is the one that ships with that app.

What you need

• A GitHub account and a new repository containing the ShopLite folder (the app is MIT licensed).
• The app's own tests (npm test) and the starter Playwright project in starter-tests/.
• Nothing to install on your machine for the pipeline itself: GitHub provides the machine.

The workflow

Create the file .github/workflows/ci.yml. GitHub runs every YAML file in that folder.

name: CI

on:
  push:
    branches: [main]
  pull_request:

# A newer push to the same branch cancels the run that is still going.
concurrency:
  group: ci-${{ github.ref }}
  cancel-in-progress: true

jobs:
  app:
    runs-on: ubuntu-latest
    timeout-minutes: 10
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: npm
      - run: npm ci
      - run: npm test

  starter-tests:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: npm
          cache-dependency-path: |
            package-lock.json
            starter-tests/package-lock.json
      - run: npm ci
      - run: npm ci
        working-directory: starter-tests
      - run: npx playwright install --with-deps chromium
        working-directory: starter-tests
      - name: Start the app
        run: npm start &
      - name: Wait until the app answers
        run: npx wait-on http://localhost:3001/api/health
        working-directory: starter-tests
      - name: Run the tests
        run: npx playwright test
        working-directory: starter-tests
      - name: Keep the results when something fails
        if: failure()
        uses: actions/upload-artifact@v4
        with:
          name: playwright-results
          path: starter-tests/test-results/
          retention-days: 7

Commit it and push. Open the Actions tab of your repository: a run appears, and you can click into each job to read the log of each step.

How to read it

Name and triggers. on says when it runs: on every push to main and on every pull request. Running on pull requests is the important part: the result shows up next to the change before anyone merges it.

concurrency. If you push twice quickly, the first run is cancelled instead of wasting minutes. The group name includes the branch, so different branches do not cancel each other.

Two jobs, run in parallel. The app job runs the app's own fast tests. The starter-tests job runs the browser tests. They are independent, so GitHub starts them together and the whole workflow takes as long as the slower one. If one needed the other to pass first, you would add needs: app.

runs-on: ubuntu-latest. A fresh Linux machine for every job. Nothing from the last run is left behind, which is exactly why a test that depended on leftover data will fail here and pass on your laptop.

timeout-minutes. A hung job stops after ten or fifteen minutes instead of burning the allowance for six hours (the default limit).

actions/checkout copies your repository onto the machine. actions/setup-node installs Node.js and, with cache: npm, saves the downloaded packages between runs so npm ci is quick next time. cache-dependency-path lists both lock files because there are two projects.

npm ci installs exactly what the lock file says, which is what you want in a pipeline (not npm install, which may change versions).

working-directory runs a step inside starter-tests/. Each run step starts in the repository root unless you say otherwise.

The important part: starting the app and waiting for it

The browser tests need the app running. Look at these three steps:

- name: Start the app
  run: npm start &
- name: Wait until the app answers
  run: npx wait-on http://localhost:3001/api/health
  working-directory: starter-tests
- name: Run the tests
  run: npx playwright test
  working-directory: starter-tests

npm start never finishes, so a normal step would wait forever. The & at the end puts it in the background and the step ends at once. But the app needs a second or two to start, so the next step uses wait-on to check the health URL until it answers. Skip that wait and the first test fails with "connection refused", a classic CI-only failure that looks flaky and is not.

Keeping evidence when it fails

- name: Keep the results when something fails
  if: failure()
  uses: actions/upload-artifact@v4
  with:
    name: playwright-results
    path: starter-tests/test-results/
    retention-days: 7

When a step fails, the later steps are skipped unless they say if: always() or if: failure(). This one runs only on failure and attaches the screenshots and traces to the run, where you can download them from the run's summary page. Without it you would only have the text log, and debugging a UI failure from text is slow. Playwright's trace: 'retain-on-failure' setting makes the trace appear in that folder.

Make it better

Run on more than one browser. Add a matrix to the test job:

strategy:
  fail-fast: false
  matrix:
    browser: [chromium, firefox]
steps:
  - run: npx playwright install --with-deps ${{ matrix.browser }}
  - run: npx playwright test --project=${{ matrix.browser }}

The job then runs once per browser, in parallel. fail-fast: false stops one failing browser from cancelling the other, so you see all the failures at once.

Run every night as well. Add a schedule and a manual button:

on:
  push:
    branches: [main]
  pull_request:
  schedule:
    - cron: '30 20 * * *'    # UTC. This is 02:00 in India.
  workflow_dispatch:

Use the nightly run for the slow, full suite and keep the pull request run small and fast.

Use a secret. If the tests need a password or token, never write it in the file. Add it under the repository's Settings, Secrets and variables, Actions, and read it like this:

- run: npx playwright test
  env:
    API_TOKEN: ${{ secrets.API_TOKEN }}

GitHub hides the value in the logs. Secrets are not passed to workflows from forks, by design.

Block merging when it is red. In Settings, Branches (or Rulesets), protect main and turn on Require status checks to pass before merging; choose the app and starter-tests checks. Now a failing change cannot be merged until it is fixed.

When it fails: common causes

Try it yourself

1. Put the app in a new repository and add the workflow. Watch it go green.
2. Change one assertion in starter-tests/tests/ui.spec.ts so it fails. Push, open the failed run, and find the failing step and the saved artifact.
3. Add the browser matrix and see two jobs appear.
4. Protect main, open a pull request with the broken test, and confirm that merging is blocked.

If you also want the Jenkins version of the same pipeline, see the Jenkins tutorial. For interviews, practise the CI/CD questions.