A Jenkins pipeline for your test suite

Jenkins is still common in Indian companies, so knowing how to put your tests into a Jenkins pipeline is a useful skill. This tutorial builds a Jenkinsfile for the free ShopLite app and its Playwright tests. If you prefer a hosted tool, the GitHub Actions version does the same job.

What you need

• A Jenkins server with the Pipeline and Docker Pipeline plugins, and at least one agent that can run Docker. (The JUnit plugin is normally installed by default.)
• Your code in a Git repository that Jenkins can read, with the Jenkinsfile at the root.
• In Jenkins: New Item, Multibranch Pipeline (or Pipeline from SCM), pointed at the repository. Jenkins then builds every branch and pull request that has a Jenkinsfile.

The one command that does the hard part

The browser tests need the app running. In a pipeline you do not want to start it in the background by hand and remember to stop it. A small tool called start-server-and-test starts the server, waits until a URL answers, runs your tests, and stops the server whatever the result:

cd starter-tests
PLAYWRIGHT_JUNIT_OUTPUT_NAME=results/junit.xml \
  npx start-server-and-test "npm --prefix .. start" http://localhost:3001/api/health "npx playwright test --reporter=junit"

That command was run locally for this tutorial. It prints the six tests passing and leaves results/junit.xml, the standard format nearly every CI server can read.

The Jenkinsfile

pipeline {
  // Run inside a container that already has Playwright and its browsers.
  // The version in the image tag must match the @playwright/test version in package-lock.json.
  agent {
    docker {
      image 'mcr.microsoft.com/playwright:v1.50.0-jammy'
      args '--ipc=host'
    }
  }

  options {
    timeout(time: 20, unit: 'MINUTES')   // a hung build must not run for ever
    timestamps()                          // time on every log line
    buildDiscarder(logRotator(numToKeepStr: '30'))
  }

  parameters {
    string(name: 'GREP', defaultValue: '', description: 'Run only tests whose title matches, for example @smoke')
  }

  triggers {
    cron('H 2 * * *')                     // also every night; the H spreads the load
  }

  stages {
    stage('Install') {
      steps {
        sh 'npm ci'
        dir('starter-tests') { sh 'npm ci' }
      }
    }

    stage('App tests') {
      steps { sh 'npm test' }
    }

    stage('Browser tests') {
      steps {
        dir('starter-tests') {
          sh '''
            export PLAYWRIGHT_JUNIT_OUTPUT_NAME=results/junit.xml
            npx start-server-and-test "npm --prefix .. start" http://localhost:3001/api/health \
              "npx playwright test --reporter=junit ${GREP:+--grep $GREP}"
          '''
        }
      }
    }
  }

  post {
    always {
      junit allowEmptyResults: true, testResults: 'starter-tests/results/junit.xml'
      archiveArtifacts artifacts: 'starter-tests/test-results/**', allowEmptyArchive: true
    }
    failure {
      echo 'Tests failed. Open the build, then the Test Result page and the archived artifacts.'
    }
  }
}

How to read it

agent { docker { ... } } runs the whole pipeline inside a container started from Playwright's own image, so every build has the same browsers and no setup step is needed. The image tag has to match the Playwright version your project uses, or Playwright will complain it cannot find its browsers.

options are safety rails. The timeout stops a stuck build; timestamps() makes it easy to see where time went; buildDiscarder keeps only the last thirty builds so the disk does not fill.

parameters adds a "Build with Parameters" form. A tester can type @smoke and run only the smoke tests without editing code. The ${GREP:+--grep $GREP} part adds the option only when something was typed.

triggers { cron('H 2 * * *') } also runs it every night. In Jenkins cron the H means "a hashed minute", so many jobs do not all start at exactly 02:00. You normally add a push trigger as well: set up a webhook from your code host so a push starts a build at once.

stages run in order. If one fails, the rest are skipped, but post still runs.

post { always { ... } } is the important part. junit reads the XML and gives Jenkins its pass and fail counts, a trend graph across builds, and a "Test Result" page listing each failed test. archiveArtifacts saves the screenshots and traces from test-results so you can download them later, because the container is gone when the build ends.

Passwords and tokens

Never put them in the Jenkinsfile. Store them under Manage Jenkins, Credentials and bind them to a single step:

stage('API tests') {
  steps {
    withCredentials([string(credentialsId: 'api-token', variable: 'API_TOKEN')]) {
      sh 'npm run test:api'
    }
  }
}

Jenkins masks the value in the log. It does not protect you from a script that prints the value in a modified form, so never echo secrets.

Running tests in parallel

Two independent stages can run side by side:

stage('Tests') {
  parallel {
    stage('API tests')     { steps { sh 'npm run test:api' } }
    stage('Browser tests') { steps { sh 'npm run test:ui' } }
  }
}

Parallel branches that start the same server on the same port will collide; give each its own port or keep the server inside one branch.

Common problems

Try it yourself

1. Create the pipeline job, run it, and open the Test Result page.
2. Break one assertion and watch the build turn red; download the archived artifacts.
3. Start a build with GREP set to one test's title and confirm only that test ran.
4. Add a webhook from your Git host so a push starts the build.
5. Compare the experience with the GitHub Actions version, then read Jenkins vs GitHub Actions.