Python API and UI tests with pytest

By the end you will have a Python test project with API and browser tests, a fixture that keeps the tests independent, and a pipeline that runs them. We use the free ShopLite practice app, which ships with a starter-tests-python folder holding everything shown here. Python 3.10 or newer and Node 20 or newer are needed (the app is Node; the tests are Python).

Set up

Start the app in one terminal (npm install, then npm start; it listens on port 3001). In another terminal, in the starter-tests-python folder:

python -m venv .venv
source .venv/bin/activate            # Windows: .venv\Scripts\activate
pip install -r requirements.txt
playwright install chromium          # downloads a browser, once
pytest

requirements.txt has three lines: pytest, requests and pytest-playwright. A virtual environment keeps these packages away from the rest of your machine. If you already have Chrome, add --browser-channel chrome and skip the download.

Settings in one file

pytest.ini tells pytest where the app is and which tests to leave out of a normal run:

[pytest]
base_url = http://localhost:3001
addopts = -m "not bug" -ra
markers =
    bug: catches a planted ShopLite bug (fails on the buggy build, passes with BUGS=none)

base_url is read by pytest-playwright, so a browser test can call page.goto("/#/products") with a relative address. -ra prints a short summary of everything that did not simply pass. Registering the bug marker stops a typo from creating a different, silent marker.

A fixture that keeps tests independent

Tests that rely on what other tests did break when the order changes. The app has a reset endpoint, so one fixture in conftest.py puts the data back before every test:

@pytest.fixture(autouse=True)
def fresh_app(base_url):
    response = requests.post(f"{base_url}/api/_reset", timeout=10)
    assert response.status_code == 204, "Reset failed. Is the app running?"

autouse=True runs it for every test without being asked. The same file holds two more fixtures: api, a small helper around requests that always sends a timeout and an optional token, and demo_token, which logs in as the demo customer and returns the token. A test names the fixtures it needs as parameters and pytest builds them first.

API tests

def test_adding_to_the_cart_updates_the_total(api, demo_token):
    api("POST", "/api/cart/items", token=demo_token, json={"productId": 6, "qty": 2})
    cart = api("GET", "/api/cart", token=demo_token).json()
    assert cart["subtotal"] == 2 * 14900
    assert cart["total"] == cart["subtotal"] - cart["discount"] + cart["tax"] + cart["shipping"]

The test does one thing, then checks the numbers a customer cares about. The second assertion states the rule (total = subtotal - discount + tax + shipping) instead of copying a number, so it keeps working when prices change. Money is in paise, which is why 14900 means 149 rupees.

Browser tests

pytest-playwright gives each test a fresh browser page through the page fixture:

from playwright.sync_api import Page, expect

def test_search_filters_the_list(page: Page):
    page.goto("/#/products")
    page.get_by_test_id("search").fill("backpack")
    page.get_by_test_id("search-button").click()
    expect(page.get_by_test_id("product-name")).to_have_text(["Backpack"])

expect(...) retries until the page catches up or a timeout passes, so you do not write sleeps. get_by_test_id uses the data-testid attributes the app provides for exactly this purpose; they do not change when the design does. Add --headed --slowmo 300 to watch the browser.

Find the planted bugs

ShopLite has bugs on purpose. The tests in test_bugs.py are written to catch them and carry the bug marker:

pytestmark = pytest.mark.bug

def test_a_wrong_password_returns_401_not_403(api):
    response = api("POST", "/api/auth/login", json={"email": "[email protected]", "password": "wrong"})
    assert response.status_code == 401

Run them deliberately:

pytest -m bug

On the default build most fail, and each failure is a bug you found, with the expected and actual values in the report. One case passes on both builds; the comment in the file explains why, and it is a good lesson about malformed-input tests. To prove the tests themselves are right, restart the app as BUGS=none npm start (Windows PowerShell: $env:BUGS="none"; npm start) and run pytest -m bug again: they all pass.

One test, many inputs

parametrize runs a test once per input and reports each separately:

@pytest.mark.parametrize("email", ["a@b", "no-at-sign.example.com", "two@@example.com"])
def test_an_email_without_a_proper_domain_is_rejected(api, email):
    response = api("POST", "/api/auth/register", json={"name": "Asha Rao", "email": email, "password": "Passw0rd1"})
    assert response.status_code == 400

Run it in GitHub Actions

Put the app folder in a repository and add this file as .github/workflows/python-tests.yml. It is the workflow that ships with the app:

name: Python tests

on:
  push:
    branches: [main]
  pull_request:

concurrency:
  group: python-tests-${{ github.ref }}
  cancel-in-progress: true

jobs:
  pytest:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: npm
      - run: npm ci

      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
          cache: pip
          cache-dependency-path: starter-tests-python/requirements.txt
      - run: pip install -r requirements.txt
        working-directory: starter-tests-python
      - run: playwright install --with-deps chromium
        working-directory: starter-tests-python

      - name: Start the app
        run: npm start &
      - name: Wait until the app answers
        run: npx --yes wait-on http://localhost:3001/api/health

      - name: Run the tests
        run: pytest
        working-directory: starter-tests-python

What each part does:

• on runs it for pushes to main and for every pull request; concurrency cancels an older run of the same branch when a new push arrives.
• The Node steps are there because the app under test is a Node app. cache: npm and cache: pip reuse downloaded packages between runs.
• playwright install --with-deps chromium downloads the browser and the operating-system libraries it needs on the clean GitHub machine.
• npm start & puts the server in the background (it never finishes, so the step would hang otherwise), and wait-on polls the health address until the app answers, so the tests do not start too early.
• pytest runs from the folder that holds pytest.ini. If any test fails, the step fails and so does the pull request check.

This workflow passes the bug tests over (they are excluded by addopts), which is what you want from a gate. The workflow file was checked with actionlint, and the tests in it were run locally against the app; it has not yet been run on GitHub.

Next steps

1. Add a smoke marker to your fastest tests and run only those on pull requests with pytest -m smoke.
2. Log in once through the API and give the browser the token, so the UI tests skip the login form.
3. Add --junitxml=results.xml and upload the file as an artifact so failures can be read after the run (see the GitHub Actions tutorial).
4. Run in parallel with pytest-xdist. The reset fixture would make parallel workers wipe each other's data on one app, so give each worker its own app instance first.
5. Rehearse the pytest interview questions with your own repository open.