Coverage, Test Speed and Tests in CI

What you'll build

A coverage report with branch coverage for a pricing module, showing exactly which conditions no test exercises, a CI threshold that fails the build when coverage drops, and a CI layout that runs fast tests first.

The idea

• Line coverage shows which lines ran; branch coverage also checks both outcomes of each condition. Use branch coverage.
• Coverage finds untested code; it doesn't prove tests are good (a test with no assertions still "covers" code).
• Set a threshold (--cov-fail-under) to stop coverage eroding; ratchet it up, don't chase 100%.
• In CI: lint and unit tests first (seconds), then integration tests with real services, then a small end-to-end suite; run tests in parallel when suites grow.

The code

# file: pricing.py
def final_price(cents: int, member: bool, coupon: str | None = None) -> int:
    if member:
        cents = int(cents * 0.9)
    if coupon == "FREESHIP":
        cents -= 0                      # shipping handled elsewhere
    elif coupon == "TENOFF":
        cents = max(0, cents - 1000)
    if cents > 100_000:
        raise ValueError("needs manual approval")
    return cents

# file: test_pricing.py
from pricing import final_price

def test_members_get_ten_percent_off():
    assert final_price(2000, member=True) == 1800

def test_ten_off_coupon():
    assert final_price(2000, member=False, coupon="TENOFF") == 1000

# file: main.py
import pytest
pytest.main(["-q", "-p", "no:cacheprovider", "--cov=pricing", "--cov-branch", "--cov-report=term-missing",
             "--cov-fail-under=90"])

..
ERROR: Coverage failure: total of 78 is less than fail-under=90
                                                                         [100%]
================================ tests coverage ================================
_______________ coverage: platform linux, python 3.12.3-final-0 ________________

Name         Stmts   Miss Branch BrPart  Cover   Missing
--------------------------------------------------------
pricing.py      10      2      8      2    78%   5, 9
--------------------------------------------------------
TOTAL           10      2      8      2    78%
FAIL Required test coverage of 90% not reached. Total coverage: 77.78%
2 passed in <time>

The report lists lines 5 and 9 as never run (the free-shipping coupon and the manual-approval limit) and two partially covered branches. The build fails because coverage is below 90%. A CI pipeline might look like this:

# .github/workflows/tests.yml
name: tests
on: [push, pull_request]
jobs:
  unit:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-python@v6
        with: { python-version: "3.14" }
      - run: pip install -r requirements-dev.txt
      - run: pytest -m "not integration and not e2e" --cov --cov-branch --cov-fail-under=85 -n auto
  integration:
    needs: unit
    runs-on: ubuntu-latest
    services:
      postgres:
        image: postgres:18
        env: { POSTGRES_PASSWORD: test }
        ports: ["5432:5432"]
    steps:
      - uses: actions/checkout@v5
      - uses: actions/setup-python@v6
        with: { python-version: "3.14" }
      - run: pip install -r requirements-dev.txt
      - run: pytest -m integration

Common mistakes

• Treating a coverage percentage as a quality target and writing assertion-free tests to hit it.
• Line coverage only, missing untested else paths.
• One giant CI job where a slow end-to-end suite delays feedback on simple unit failures.
• Not tracking test duration; a suite that grows from 2 to 20 minutes changes how people work.

Key points

• Use branch coverage to find untested logic.
• A ratcheting threshold prevents erosion; 100% isn't the goal.
• Run fast tests first and parallelise as suites grow.

Further reading: pytest-cov · coverage.py: branch coverage · GitHub Actions: service containers