Testing in Practice · Capstone
Capstone: A Test Strategy for a Checkout Service
Design the test strategy for an e-commerce checkout: levels, data, external dependencies, end-to-end journeys, CI and flakiness policy.
Scenario
A team owns the checkout of an online shop: a web front end, a checkout API (Python), PostgreSQL, a payment provider, a tax calculation service owned by another team, and an email service. Bugs in checkout cost money directly, but the current test suite takes 45 minutes, fails randomly about once a day, and still missed a tax rounding bug last month.
Requirements
- Catch pricing, tax and payment bugs before release.
- CI feedback within 10 minutes for most changes.
- No flaky failures blocking merges.
- Changes in the tax service's API must not silently break checkout.
Constraints
- Six developers, GitHub Actions, Docker available in CI.
Deliverables
- Test levels and what each covers.
- Handling external dependencies.
- Test data and the database.
- End-to-end coverage.
- CI pipeline and flakiness policy.
Hints
- Rounding and money bugs are a good fit for a particular testing technique.
- The tax service is owned by another team.
Draft each deliverable below before revealing the model answer and rubric.
Further reading: The practical test pyramid · pytest documentation
Deliverables
Which test levels do you use, and what does each cover?
Model answer
- Unit tests for pricing, discounts, tax rounding and state transitions (pure functions, many parametrised cases), plus Hypothesis properties: totals equal the sum of lines, rounding never loses or creates cents, discounts never make totals negative.
- API tests through TestClient for every endpoint, including validation and error responses.
- Integration tests for repositories and transactions against PostgreSQL in a container.
Rubric
Look for most logic at fast levels, property-based tests for money, and real-database integration tests.
How do you handle external dependencies?
Model answer
- Payment provider: a fake gateway in unit and API tests; the provider's sandbox in a small nightly suite; webhooks tested with recorded, signed payloads.
- Tax service: consumer-driven contract tests (Pact) published to the tax team's CI; a stub in our tests generated from the contract.
- Email: a fake that records messages; assert on content and recipients.
Rubric
Strong answers use fakes at boundaries, contracts for the cross-team API, and limited sandbox runs.
How do you manage test data and the database?
Model answer
- Schema migrations applied once per test session to a containerised PostgreSQL; each test in a rolled-back transaction or with truncation between tests.
- Factories for orders, carts and customers with sensible defaults; no shared fixtures that tests mutate.
- Seeded data only for the end-to-end environment, reset before each run.
Rubric
Look for isolated, repeatable data setup with the production engine.
What do the end-to-end tests cover?
Model answer
- A handful of Playwright journeys: guest checkout with card, logged-in checkout with saved address, a declined payment, applying a coupon.
- Role-based locators, web-first assertions, traces and screenshots on failure; payment provider in sandbox mode.
Rubric
Strong answers keep end-to-end tests few and critical, with robust locators and good failure artefacts.
How do you organise CI and handle flakiness?
Model answer
- Stages: lint and type checks plus unit tests (parallel, 2-3 min), API and integration tests with service containers (5 min), end-to-end on merge to main and nightly.
- Branch coverage threshold on the pricing and checkout modules; test durations tracked.
- Flakiness policy: a flaky test is quarantined only with a ticket and an owner, fixed within a week; tests run in random order (pytest-randomly) to expose dependencies; no blanket retries.
Rubric
Look for staged CI within the time budget, coverage where it matters, and a strict, owned flakiness policy.
Open in the app to mark lessons complete, take notes and track your progress.