Sign in Get started

DevOps in Production · Capstone Case Studies

Capstone: A Delivery Platform for Twelve Teams

Twelve product teams deploy monthly with hand-run scripts. Design the pipelines, runtime and practices that let them deploy daily and safely.

Create a free account to save this page, take notes and track your progress. Sign in

Scenario

Example company: a 120-engineer company with 12 product teams and 40 services (Java, Node.js, Python) on AWS. Today: a shared Jenkins server with hand-written jobs, deploys once a month in a 4-hour window with SSH scripts, staging built by hand, secrets in Jenkins credentials, alerts on CPU and disk only. Change failure rate is high and every release has a rollback plan nobody has tested.

The CTO wants daily deploys within six months, without more incidents.

Requirements

  • Every team can deploy any service to production on demand, safely.
  • Reproducible builds and auditable releases (SOC 2 is coming).
  • No long-lived credentials in CI.
  • Recover from a bad release in minutes.

Deliverables

  1. Source control, branching and the CI pipeline.
  2. Runtime platform and deployment strategy.
  3. Configuration, secrets and environments.
  4. Observability and on-call.
  5. A six-month rollout plan and how you'll measure progress.

Hints

  • Standardise through a "golden path" that teams can adopt, not a mandate on day one.
  • Rollback speed depends on databases as much as on pipelines.
  • Measure with the DORA metrics before and after.

Draft each deliverable before revealing the model answer and rubric.

Further reading: DORA · Google SRE workbook: alerting on SLOs

Deliverables

Source control, branching and CI.

Model answer

Trunk-based development with short-lived branches, required reviews and status checks; feature flags for unfinished work. GitHub Actions (or GitLab CI) with reusable workflow templates per language: lint, unit tests, build image, scan (dependencies, image, secrets), sign and push to ECR tagged with the commit SHA, generate an SBOM. Target under 10 minutes with caching and parallel jobs. OIDC to AWS with per-repository roles; no stored keys.

Rubric

Full marks: trunk-based + flags, templated pipelines with security scanning and signing, build once tagged by SHA, OIDC. Deduct for per-environment rebuilds or stored credentials.

Runtime platform and deployment strategy.

Model answer

Amazon EKS (or ECS if the team lacks Kubernetes experience) with a standard Helm chart or Kustomize base per service type: readiness/liveness probes, graceful shutdown, resource requests from measurements, PodDisruptionBudgets. GitOps (Argo CD) promotes the same image digest dev -> staging -> prod via PRs to an environment repo. Canary releases with automated analysis (Argo Rollouts) for critical services, rolling updates elsewhere; one-click rollback = revert the environment PR. Database changes follow expand/contract, enforced by migration linting.

Rubric

Full marks: standardised manifests with probes/shutdown/resources, GitOps promotion of one digest, progressive delivery with automated rollback, migration discipline.

Configuration, secrets and environments.

Model answer

Config in Git with base + overlays; environment differences limited to scale, endpoints and credentials. Secrets in AWS Secrets Manager, synced with External Secrets Operator; workloads use IAM roles for service accounts (Pod Identity). Environments created by Terraform modules from the same code; drift detected by Argo CD and scheduled Terraform plans. Ephemeral preview environments for big changes.

Rubric

Full marks: config as code with minimal overlays, secrets in a manager with workload identity, IaC-built environments and drift detection.

Observability and on-call.

Model answer

OpenTelemetry SDKs/auto-instrumentation in the golden-path templates; metrics in Prometheus (Amazon Managed Prometheus), logs to a central store as structured JSON, traces to a tracing backend. Each user-facing service defines SLIs and SLOs; paging only on multi-window burn rates with runbooks; cause alerts as tickets. Team-owned on-call per service group with a weekly alert review; blameless postmortems with tracked actions.

Rubric

Full marks: standard instrumentation, SLO-based paging, actionable alerts with runbooks, sustainable on-call and postmortems.

Six-month rollout plan and measurement.

Model answer

  • Month 1: baseline DORA metrics; build the golden path with two volunteer teams; OIDC and secrets manager.
  • Months 2-3: migrate services team by team (templates do most of the work); GitOps promotion; SLOs for the top 10 user journeys.
  • Months 4-5: canary analysis for critical services, migration linting, alert clean-up, game-day rollback drills.
  • Month 6: retire the old Jenkins jobs and deploy windows.

Measure deployment frequency, lead time, change failure rate and time to restore monthly per team, plus pipeline duration, pages per shift and adoption of the golden path.

Rubric

Full marks: incremental adoption with early adopters, clear milestones, DORA baseline and tracking, and retirement of the old path.

Open in the app to mark lessons complete, take notes and track your progress.

Was this page helpful?