What belongs in a Kustomize base versus an overlay?

Everything common to all environments goes in the base; overlays hold only the differences per environment, such as namespace, image tags, replica counts and resource patches.

What are Kustomize components for?

Optional, reusable bundles of resources and patches (for example a debug sidecar or a feature) that some overlays include and others don't.

How do you make environment differences reviewable?

Render every overlay in CI and diff the outputs between environments in pull requests, so every difference is visible and deliberate.

Copy-Pasted Environments That Drift — Helm, Kustomize and GitOps with Argo CD

Scenario

Example company: a repository has dev/, staging/ and prod/ folders with full copies of every manifest. A security fix (readOnlyRootFilesystem: true) is applied to dev/ and staging/, but prod/ is forgotten.

What you see in production

An audit finds production running with a writable root filesystem and an old probe configuration: dozens of small differences between environments, most unintentional. Nobody can say which differences are deliberate.

The naive code

deploy/
  dev/deployment.yaml        # full copy
  staging/deployment.yaml    # full copy, slightly different
  prod/deployment.yaml       # full copy, slightly more different

Reproduce it

A base plus overlays that contain only real differences, rendered with kubectl kustomize, and the environment differences listed:

One base, two overlays, explicit differences

mkdir -p base overlays/staging overlays/prod
cat > base/deployment.yaml <<'EOF'
apiVersion: apps/v1
kind: Deployment
metadata: {name: api}
spec:
  replicas: 1
  selector: {matchLabels: {app: api}}
  template:
    metadata: {labels: {app: api}}
    spec:
      containers:
        - name: api
          image: example/api:1.0.0
          securityContext: {readOnlyRootFilesystem: true}
EOF
printf 'resources: [deployment.yaml]\n' > base/kustomization.yaml
printf 'resources: [../../base]\nnamespace: staging\nimages: [{name: example/api, newTag: 1.5.0}]\n' > overlays/staging/kustomization.yaml
cat > overlays/prod/kustomization.yaml <<'EOF'
resources: [../../base]
namespace: prod
images: [{name: example/api, newTag: 1.4.2}]
replicas: [{name: api, count: 4}]
EOF
for env in staging prod; do kubectl kustomize overlays/$env > $env.yaml; done
echo "--- differences between staging and prod (all deliberate, all in the overlays):"
diff staging.yaml prod.yaml | grep '^[<>]' | sed 's/^</  staging:/; s/^>/  prod:   /'
echo "--- the security setting both environments inherit: $(grep -c 'readOnlyRootFilesystem: true' prod.yaml) in prod, $(grep -c 'readOnlyRootFilesystem: true' staging.yaml) in staging"

mkdir -p base overlays/staging overlays/prod
cat > base/deployment.yaml <<'EOF'
apiVersion: apps/v1
kind: Deployment
metadata: {name: api}
spec:
  replicas: 1
  selector: {matchLabels: {app: api}}
  template:
    metadata: {labels: {app: api}}
    spec:
      containers:
        - name: api
          image: example/api:1.0.0
          securityContext: {readOnlyRootFilesystem: true}
EOF
printf 'resources: [deployment.yaml]\n' > base/kustomization.yaml
printf 'resources: [../../base]\nnamespace: staging\nimages: [{name: example/api, newTag: 1.5.0}]\n' > overlays/staging/kustomization.yaml
cat > overlays/prod/kustomization.yaml <<'EOF'
resources: [../../base]
namespace: prod
images: [{name: example/api, newTag: 1.4.2}]
replicas: [{name: api, count: 4}]
EOF
for env in staging prod; do kubectl kustomize overlays/$env > $env.yaml; done
echo "--- differences between staging and prod (all deliberate, all in the overlays):"
diff staging.yaml prod.yaml | grep '^[<>]' | sed 's/^</  staging:/; s/^>/  prod:   /'
echo "--- the security setting both environments inherit: $(grep -c 'readOnlyRootFilesystem: true' prod.yaml) in prod, $(grep -c 'readOnlyRootFilesystem: true' staging.yaml) in staging"

Output

--- differences between staging and prod (all deliberate, all in the overlays):
  staging:   namespace: staging
  prod:      namespace: prod
  staging:   replicas: 1
  prod:      replicas: 4
  staging:       - image: example/api:1.5.0
  prod:          - image: example/api:1.4.2
--- the security setting both environments inherit: 1 in prod, 1 in staging

Run this on your computer — see “Get Started” for setup.

Root cause

Copies drift because every change must be repeated in every copy, and missing one is invisible. With full copies, intentional and accidental differences look the same, so reviews can't catch drift.

The fix

One base with everything common; overlays per environment containing only what differs (namespace, image tags, replicas, resources, a few patches).
Components for optional features shared by some overlays (for example "enable debug sidecar").
Render every overlay in CI and diff environments in pull requests.
Keep overlays small; if an overlay grows large, the base is probably missing a concept.

Trade-offs

Deep overlay hierarchies become hard to follow; keep at most two levels.
Some differences (whole optional resources) are clearer as components than patches.

Detect it in production

Rendered diff between environments reviewed regularly; anything unexpected is a bug.
Policy checks (security settings) run on every rendered overlay.

Key points

Copy-pasted environments drift silently.
Put common configuration in a base; overlays hold only real differences.
Render and diff every overlay in CI.

Further reading: Kustomize: bases and overlays · Kustomize components