Helm, Kustomize and GitOps with Argo CD · GitOps with Argo CD

Copy-Pasted Applications for Every Cluster

Hundreds of hand-written Application manifests drift and get forgotten; ApplicationSets generate them from clusters, directories or lists.

Create a free account to save this page, take notes and track your progress. Sign in

Scenario

Example company: the platform team runs 12 clusters (4 regions × dev, staging, prod) and 15 platform add-ons. Each add-on has an Argo CD Application per cluster: 180 YAML files, mostly identical.

What you see in production

A new cluster in a fifth region launches without the logging agent and network policies, because three of the fifteen Applications were never copied. Updating a chart version means editing dozens of files, and some are always missed.

The naive code

apps/
  logging-eu-dev.yaml
  logging-eu-staging.yaml
  logging-eu-prod.yaml
  ... 177 more

Reproduce it

A cluster generator: every registered cluster with a label gets each add-on, with values chosen from its labels:

Generate Applications from clusters and add-onsTry it yourself
clusters = [
    {"name": "eu-prod", "labels": {"env": "prod", "region": "eu"}},
    {"name": "us-prod", "labels": {"env": "prod", "region": "us"}},
    {"name": "eu-dev", "labels": {"env": "dev", "region": "eu"}},
    {"name": "ap-prod", "labels": {"env": "prod", "region": "ap"}},          # the new cluster
]
addons = ["logging", "network-policies", "cert-manager"]

def application(addon, cluster):
    env = cluster["labels"]["env"]
    return {"name": f"{addon}-{cluster['name']}",
            "path": f"addons/{addon}", "valuesFile": f"values-{env}.yaml",
            "destination": cluster["name"], "syncPolicy": "automated+selfHeal" if env == "prod" else "automated"}

generated = [application(a, c) for c in clusters for a in addons]      # matrix: clusters x add-ons
print(len(generated), "Applications generated from", len(clusters), "clusters x", len(addons), "add-ons")
for app in generated:
    if app["destination"] == "ap-prod":
        print(" ", app)

Output

12 Applications generated from 4 clusters x 3 add-ons
  {'name': 'logging-ap-prod', 'path': 'addons/logging', 'valuesFile': 'values-prod.yaml', 'destination': 'ap-prod', 'syncPolicy': 'automated+selfHeal'}
  {'name': 'network-policies-ap-prod', 'path': 'addons/network-policies', 'valuesFile': 'values-prod.yaml', 'destination': 'ap-prod', 'syncPolicy': 'automated+selfHeal'}
  {'name': 'cert-manager-ap-prod', 'path': 'addons/cert-manager', 'valuesFile': 'values-prod.yaml', 'destination': 'ap-prod', 'syncPolicy': 'automated+selfHeal'}

Root cause

Applications are per cluster and per component, so the number grows multiplicatively, and every copy is a chance to forget or diverge. Nothing ties "a new cluster" to "install the standard add-ons".

The fix

An ApplicationSet generates Applications from a template and generators:

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata: {name: platform-addons}
spec:
  generators:
    - matrix:
        generators:
          - clusters: {selector: {matchLabels: {tier: workload}}}
          - list: {elements: [{addon: logging}, {addon: network-policies}, {addon: cert-manager}]}
  template:
    metadata: {name: '{{.addon}}-{{.name}}'}
    spec:
      project: platform
      source:
        repoURL: https://git.example.com/platform/addons.git
        targetRevision: main
        path: 'addons/{{.addon}}'
        helm: {valueFiles: ['values-{{index .metadata.labels "env"}}.yaml']}
      destination: {server: '{{.server}}', namespace: '{{.addon}}'}
      syncPolicy: {automated: {prune: true, selfHeal: true}}
  goTemplate: true

Registering a cluster with the right labels is then enough for it to receive every add-on.

Trade-offs

  • A template bug affects every generated Application; roll out ApplicationSet changes progressively (the RollingSync strategy) and test on dev clusters first.
  • Generated Applications are harder to customise one-off; use labels and values files rather than exceptions.

Detect it in production

  • Clusters missing expected Applications (inventory compared with the add-on list).
  • Applications not owned by any ApplicationSet in platform namespaces (hand-made drift).

Key points

  • Applications multiply per cluster and component; copies drift.
  • ApplicationSets generate them from cluster, list, Git and matrix generators.
  • Label clusters and let generators decide what each receives.

Further reading: ApplicationSet controller · Generators

Practice questions

What problem does an ApplicationSet solve?

Answer

It generates many Argo CD Applications from one template and generators (clusters, lists, Git directories, matrices), replacing hand-copied Applications per cluster and component.

What does the matrix generator do?

Answer

It combines the outputs of two generators, producing one parameter set for every pair, for example every cluster × every add-on.

Why roll out ApplicationSet template changes progressively?

Answer

One template change affects every generated Application; a progressive strategy limits a bad change to a few clusters first.

Open in the app to mark lessons complete, take notes and track your progress.

Was this page helpful?