Kubernetes in Production · Configuration and Security
Over-Privileged Service Accounts
A compromised pod with broad RBAC permissions owns the cluster; audit what each identity can do and grant only what it needs.
Scenario
Example company: a deployment tool, a monitoring agent and several apps all use service accounts bound to cluster-admin "to avoid permission errors". A vulnerable image in one app is exploited.
What you see in production
The attacker reads the pod's mounted service account token, lists all Secrets in all namespaces (database passwords, cloud keys), and creates a privileged pod on every node. The incident review finds 14 service accounts with cluster-admin.
The naive code
kubectl create clusterrolebinding app-admin --clusterrole=cluster-admin --serviceaccount=shop:default
Reproduce it
Compare what an over-privileged account and a least-privilege account may do:
kubectl create namespace prod-rbac > /dev/null
trap 'kubectl delete namespace prod-rbac --wait=false > /dev/null; kubectl delete clusterrolebinding prod-rbac-too-much > /dev/null 2>&1' EXIT
kubectl -n prod-rbac create serviceaccount too-much > /dev/null
kubectl -n prod-rbac create serviceaccount just-enough > /dev/null
kubectl create clusterrolebinding prod-rbac-too-much --clusterrole=cluster-admin --serviceaccount=prod-rbac:too-much > /dev/null
kubectl -n prod-rbac create role read-config --verb=get --resource=configmaps --resource-name=app-config > /dev/null
kubectl -n prod-rbac create rolebinding just-enough --role=read-config --serviceaccount=prod-rbac:just-enough > /dev/null
printf '%-40s %-10s %s\n' "action" "too-much" "just-enough"
for action in "get secrets -A" "create pods -n kube-system" "delete nodes" "get configmaps/app-config -n prod-rbac" "get configmaps/other -n prod-rbac"; do
a=$(kubectl auth can-i $action --as=system:serviceaccount:prod-rbac:too-much 2>/dev/null)
b=$(kubectl auth can-i $action --as=system:serviceaccount:prod-rbac:just-enough 2>/dev/null)
printf '%-40s %-10s %s\n' "$action" "$a" "$b"
done
Output
action too-much just-enough get secrets -A yes no create pods -n kube-system yes no delete nodes yes no get configmaps/app-config -n prod-rbac yes yes get configmaps/other -n prod-rbac yes no
Run this on your computer — see “Get Started” for setup.
Root cause
Service account tokens are mounted into pods by default, so any code running in the pod acts with that account's permissions. Broad bindings turn a single compromised container into cluster compromise. Permissions are usually granted generously to make an error go away and never revisited.
The fix
- One ServiceAccount per workload;
automountServiceAccountToken: falsefor workloads that don't call the API (most apps). - Namespaced Roles with specific verbs, resources and even
resourceNames; ClusterRoles only for genuinely cluster-wide needs. - No
cluster-adminfor workloads or CI; humans get it through short-lived, audited elevation. - Review regularly with
kubectl auth can-i --list --as=...or tools like rbac-tool, and alert on new cluster-admin bindings. - For cloud APIs, use workload identity (IRSA, Workload Identity) instead of long-lived keys in Secrets.
Trade-offs
- Fine-grained roles take time to design and maintain; start with the highest-risk identities (CI, operators, internet-facing apps).
- Some operators legitimately need broad permissions; isolate them in their own namespaces and review their RBAC.
Detect it in production
- Inventory of ClusterRoleBindings to cluster-admin and of roles with
*verbs or resources. - Audit logs: service accounts reading Secrets they've never read before.
- Pods with mounted tokens that never call the API.
Key points
- A mounted token gives code in the pod that service account's permissions.
- One service account per workload, least-privilege roles, no automount unless needed.
- Audit with
kubectl auth can-iand watch for broad bindings.
Further reading: RBAC good practices · Configure service accounts for pods
Practice questions
Why is binding cluster-admin to an application's service account dangerous?
Answer
The token is mounted in the pod, so anyone who compromises the application can do anything in the cluster, including reading all Secrets and creating privileged pods.
When should automountServiceAccountToken be false?
Answer
For workloads that don't call the Kubernetes API, which is most applications, so there's no token to steal.
How do you check what a service account is allowed to do?
Answer
kubectl auth can-i <verb> <resource> --as=system:serviceaccount:<ns>:<name>, or --list for everything, and review Role/ClusterRole bindings.
Open in the app to mark lessons complete, take notes and track your progress.