Cloud Engineering in Production · Security Incidents
The Function With Admin Rights
An SSRF bug in one service exposed credentials for a role that could do everything. Scope roles to what's used, add boundaries, and harden metadata access.
Scenario
Example team: an image-thumbnail service runs on EC2. It fetches images from URLs supplied by users. Its instance role was given AdministratorAccess "temporarily" during a deadline.
What you see in production
An attacker submits the URL http://169.254.169.254/latest/meta-data/iam/security-credentials/thumbnail-role. The service dutifully fetches it and returns the role's temporary credentials. With them, the attacker reads every S3 bucket in the account and creates a new IAM user for persistence. CloudTrail shows the calls coming from outside AWS using the instance role's credentials.
The naive code
def thumbnail(url):
image = requests.get(url, timeout=5).content # fetches any URL, including internal addresses
return make_thumbnail(image)
with the instance role allowed "Action": "*", "Resource": "*" and the instance metadata service allowing IMDSv1 (plain GET requests).
Reproduce it
Compare what the role was granted with what the service actually used, and what an attacker could do with it:
granted = ["*"]
used_by_service = {"s3:GetObject:acme-uploads/*", "s3:PutObject:acme-thumbnails/*"}
attacker_wants = ["s3:GetObject:acme-customer-exports/*", "iam:CreateUser:*", "s3:PutObject:acme-thumbnails/*"]
def allowed(perm, policy):
return "*" in policy or perm in policy
for label, policy in [("AdministratorAccess", granted), ("scoped role", sorted(used_by_service))]:
hits = [p for p in attacker_wants if allowed(p, policy)]
print(f"{label:20} attacker succeeds with {len(hits)}/{len(attacker_wants)}: {hits}")
Output
AdministratorAccess attacker succeeds with 3/3: ['s3:GetObject:acme-customer-exports/*', 'iam:CreateUser:*', 's3:PutObject:acme-thumbnails/*'] scoped role attacker succeeds with 1/3: ['s3:PutObject:acme-thumbnails/*']
Root cause
Two weaknesses combined. An SSRF bug let attackers make the server request internal addresses, including the metadata endpoint that hands out the role's credentials (IMDSv1 answers simple GETs). And the role had far more privilege than the service used, so stolen credentials meant full account access instead of access to two prefixes.
The fix
- Least privilege: scope the role to the exact actions and resources used (policy generation). IAM Access Analyzer can generate a policy from the role's CloudTrail activity.
- Require IMDSv2 (session tokens via PUT with a hop limit), which blocks most SSRF-based credential theft; set it as the account default.
- Fix the SSRF: allow-list destinations or resolve and block private, link-local and metadata addresses before fetching; fetch through an egress proxy.
- Permission boundaries for roles that teams create themselves, so no role can exceed an agreed maximum.
- Detect credential use from outside AWS (GuardDuty finding
InstanceCredentialExfiltration).
resource "aws_launch_template" "thumbnail" {
metadata_options {
http_tokens = "required" # IMDSv2 only
http_put_response_hop_limit = 1
}
}
Azure and Google Cloud: managed identities and service accounts obtain tokens from metadata endpoints that require a special header (Metadata: true, Metadata-Flavor: Google), which defeats simple SSRF; least privilege still applies.
Trade-offs
- Tightly scoped policies need updating when features change; generate them from activity and review in pull requests.
- IMDSv2 with hop limit 1 can break containers that need metadata through an extra network hop; raise the hop limit to 2 deliberately or use pod-level identity.
- Permission boundaries add complexity; use them where teams self-serve IAM.
Detect it in production
- Access Analyzer unused-access findings per role.
- Metadata service metrics: any IMDSv1 calls (
MetadataNoToken) mean something still uses it. - GuardDuty credential exfiltration findings; CloudTrail calls by instance roles from non-AWS IP addresses.
Key points
- Admin roles turn any application bug into an account takeover.
- Scope roles to used actions and resources; generate and review policies.
- Require IMDSv2 and fix SSRF; detect role credentials used outside the cloud.
Practice questions
How does SSRF lead to cloud credential theft, and how does IMDSv2 help?
Answer
A server tricked into requesting the metadata endpoint can return the instance role's temporary credentials. IMDSv2 requires a session token obtained with a PUT request and a hop limit, which simple SSRF requests can't provide.
How do you find the minimum permissions a role needs?
Answer
Start from the actions and resources the code uses; use IAM Access Analyzer policy generation from CloudTrail activity and unused-access findings to tighten existing roles.
What is a permission boundary?
Answer
A managed policy that sets the maximum permissions an IAM role or user can have, regardless of its identity policies; useful when teams create their own roles.
Open in the app to mark lessons complete, take notes and track your progress.