Where does personal data in an LLM feature typically leak?

Into every copy of the prompt and response: application logs, tracing tools, eval sets in version control, exports and spreadsheets, often with broad access and no retention limit.

What is reversible redaction and when is it useful?

Replacing sensitive values with tokens and storing the mapping securely, so the model and logs see only tokens while the application can restore the original where needed, for example in the reply to the user.

Why combine pattern matching with a Luhn check for card numbers?

Patterns alone match many digit sequences that are not cards; the Luhn checksum filters out most false matches.

PII and Secrets in Prompts and Logs — LLM Evals and Guardrails

Scenario

Example company: a support assistant logs full prompts and responses for debugging and copies conversations into the eval set.

What you see in production

A privacy review finds card numbers, email addresses and an API key pasted by customers in plain text across the logging system, the tracing tool, the eval set in Git and a spreadsheet shared with a contractor. The data is retained indefinitely and visible to the whole engineering team.

The naive code

logger.info("prompt=%s response=%s", prompt, response)        # everything, forever
eval_cases.append({"input": user_message})                     # copied as-is into Git

Reproduce it

Redact before sending, logging or saving, with reversible tokens where the original is needed later:

Redact card numbers, emails and keys with reversible tokensTry it yourself

import re

PATTERNS = {"CARD": r"\b\d(?:[ -]?\d){12,15}\b", "EMAIL": r"[\w.+-]+@[\w-]+(?:\.[\w-]+)+",
            "APIKEY": r"\b(?:sk|pk)_(?:live|test)_[A-Za-z0-9]{8,}\b"}

def redact(text, vault):
    for kind, pattern in PATTERNS.items():
        for match in set(re.findall(pattern, text)):
            token = f"[{kind}_{len(vault) + 1}]"
            vault[token] = match
            text = text.replace(match, token)
    return text

vault = {}
message = "My card 4111 1111 1111 1111 was charged twice, email me at [email protected]. Key sk_live_abc123XYZ789"
safe = redact(message, vault)
print("sent/logged:", safe)
print("vault keys :", sorted(vault))
reply = "I've refunded the duplicate charge on [CARD_1] and sent confirmation to [EMAIL_2]."
print("shown to user:", re.sub(r"\[[A-Z]+_\d+\]", lambda m: vault[m.group(0)] if "EMAIL" in m.group(0) else "card ending 1111", reply))

Output

sent/logged: My card [CARD_1] was charged twice, email me at [EMAIL_2]. Key [APIKEY_3]
vault keys : ['[APIKEY_3]', '[CARD_1]', '[EMAIL_2]']
shown to user: I've refunded the duplicate charge on card ending 1111 and sent confirmation to [email protected].

Root cause

Personal data enters through user messages and tool results, and every copy of a prompt (provider request, application logs, traces, eval sets, exports) becomes another place it lives. Without redaction at the boundaries and retention limits, sensitive data spreads to systems with broad access and long lifetimes.

The fix

Minimise: do not send data the model does not need; look up account details with tools instead of asking users to paste them.
Redact at boundaries: before sending to the model where possible, and always before logging, tracing and saving eval cases; use reversible tokens when the original must be restored in the reply.
Detect well: combine patterns (cards with a Luhn check, emails, known key formats) with a named-entity model for names and addresses.
Limit retention and access for logs and traces containing conversation content; separate access for full payloads.
Review your model provider's data handling settings (retention, training use) and document them.
Never commit real conversations to Git; anonymise eval cases and review them before adding.

Trade-offs

Redaction before the model can remove context the model needs (for example an order number); redact by type, deliberately.
Pattern detection misses some data and over-matches other data; measure on samples.
Reversible token vaults are themselves sensitive; store them with strict access and short retention.

Detect it in production

Periodic scans of logs, traces and eval sets for sensitive patterns.
Count of redactions per type over time; sudden drops can mean the redactor broke.
Access logs for full-payload trace data.

Key points

Every copy of a prompt is another home for personal data.
Minimise what you send, redact at every boundary, and use reversible tokens where needed.
Limit retention and access, and never commit real conversations to Git.

Further reading: OWASP LLM02: Sensitive information disclosure · OWASP logging cheat sheet · Luhn algorithm