Redact personal data before prompts and logs

Approach

Replace sensitive values with numbered placeholders before calling the model or logging, keep the mapping in memory for the request, and restore values only in the final response where the user should see them.

Solution

import re

PATTERNS = {
    "CARD": re.compile(r"\b\d(?:[ -]?\d){12,15}\b"),
    "EMAIL": re.compile(r"[\w.+-]+@[\w-]+(?:\.[\w-]+)+"),
    "IBAN": re.compile(r"\b[A-Z]{2}\d{2}(?:\s?[A-Z0-9]{4}){2,7}\b"),
}


def luhn_ok(number: str) -> bool:
    digits = [int(c) for c in re.sub(r"\D", "", number)][::-1]
    total = sum(d if i % 2 == 0 else (d * 2 - 9 if d * 2 > 9 else d * 2) for i, d in enumerate(digits))
    return total % 10 == 0


def redact(text: str) -> tuple[str, dict[str, str]]:
    vault: dict[str, str] = {}
    for kind, pattern in PATTERNS.items():
        for match in dict.fromkeys(pattern.findall(text)):
            if kind == "CARD" and not luhn_ok(match):
                continue
            token = f"[{kind}_{len(vault) + 1}]"
            vault[token] = match
            text = text.replace(match, token)
    return text, vault


def restore(text: str, vault: dict[str, str], allowed: set[str] = frozenset({"EMAIL"})) -> str:
    for token, value in vault.items():
        kind = token[1:].split("_")[0]
        text = text.replace(token, value if kind in allowed else f"{kind.lower()} ending {value[-4:]}")
    return text


safe, vault = redact("Card 4111 1111 1111 1111 was charged twice; email [email protected]")
print(safe)                                          # send this to the model and to logs
print(restore("Refund issued to [CARD_1]; receipt sent to [EMAIL_2].", vault))

Pitfalls

• Redacting the prompt but logging the raw request elsewhere (HTTP middleware, tracing auto-instrumentation).
• Regexes without validation (Luhn for cards) that redact order numbers and break answers.
• Storing the vault alongside the logs.

Variations

• Add a named-entity model for names and addresses.
• Look up account data with tools so users never need to paste it.

Further reading: OWASP LLM02: Sensitive information disclosure