Customer Data in Our Logs

Scenario

Example deployment: to debug a failing payroll integration, an FDE turns on request/response body logging. The logs ship to the vendor's SaaS logging platform, which the customer's contract doesn't list as a sub-processor.

What you see in production

Two weeks later the customer's privacy officer finds employee names, bank account numbers and salaries in log excerpts pasted into a support ticket. The debug flag is still on. The customer invokes the incident clause in the contract; legal teams on both sides spend a month on notifications and audits.

The naive code

logger.debug("payroll response: %s", response.text)        # full body, every request
logging.basicConfig(level=os.environ.get("LOG_LEVEL", "DEBUG"))

Reproduce it

Structured logging with an allow-list of safe fields, plus redaction as a safety net:

import json
import re

SAFE_FIELDS = {"employee_id", "status", "pay_period", "error_code"}
IBAN = re.compile(r"\b[A-Z]{2}\d{2}[A-Z0-9]{11,30}\b")
EMAIL = re.compile(r"[\w.+-]+@[\w-]+\.[\w.]+")

def safe_log(event, record):
    fields = {k: v for k, v in record.items() if k in SAFE_FIELDS}
    dropped = sorted(set(record) - SAFE_FIELDS)
    line = json.dumps({"event": event, **fields, "dropped_fields": dropped})
    return EMAIL.sub("<email>", IBAN.sub("<iban>", line))      # safety net for values in free text

record = {"employee_id": "E-1042", "name": "Ana Silva", "iban": "DE89370400440532013000", "salary": 5200,
          "status": "failed", "error_code": "ACCOUNT_CLOSED", "pay_period": "2026-03",
          "message": "transfer to DE89370400440532013000 for [email protected] failed"}
print("naive: ", json.dumps(record)[:90] + "...")
print("safe:  ", safe_log("payroll.transfer_failed", record))

naive:  {"employee_id": "E-1042", "name": "Ana Silva", "iban": "DE89370400440532013000", "salary":...
safe:   {"event": "payroll.transfer_failed", "employee_id": "E-1042", "status": "failed", "error_code": "ACCOUNT_CLOSED", "pay_period": "2026-03", "dropped_fields": ["iban", "message", "name", "salary"]}

Root cause

Logging was treated as a private debugging tool when it's actually a data flow: logs are copied, retained, searched and shared, often outside the customer's environment. Full-body logging captured personal data, nothing redacted it, the debug setting had no expiry, and the destination wasn't an agreed sub-processor.

The fix

1. Log identifiers and outcomes, not payloads: structured logs with an allow-list of safe fields (IDs, statuses, error codes, timings).
2. Redact at the source as a safety net (emails, account numbers, tokens) before logs leave the process.
3. Time-boxed debug mode: debug logging per customer with an automatic expiry, and debug logs kept inside the customer's environment.
4. Agree data flows: which telemetry may leave the customer environment, where it's stored, for how long; keep the sub-processor list accurate.
5. Retention and access: short retention for detailed logs, access limited and audited.

Trade-offs

• Less logging can make debugging harder; invest in good error codes, correlation IDs and reproducible test fixtures instead.
• Regex redaction misses things; it's a safety net behind allow-listing, not a substitute.
• Keeping logs in the customer's environment means you need their help (or a diagnostics bundle) to debug.

Detect it in production

• Automated scanning of log streams for personal-data patterns, alerting on hits.
• Alerts when debug logging is enabled longer than its expiry.
• Periodic review of what each service logs, as part of security reviews.

Key points

• Logs are a data flow; treat them like any other copy of customer data.
• Allow-list fields, redact at the source, and time-box debug logging.
• Agree with the customer what telemetry may leave their environment.