Use this retention and redaction table
Inventory every copy of a model interaction: prompt, retrieved text, tool call, provider trace, analytics event, and support console. Decide which fields are needed for each purpose before data is logged.
| Item | Check or owner | Evidence |
|---|---|---|
| Prompt | May include account details | Filter before storage |
| Retrieved chunk | May carry secrets | Store document ID, not text |
| Tool call | Needs audit trail | Log action and actor, mask values |
| Model reply | May repeat private data | Limit access and retention |
Test the boundary
Seed a fake account number and look for it across those stores. Redact at collection, then test exports and staff permissions. A masked UI can still leave the full value in an upstream trace.
Worked synthetic case
Synthetic case: A customer asks an assistant about a declined transfer and includes a full account number. The chat screen masks it, but the prompt trace and analytics event store the full value.
Seed a fake number, run one interaction, and search provider request logs, app traces, tool logs, error queues, analytics, support exports, and backups. The pass condition is that broad logs contain only an event ID or masked value and any restricted trace has an owner and expiry.
Redact before the first write, not only before display. Separate short-lived debugging from longer audit events. This makes debugging less convenient, so keep safe correlation IDs and narrow secure traces for real investigations.
The Nigeria Data Protection Act is the legal source for data handling; this page gives a test method. Set actual retention and lawful basis with the privacy owner rather than copying a sample period.
Test the log at every sink
Seed a fake account number and a fake identity value in a prompt. Trace the request through API logs, model provider request records, observability spans, support screenshots, and error reports. For each sink, record whether the value is stored, masked, or absent, and how long it remains. A redacted application log does not protect a full prompt copied into a trace attribute. If the value leaks, remove it at the logging boundary, then run the same request and search all sinks again. Keep an opaque request ID for debugging and restrict the separate source record. Do not publish raw prompts as proof when they contain customer data.
Use canary fields with distinct rules
Synthetic request LOG-41 contains name CANARY-NAME-41, account marker CANARY-ACCOUNT-41, and a fake access token CANARY-TOKEN-41. Define expected handling before running: broad analytics may keep LOG-41 and outcome only; restricted support may keep a masked account marker; no sink may store the token. Trigger success, validation error, model timeout, and a failed tool response. Search exact markers and transformed copies such as JSON-escaped text.
Record the sink and path that first stored each value. A request body captured by an error tracker before your redactor runs needs a fix at that capture point. Masking a later display does not repair the earlier copy. Prefer structured fields and a field allowlist to a broad regular expression that misses nested tool results.
Record what cannot be inspected
A provider may offer no request-body search, and encrypted backups may need a restore before inspection. Mark such sinks unverified. Check configuration, contract, and vendor controls as separate evidence; none substitutes for reading a copy when the test claims the value is absent. Use a disposable restore for one backup sample and record its date.
If logs follow a deletion schedule, test expiry in the active stores and the restore process. An immutable backup may age out under its own retention rule; a restore must not bring deleted records back into normal use without controls. The privacy owner sets the lawful retention basis and time. Keep test-marker results and role checks in the report, rather than real account data. OWASP’s logging guide supports excluding secrets and testing failed logging paths.