Site icon Digital Thought Disruption

How to Keep Secrets and Sensitive Data Out of AI Prompts, Embeddings, Traces, and Logs

Paul Bryant

TL;DR

Sensitive data protection for AI is not a prompt-writing problem. It is a data-path control problem.

The safest operating model classifies data before it enters the AI workflow, blocks credentials and restricted records by default, redacts or tokenizes approved confidential data, preserves source permissions in retrieval, disables prompt and tool-content telemetry unless explicitly approved, scans repositories and prompt assets for secrets, encrypts every persistent store, and gives each data surface a defined retention and deletion path.

When a leak occurs, treat the exposed value as compromised. Stop capture, revoke or rotate credentials, quarantine affected indexes and telemetry, determine every replicated copy, purge or rebuild the contaminated data set, and validate that the system cannot reproduce the exposure before service is restored.

Introduction

An AI application can leak sensitive data without ever suffering a traditional database breach.

A developer pastes a production token into a troubleshooting prompt. A retrieval pipeline embeds a restricted document into a shared vector index. An agent writes tool arguments into a trace. A debug logger records the complete prompt and response. A support engineer exports telemetry to a ticket. A backup preserves data that the primary system later deletes.

Each individual action may look operationally convenient. Together, they create a distributed sensitive-data problem across systems that were rarely designed under one retention, access, and incident-response model.

This is why “do not paste secrets into AI” is necessary but insufficient. User training cannot protect a workflow that automatically copies raw content into prompts, embeddings, caches, traces, logs, evaluation sets, and long-lived observability platforms.

A production design must assume that sensitive data will eventually approach the AI boundary. The architecture should decide what happens before the model, vector database, or telemetry backend receives it.

Scenario and Operational Risk

Assume an enterprise AI assistant can answer operational questions using internal documentation and call approved tools. The application uses retrieval-augmented generation, distributed tracing, centralized logs, a prompt repository, and an evaluation pipeline.

During an incident review, the security team discovers that a complete database connection string appeared in a trace attribute. The same value was also present in the original user prompt, an application log, a failed-request dead-letter queue, and a copied troubleshooting ticket.

The first discovery is not the full incident. It is only the first confirmed location.

AI data exposure expands quickly because one request can create multiple replicas:

SurfaceHow sensitive data arrivesWhy it persists
Prompt and conversation stateUser input, system instructions, memory, retrieved contextSession history, caches, provider retention, debugging
Embeddings and vector storesDocuments, tickets, source code, records, chat transcriptsIndexes, replicas, snapshots, stale chunks
Tool callsArguments, results, API errors, file contentTraces, audit records, retry queues
Traces and logsAutomatic instrumentation, exception capture, request bodiesCentral retention, exports, dashboards, archives
Evaluation dataFailed examples, human review sets, regression casesVersioned test corpora and experiment tracking
Build and source systemsPrompt templates, notebooks, configuration, test fixturesGit history, pull requests, issue comments, artifacts
Backups and exportsSnapshots, support bundles, analyst downloadsIndependent retention and delayed deletion

The operational risk is not limited to disclosure. Sensitive data can also be retrieved by the wrong user, retained beyond policy, copied into a lower-trust environment, exposed to a third-party processor, or preserved after the source record was deleted.

Protection Scope and Assumptions

This runbook applies to production and preproduction AI systems that use one or more of the following:

The runbook assumes the organization already has an identity provider, a secrets manager, encryption key management, incident response, data ownership, and a basic information-classification policy. The AI platform should consume those controls rather than invent weaker parallel versions.

This is a technical operating model, not a substitute for legal, privacy, records-management, or regulatory guidance. Retention periods, breach-notification requirements, and approved processing locations must be set by the appropriate organizational owners.

The Sensitive Data Control Path

The most important architectural decision is where policy enforcement occurs. A warning in the user interface is advisory. A system prompt telling the model not to expose data is probabilistic. A gateway that blocks or transforms content before it reaches the model is enforceable.

The reader should notice that classification and policy decisions occur before prompt assembly, embedding generation, and telemetry export. The model is not the security boundary.

A mature implementation repeats inspection at more than one point. Input controls stop sensitive content before processing. Retrieval controls prevent unauthorized context from being selected. Output controls catch generated or retrieved disclosures. Telemetry controls prevent diagnostic systems from becoming the final leak destination.

Build a Classification Model That AI Systems Can Enforce

A policy that says “protect sensitive data” is not executable. The AI data path needs a small, testable classification model with a defined action for each class.

A practical baseline is:

ClassificationTypical examplesDefault AI action
PublicPublished documentation, approved public contentAllow
InternalRoutine procedures, non-sensitive architecture notesAllow in approved environments with normal access controls
ConfidentialCustomer records, internal financial data, employee information, proprietary codeRedact, tokenize, minimize, or require an approved use case
RestrictedRegulated records, high-impact personal data, legal hold material, sensitive investigationsDeny unless a specifically approved architecture exists
CredentialPasswords, private keys, access tokens, connection strings, signing materialDeny, alert, revoke or rotate if exposure is real

Do not classify only by document. Classification can change at the field, chunk, query, and output level. A public runbook can still contain a copied credential. An internal ticket can contain a restricted customer record. A tool result can combine several classes in one response.

Assign an Action, Owner, and Evidence Requirement

Each classification needs three operational properties:

Without ownership, every team assumes another layer is filtering the data. Without evidence, the organization cannot prove that a control ran or determine why an exception was allowed.

Put DLP and Redaction Before Prompt Assembly

Prompt assembly is where system instructions, user messages, conversation memory, retrieved documents, and tool results become one model request. It is also the last reliable point to stop data before it leaves the application boundary.

Inspect each input independently before concatenation. That preserves provenance and makes policy decisions explainable.

Separate Secrets from Instructions

Never place credentials, private endpoints, signing material, database passwords, or authorization rules inside a system prompt. A system prompt is model input, not a secrets manager or policy engine.

The safer pattern is:

An agent that calls a billing tool should receive the approved invoice status, not the tool’s bearer token, database connection string, or raw backend response.

Choose the Transformation by Use Case

Redaction is not always the right transformation.

For example, replacing a real customer identifier with [CUSTOMER_ID_1] can preserve the structure of a troubleshooting conversation without exposing the source value. The mapping must remain outside model and telemetry paths, in a tightly controlled store, when re-identification is required.

Use a Deny-by-Default Policy

The following vendor-neutral YAML shows the policy shape. It is not tied to a specific gateway. Replace the data classes, exception workflow, retention periods, and owner groups with organizational values.

policy_version: 1
policy_name: ai-sensitive-data-boundary

default_action: deny

classifications:
  public:
    prompt_action: allow
    embedding_action: allow
    telemetry_content: deny

  internal:
    prompt_action: allow
    embedding_action: allow_with_source_acl
    telemetry_content: deny

  confidential:
    prompt_action: redact_or_tokenize
    embedding_action: require_approved_dataset
    telemetry_content: deny

  restricted:
    prompt_action: deny
    embedding_action: deny
    telemetry_content: deny
    exception_required: true

  credential:
    prompt_action: deny_and_alert
    embedding_action: deny_and_alert
    telemetry_content: deny_and_alert
    incident_severity: high

telemetry:
  capture_prompt_content: false
  capture_response_content: false
  capture_tool_arguments: false
  capture_tool_results: false
  allowed_attributes:
    - service.name
    - deployment.environment.name
    - gen_ai.agent.name
    - gen_ai.operation.name
    - gen_ai.provider.name
    - gen_ai.request.model
    - gen_ai.response.model
    - gen_ai.usage.input_tokens
    - gen_ai.usage.output_tokens
    - error.type

exceptions:
  require_owner: true
  require_expiration: true
  require_ticket: true
  maximum_duration_hours: 24

Successful enforcement means a blocked request never reaches the model, embedding service, retry queue, or content-bearing telemetry field. A policy that blocks the provider request but logs the rejected raw prompt has still failed.

Use Layered Detection Instead of One Regex

Sensitive-data detection should combine several methods because no single detector covers every data type.

A practical pipeline can include:

Detection systems produce false positives and false negatives. The answer is not to disable them. Tune them with labeled examples from the organization’s real data and measure both precision and recall.

Add a Sanitization Boundary in Code

The following simplified Python example shows an application-side boundary. It blocks credentials, replaces selected confidential values with typed placeholders, and returns findings separately from sanitized content. It intentionally does not log the original text.

In production, replace the example patterns with a supported DLP or PII detector, organization-specific recognizers, and a policy service. Keep the interface stable so the application does not need to know which detection engine is underneath it.

from __future__ import annotations

import re
from dataclasses import dataclass
from enum import StrEnum
from typing import Iterable


class DataClass(StrEnum):
    CONFIDENTIAL = "confidential"
    CREDENTIAL = "credential"


@dataclass(frozen=True)
class Finding:
    name: str
    data_class: DataClass
    start: int
    end: int


class SensitiveDataBlocked(RuntimeError):
    pass


DETECTORS: tuple[tuple[str, DataClass, re.Pattern[str]], ...] = (
    (
        "private_key_material",
        DataClass.CREDENTIAL,
        re.compile(r"-----BEGIN [A-Z ]+PRIVATE KEY-----"),
    ),
    (
        "generic_bearer_value",
        DataClass.CREDENTIAL,
        re.compile(r"(?i)\bbearer\s+[a-z0-9._~+/=-]{20,}"),
    ),
    (
        "email_address",
        DataClass.CONFIDENTIAL,
        re.compile(r"\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}\b", re.I),
    ),
)


def detect(text: str) -> list[Finding]:
    findings: list[Finding] = []
    for name, data_class, pattern in DETECTORS:
        for match in pattern.finditer(text):
            findings.append(
                Finding(
                    name=name,
                    data_class=data_class,
                    start=match.start(),
                    end=match.end(),
                )
            )
    return sorted(findings, key=lambda item: item.start)


def sanitize(text: str, findings: Iterable[Finding]) -> str:
    findings_list = list(findings)

    if any(item.data_class == DataClass.CREDENTIAL for item in findings_list):
        raise SensitiveDataBlocked("Credential-like content was blocked")

    output = text
    for item in sorted(findings_list, key=lambda value: value.start, reverse=True):
        replacement = f"[{item.name.upper()}]"
        output = output[: item.start] + replacement + output[item.end :]

    return output


def prepare_model_input(raw_text: str) -> tuple[str, list[str]]:
    findings = detect(raw_text)
    sanitized = sanitize(raw_text, findings)
    finding_names = sorted({item.name for item in findings})
    return sanitized, finding_names

The application should record that a credential detector blocked the request, along with a policy ID, request ID, environment, and detector version. It should not record the matched value or the surrounding raw text.

What can go wrong:

For those reasons, use the code pattern as an architectural boundary, not as a complete enterprise DLP implementation.

Protect Embeddings and Vector Stores as Sensitive Data Systems

Embeddings are not a privacy eraser. They are derived from source data and can preserve information that should remain protected. The vector store also usually contains chunk text, document metadata, source identifiers, access labels, and retrieval history.

Treat the retrieval layer as a data platform with the same rigor as a database.

Preserve Source Authorization

Every indexed chunk should retain enough metadata to enforce the source system’s permissions at query time. At minimum, include:

Do not rely on the model to ignore a retrieved chunk that the user was not authorized to see. Unauthorized content should never enter the model context.

Partition High-Risk Data

Shared indexes increase the blast radius of a permission error. Use separate collections, namespaces, projects, accounts, or clusters when tenant, regulatory, geographic, or administrative boundaries require stronger isolation.

Logical filters are useful, but they are not always an adequate substitute for physical or administrative separation. The right boundary depends on the impact of a missed filter and the ability to prove isolation.

Design Deletion Before Ingestion

A source-record deletion must propagate through:

Store a stable lineage key so the platform can find every derived object for a source record. If the only way to delete one record is to rebuild the entire index, document that behavior and make the rebuild process repeatable.

Make Observability Useful Without Capturing Content

AI observability does not require full prompt capture. Operators can diagnose most availability, latency, model-routing, token, retry, and tool-performance problems with metadata.

A production-safe telemetry contract should prefer:

It should exclude by default:

Filter at the Application and Collector

Filtering only at the telemetry backend is too late. Raw content may already have crossed networks, entered queues, or been retained by an agent or collector.

Use at least two enforcement points:

  1. Application instrumentation: do not create sensitive attributes.
  2. Collector or gateway: delete, hash, or transform prohibited fields before export.

Backend controls are still useful as a final guardrail, but they should not be the first place sensitive content is removed.

Treat Debug Capture as a Controlled Exception

There are cases where short-lived content capture is necessary to diagnose a production failure. That should be an exception workflow, not a permanent feature flag.

A controlled debug session should define:

Prefer synthetic or replayed sanitized requests before enabling production content capture.

Scan Source Control, Prompts, and Build Artifacts for Secrets

Secrets do not enter AI systems only through user prompts. They also appear in:

Enable secret scanning across the full repository history, not only the latest branch. Add push protection or an equivalent pre-receive control so known credentials are blocked before they become durable history.

Use custom patterns for organization-specific tokens and internal connection strings. Validate findings when the scanning platform supports it, but do not delay revocation of a confirmed credential while waiting for perfect classification.

A leaked secret should be rotated or revoked. Deleting the line from the repository does not make a credential safe again.

Set Retention by Surface, Not by Application

One “AI retention period” is rarely sufficient because different data surfaces serve different purposes.

Use a retention matrix such as the following as a design starting point. Final values must align with records, privacy, security, operational, and legal requirements.

Data surfaceRecommended default postureRequired deletion trigger
Metrics without contentRetain long enough for capacity and trend analysisEnd of approved operational period
Traces without prompt or tool contentShort operational windowAge-based deletion or incident closure
Debug traces with approved contentHours or a few days, not routine retentionAutomatic expiration plus verified purge
Application logsStructured metadata only, shortest useful periodAge, incident closure, or policy change
Conversation memoryUser- and use-case-specificSession expiration, user request, account closure
Vector chunks and embeddingsMatch source authorization and lifecycleSource deletion, permission change, dataset retirement
Evaluation setsSanitized and versionedTest retirement, consent or source deletion obligation
Security audit evidenceGoverned security retentionPolicy-defined expiration or legal hold release
Backups and snapshotsEncrypted and time-boundedBackup lifecycle expiration and deletion verification

Retention should be enforceable through lifecycle policies, not calendar reminders. Every exception should expire automatically.

Verify Deletion Across Replicas

Deletion proof should answer:

A successful API deletion response is not sufficient evidence when the workflow created multiple derived copies.

Encrypt Every Persistent Store and Separate Key Ownership

Encryption does not replace redaction or access control, but it reduces the impact of storage theft, snapshot exposure, and misplaced exports.

Protect:

Use encryption in transit and at rest. Prefer customer-managed keys or equivalent controls when risk, regulation, or operational ownership requires them. Separate key administrators from data administrators where practical, and restrict export, snapshot, and restore permissions.

Service identities should receive only the permissions required for their role. The embedding worker should not automatically have broad read access to every document collection. The telemetry collector should not be able to decrypt restricted source data. The model runtime should not be able to enumerate secrets.

Run Validation Before Production and After Every Material Change

Sensitive-data controls need regression tests just like application logic. Build a synthetic corpus containing fake credentials, personal information, proprietary terms, encoded values, and known false positives.

Admission-Control Tests

Confirm that:

Retrieval Tests

Confirm that:

Telemetry Tests

Confirm that:

Secret-Scanning Tests

Commit synthetic test patterns in an isolated test repository and verify that scanning and push protection detect them. Do not use active credentials for control testing.

Release Gate

Fail the release when any synthetic restricted value reaches a model request, embedding record, trace, log, evaluation set, or export without the expected policy action.

Contain a Sensitive Data Leak as a Multi-Surface Incident

When sensitive content is found, do not begin by deleting the first log entry. First contain active exposure and preserve enough non-sensitive evidence to determine scope.

The decision flow below separates credential response from non-credential data response while preserving a common containment and validation path.

Containment Actions

Use the following sequence:

  1. Classify the exposed value. Determine whether it is a credential, personal data, proprietary information, regulated content, or a combination.
  2. Revoke or rotate active credentials immediately. Assume copying occurred even when access logs do not show misuse.
  3. Disable content-bearing telemetry. Turn off prompt, response, tool-argument, and tool-result capture at the application and collector.
  4. Pause affected ingestion. Stop new documents from entering the contaminated pipeline.
  5. Restrict access. Reduce access to affected dashboards, vector collections, buckets, queues, and exports.
  6. Quarantine affected indexes or data sets. Prevent further retrieval while scope is determined.
  7. Preserve safe evidence. Retain identifiers, timestamps, hashes, policy versions, and access events without copying the sensitive value into the incident record.

Scope the Replication Path

Search every system that could have received the value:

Search with a cryptographic hash, stable fingerprint, source record ID, or tightly restricted exact-match procedure when possible. Avoid copying the raw value into general-purpose search tools.

Eradication and Recovery

Depending on the surface, remediation may require:

Restore service only after a validation test confirms that the value cannot be retrieved, reproduced, or observed through the affected paths.

Notification and Post-Incident Work

Security, privacy, legal, records management, data owners, application owners, and third-party providers may all need to participate. The incident process should determine notification obligations based on the data, affected people or systems, processing locations, contracts, and applicable requirements.

The post-incident review should produce control changes, not only awareness reminders. Ask which automated boundary failed, why the data was replicated, why retention allowed it to persist, and which test should prevent recurrence.

Common Failure Modes

Relying on a System Prompt to Protect Secrets

A prompt instruction is not deterministic access control. Secrets and authorization logic should remain outside model-visible context.

Redacting the Prompt but Logging the Original

The blocked or transformed request often appears in debug messages, exception objects, queue payloads, and trace events. Inspect the failure path, not only the successful provider call.

Treating Embeddings as Anonymous

Derived data can still be sensitive, and vector stores commonly retain source text and metadata. Apply classification, access control, encryption, lineage, and deletion.

Capturing Everything for Observability

Full content may make one debugging session easier while creating a long-lived privacy and security problem. Start with metadata, then enable narrowly scoped content capture only through an expiring exception.

Using DLP as the Only Control

Detectors miss context, encoded content, novel secret formats, and business-sensitive information that has no obvious pattern. Combine DLP with source authorization, minimization, isolation, output filtering, retention, and incident response.

Deleting Only the Primary Record

AI workflows create derived and replicated data. A complete deletion must include chunks, embeddings, caches, logs, evaluations, snapshots, and exports according to their approved lifecycle.

Keeping Permanent Exceptions

A temporary diagnostic flag becomes a permanent data-collection path unless it has automatic expiration. Exceptions need an owner, reason, scope, and end time.

Operational Ownership

Sensitive-data protection fails when every team owns one component but nobody owns the complete path.

Control areaPrimary ownerRequired partners
Classification and approved useData ownerPrivacy, security, legal, records management
Prompt and input inspectionApplication teamAI platform, security engineering
DLP and redaction serviceSecurity or platform teamApplication teams, data owners
Vector authorization and lifecycleAI platform or data platformSource-system owners, identity team
Telemetry contract and filteringObservability platformApplication teams, security, privacy
Secrets management and rotationSecurity platformApplication and infrastructure teams
Retention and deletionRecords and data governancePlatform owners, privacy, legal
Incident containmentSecurity incident responseAll affected system and data owners

One role should be accountable for proving the end-to-end control, even when implementation spans several platforms.

Production Readiness Checklist

Before enabling production traffic, confirm:

Conclusion

Keeping secrets and sensitive data out of AI systems requires more than asking users to be careful. The control must exist in the architecture.

Classify data before it enters the workflow. Block credentials and restricted content by default. Redact, tokenize, or minimize confidential data only when the use case is approved. Preserve source authorization through retrieval. Treat embeddings and vector stores as sensitive data systems. Capture observability metadata without collecting raw content. Scan code and prompt assets before they become durable history. Encrypt persistent stores, define retention by surface, and design deletion before ingestion.

When exposure occurs, assume replication. Rotate active credentials, stop capture, quarantine affected data sets, trace every derived copy, purge or rebuild contaminated stores, and prove that the value cannot be recovered before restoring normal operation.

The practical security boundary is not the model. It is the set of deterministic controls around the complete AI data path.

External References

Exit mobile version