What is AI data leakage?

AI data leakage is the escape of sensitive data through an AI system. It can be a customer list pasted into a public chatbot, an internal assistant that quotes a salary spreadsheet to the wrong employee, or an agent tricked into e-mailing a file outside the company.

AI tools raise the risk because they read so much. A single question can touch several documents, call tools across apps and send all of that text to a model provider. The OWASP Top 10 for LLM Applications ranks sensitive information disclosure (LLM02) and prompt injection (LLM01) among the top risks for this reason.

What are the ways data leaks through AI?

There are six main leak paths. Most real incidents involve one or two of them.

1. Prompts. People type or paste confidential data into an AI tool. With unapproved tools, this is the classic shadow AI problem.
2. Retrieval oversharing. An assistant that searches company documents returns content the asker is not allowed to see, because the search ignores file permissions.
3. Tool results. An agent calls a CRM, database or mailbox and gets back more than it needs, such as whole records with personal data. That text goes to the model too.
4. Outputs. The answer repeats sensitive data to the wrong person, or an agent sends data out through e-mail, chat or an API call. Prompt injection often aims at this step.
5. Training and retention. The provider keeps prompts for a period, or uses them to improve models, depending on the plan and settings.
6. Logs and history. Conversation history, debug logs and analytics tools hold full prompts and answers long after the conversation ends.

How serious is each leak path?

Leak path	Example	Who sees the data	Main control
Prompts	Pasting a contract into a personal chatbot account	An unvetted provider	Approved tools, policy, masking
Retrieval oversharing	Assistant quotes the board pack to an intern	Colleagues without access	Permission-aware retrieval
Tool results	CRM lookup returns full customer records	Model provider, logs	Narrow tools, masking before the model
Outputs and actions	Injected instruction makes an agent e-mail a file out	Anyone outside	Approvals before sending, least privilege
Training and retention	Consumer plan keeps chats to improve models	The provider	Business terms, no-training commitments
Logs and history	Debug logs keep every prompt forever	Anyone with log access	Retention limits, access to logs

Why is retrieval oversharing so common?

Many AI assistants are built on retrieval augmented generation (RAG): search the documents, then pass the best matches to the model. The easy version indexes everything with one service account and filters nothing. Everyone can then ask about anything the service account could read.

Filtering by the asker’s permissions fixes the leak, but it has a hidden cost. When permissions are clustered by topic, for example a finance folder only finance can open, a search that is filtered after it has run can quietly return very little. In our benchmark on 100,000 documents with synthetic, team-shaped permissions, a hybrid search filtered afterwards returned nothing for 64% of searches when the asker could see 10% of the documents, and for 99% when they could see 1%. With the permissions inside the search, the asker got the same top ten results as a system holding only their documents (agreement with such a system, not a measure of relevance; the vector part of the search is exact up to 50,000 visible chunks). The figures and their conditions are on the benchmarks page, and the reasoning is in the benchmark write-up. The answer is to apply the asker’s permissions inside the search and make sure the search still reaches enough of what they may see. See RAG security for the details.

How do provider terms affect AI data privacy?

Where your data goes after it reaches a model matters as much as what you send. Check three things for every provider and plan:

• Training. Is your data used to train or improve models? For example, OpenAI states in its API data usage documentation that data sent to the API is not used for training unless you opt in.
• Retention. How long are prompts and outputs kept? The same OpenAI page says API abuse monitoring logs are kept for up to 30 days by default, with reduced retention options for eligible customers.
• Location and processors. Where is data processed, and which sub-processors touch it?

Consumer plans and business plans from the same provider often have different terms. That gap is one reason shadow AI is risky.

How do you prevent AI data leakage?

No single control closes every path. Layer them.

1. Give people an approved tool so they stop pasting data into personal accounts.
2. Apply permissions inside retrieval, so the AI only reads what the asker could open.
3. Keep tools narrow. Limit which tables, folders and actions an agent can reach.
4. Mask personal data before the model in documents and tool results. See PII masking for AI.
5. Require approval before an agent writes, sends or deletes anything.
6. Choose provider terms deliberately: no training, known retention, known location. Or run models in your own environment (see private AI).
7. Limit logs and retention, and control who can read conversation history.
8. Keep an audit trail of runs and approvals, and review it.

Frameworks help structure the work. The NIST AI RMF Generative AI Profile (NIST AI 600-1) lists data privacy and information security among the main generative AI risks and suggests actions for each.

What cannot be fully prevented?

• Prompt injection can be reduced but not eliminated. Limit the damage with least privilege and approvals.
• Detection gaps. Masking relies on patterns and detectors and will miss some values.
• People. Approvals only work if approvers read what they approve.

How promptev handles AI data leakage

• The asker’s identity and groups are applied inside the search, so a file they cannot open is never fetched. promptev follows Google Drive, SharePoint, OneDrive and Dropbox sharing. More on permission-aware answers.
• Masking hides e-mails, phone numbers, US SSNs, card numbers, IBANs, API keys and your own patterns in found documents and tool results before the model reads them.
• Agents use your own model keys, and your data is never used for training.
• Database tools are read-only by default and limited to picked tables. By default, connector tools that write ask for approval once, and destructive tools always ask.
• Each person sees only their own conversations, and conversation files can be kept for 30, 60, 180 or 365 days.
• An audit trail records every run, approval and settings change, with who did it.