LLM Data Leakage: What an AI Might Reveal by Accident

The 30-second explanation

The assistant knows things: its hidden instructions, the files it can read, sometimes what earlier users asked. If nobody controls what it says, it can repeat those things to the wrong person.

How it works

1. 1

   1. Context

   Documents and instructions are given to the model.

2. 2

   2. Request

   A user asks something clever or broad.

3. 3

   3. Reveal

   The model repeats content it should not.

4. 4

   4. Control

   Authorisation and output checks prevent it.

The model itself does not know who is allowed to see what. Access control has to happen before information reaches it: only retrieve documents the current user may read, keep sessions separate, and never place secrets in prompts. After the model answers, output handling matters too: content should be checked, and features like automatic link rendering can be abused to send data elsewhere.

Real-world example

How to spot it

• Retrieval without permission checks

  Search that ignores who is asking.

• Secrets in prompts or context

  Keys or private details supplied to the model.

• Shared sessions

  Context that carries over between users.

• Unsafe output rendering

  Links or images built from model output that could carry data.

What to do

1. 1Check user permissions before retrieving any document for the model.
2. 2Keep secrets out of prompts and separate sessions and tenants strictly.
3. 3Review and constrain how model output is rendered, especially links and images.