Data Handling: What Leaves, Where It Goes, Who Reads It

A support agent pastes a customer email into a prompt so the model can summarise it. The email contains a card number. That content is now in a third party's system, and the person who did it was doing their job.

Data handling is one of the clusters every vendor-risk questionnaire covers, and it is the one where vendors most often lose credibility — not by failing a control, but by over-promising and being caught on a detail. "Your data never leaves your environment" is rarely true in full, and a reviewer who finds one counter-example re-reads every other answer you gave.

Why the DLP you already own does not cover this

Most enterprises already run DLP for email, for file uploads, and for endpoints. None of it is positioned where an agent's traffic goes. The request leaves a process, not a mailbox, and it goes to an API rather than a recipient.

That leaves two honest options: inspect where the traffic is assembled, or accept that this class of leak is outside your controls and say so. Both are defensible in a review. Claiming coverage you do not have is not.

Your model provider is not your DLP. Their redaction is a courtesy in a terms page, not a control in your evidence pack.

Two scan layers, and they answer different questions

It is worth separating these, because they get conflated constantly — including, until this rewrite, in this post.

Content Shield looks for attacks on the model: attempts to override system instructions, change the assistant's identity, bypass safety checks, extract secrets, jailbreak, inject fake system tags, or exfiltrate data. It is about someone trying to make the model misbehave.

DLP looks for regulated content in the traffic itself — identifiers, financial data, credentials, contact details. It is about what is in the payload regardless of anyone's intent.

• Identity — national identifiers, including the Israeli Teudat Zehut.
• Financial — card numbers, IBANs.
• Credentials — API keys, JWTs, and password assignments in free text.
• Contact and network — email addresses, international phone numbers, IP addresses.

Deliberately no count here. The previous version of this post named a number, put it in the title, and got it wrong — it attributed the attack-pattern count to the DLP set, and then listed detections that did not exist. A number in marketing copy is a maintenance commitment that quietly drifts; the current set lives in the docs, which are generated from the code.

Scan both directions

Inbound scanning catches what a user pastes. Outbound scanning catches what the model produces, which matters more than it sounds — a model can reconstruct or echo an identifier that never appeared verbatim in the prompt.

Outbound scanning costs more, because there is more text and it arrives incrementally. That is a real trade-off and it belongs in the policy rather than in a default: the categories where an outbound leak is a reportable event are not the categories where it is a nuisance.

Three actions

For each class the policy picks one. Block stops the request and returns an error — correct when the content must never reach a third party. Redact substitutes a placeholder and lets the request through — correct when the model does not need the identifier to do the work. Alert lets it through and records it — correct while you are still learning what your traffic actually contains.

{
  "dlp": {
    "identity":    "block",
    "financial":   "block",
    "credentials": "block",
    "contact":     "redact",
    "scan_outbound": true
  }
}

Starting everything on alert is the honest way to begin. You will discover that traffic you assumed was clean is not, and you will make the block decisions with data instead of guesses. What you should not do is leave it there — alert is a discovery mode, and a reviewer can tell the difference between a policy and a pending decision.

One invariant: scan the original

This is the part worth stealing regardless of what you build on. Scanning has to happen on the content as it arrived, before any stage that can rewrite fields.

If a rewrite runs first, the scanner sees a transformed payload and its verdict describes something the user never sent. The finding becomes unreproducible, which is the worst property a security record can have — you cannot investigate it and you cannot defend it. Ordering is not a performance detail here; it decides whether the evidence means anything.

What this leaves for the questionnaire

The data-handling cluster is answered well by three things a reviewer can verify: which categories are inspected, what happens on a match, and a record of matches over a period long enough to cover the question. Retention is the part that gets skipped — a two-week log answers nothing about last quarter.

And be exact about custody. For Dobby's regulated modules, raw prompt and response content is routed to a dataset the customer owns rather than held by Dobby; run metadata and verdicts are processed in Dobby's managed control plane. Stating it at that resolution is stronger than a blanket reassurance, because it is the resolution at which it is true.

Where Dobby fits

Dobby reads run telemetry out-of-band and never sits in the request path. Where content inspection is in place, its findings land against the run and are evaluated against the frameworks you have activated.

Where the telemetry does not carry payload visibility, the data-handling controls return unverifiable rather than compliant, and the gap report names the missing evidence. This is the control where that state does the most work: a scanner that cannot see payloads cannot prove anything about them, and reporting a pass anyway would be the exact failure this cluster exists to catch.