Technically AcceptableIndependent technical publication
← Writing archive

Designing AI Security Around the Three Ingredients

Untrusted content, sensitive data, and outbound actions cannot share one component.

  • AI Security
  • Local AI
Diagram comparing a single AI assistant that combines untrusted content, sensitive data, and outbound access with separated reader and planner components that are expected to block exfiltration at an allowlist.

The build reads documents from outside and holds data that should not leave. That is the whole problem stated once.

This is the current design state. Nothing has been run against it yet.

The security does not live in the model. It lives in the shape of the system around it.

The reading that set the direction

Two sources set the direction for this build.

The first is that prompting alone does not provide a durable security boundary. In one adaptive evaluation, attacks bypassed all eight prompt injection defenses tested and exceeded 50 percent attack success against each.

The second is CaMeL, from researchers at Google, Google DeepMind, and ETH Zurich, plus the system-level design patterns Beurer-Kellner and colleagues published in 2025. CaMeL supplies the architecture this build follows. Rather than asking a model to behave, it has a privileged model emit a program, and then a constrained interpreter enforces a data flow policy while it runs. A separate quarantined model gets called only to pull values out of untrusted text. The guarantee is not that the model made a good decision. The guarantee is that the interpreter blocks actions forbidden by the policies it enforces.

That is the move. Put the security property in deterministic enforcement, where the policy allows or blocks the action.

Once you accept that, the design question changes completely. You stop asking how to make the model resist a poisoned document. You start asking what the system can still do after the model has already been fooled.

The three ingredients as a build constraint

Simon Willison’s lethal trifecta is usually used as a diagnosis. An agent is exposed when one component holds private data access, exposure to content you did not author, and a path to communicate outward. Three legs, all present, you have a problem.

It works better as a design constraint than as a diagnosis.

Diagnosis tells you that you are exposed, which you probably suspected. As a constraint it tells you something more useful: the design does not depend on stopping the injection. It aims to keep at least one leg structurally absent from any component that could be fooled. If the enforced boundary holds, a successful injection still cannot produce the prohibited action.

So that is the organizing principle for this build. Every component gets one question asked of it. Which of the three legs does this thing hold, and which one am I removing. Anything that does not answer that question is depth, not control. Depth is fine. Depth is not the thing I am counting on.

What is designed

The architecture is a component set, not a monolith. Twelve pieces in the full reference design. Five of them plus a small number of attacks make up the first version, because I would rather have five components that can be tested directly than twelve that remain diagrams.

The five in version one:

  • Local inference, so model inference does not send sensitive material to a model provider
  • A split control plane, one model that reads untrusted content and one that acts, with a structured handoff between them
  • Egress control, an enumerated list of reachable destinations, denied by default
  • Retrieval with trust tiers, so the system knows the provenance of every piece of text it is holding
  • Audit and telemetry, a record of every action attempted, written where the agent cannot alter it

Here is how the planned components map onto the constraint:

Comparison table showing five AI security components, the threat condition each removes, and the function each actually provides.

Two of those rows are the honest ones. Local inference does not remove a leg. It is a data-residency requirement for this build, but it buys no security against this failure mode. Audit is the same. A log does not stop anything. It provides the record used to determine what happened.

What is standing

The implementation is smaller than the design.

Local inference is running on qwen3.6:35b on hardware I control.

The current build has one document-reading tool. Its functional core is about forty lines long.

The job sounds simple. The agent asks for a document by name, and the tool returns the text. The containment rule is that the requested name has to resolve inside the corpus directory and nowhere else. Both paths get canonicalized before validation, consistent with MITRE CWE-22, because checking the raw string misses a relative path that climbs out. The test cases cover a climb into the sibling directory that holds the answer key, the same climb at depth, an absolute path that discards the corpus root entirely, and a name that is simply missing. The last one distinguishes a typo from an attempted traversal.

Two decisions matter.

In this implementation, the tool returns a refusal string instead of raising an exception. That keeps the agent loop alive after a refused call. The refusal message returns to the model’s context, which means the text of the refusal becomes part of the surface an attacker can see. Error strings are content.

For this test harness, no legitimate task asks for a path outside the corpus root. A successful read is routine. A refused traversal is therefore a high-signal event, so refusals get logged with the full attempted call rather than a summary.

At this design checkpoint, I had nine paired documents in the corpus. Egress control was designed and not built.

Mapped back onto the constraint, the tool bounds the first leg. It puts a hard edge on what the component can reach, and a bounded data leg beats an unbounded one. It does not remove a leg. Nothing implemented at this checkpoint removes a leg. Local inference provides data residency, logging provides instrumentation, and the two controls intended to sever a leg are still design work.

What version one has to survive

The proposed test uses a corpus of synthetic work documents with realistic document markings.

Each document exists in two versions, clean and poisoned. Same content, same structure, same markings. The only difference is an embedded instruction addressed to whatever software processes the file, telling it to retrieve something out of scope and send it somewhere off the list. Identical pairs mean the comparison has one variable in it.

The plan is to build the vulnerable configuration first. Run the attacks against it. Show the leak, with the action record, in full. Only then build the defended configuration and run the identical attacks, unchanged, with no tuning in between.

These are predictions, not results:

The vulnerable configuration should leak on most variants, and if it does not, my attacks are too weak and the corpus goes back for another pass.

The defended configuration is the interesting one. My expectation is that it does not fail on injection. It fails on plumbing. An allowlist entry that is broader than it needed to be. A field that got passed through as prose because a tool signature wanted a string. An action that executed correctly and never made it into the audit record.

Where it goes from there depends on which of those shows up. If the failures are allowlist scope, egress control gets tightened and the interesting work moves to how you define a destination policy that is narrow enough to matter and broad enough to use. If the failures are at the handoff between the two model roles, that boundary gets hardened and the tool signatures get rewritten to refuse free text. If the failures are in the record, then the instrumentation needs work.

The measurement, throughout, is whether a prohibited action occurred. Not whether the output looked risky. An action either happened or it did not.

Return to writing archive