Technically AcceptableIndependent technical publication
← Writing archive
ESSAY

More Thoughts on Prompt Injection: Moving Security Out of the Model

You Cannot Argue a Model Into Being Safe

By Travis Shea
  • AI Security

Here is the problem in one sentence. A language model reads your instructions and whatever junk it scraped off the internet through the same door, and it cannot tell which one is the boss.

That is the vulnerability. Prompt injection is not a bug someone will patch. It is the nature of the thing. The model takes command and content as one stream, and a good enough attacker buried in that stream will sneak a command through.

Asking nicely does not work

The common fixes all live inside the model. Write a firmer system prompt. Add a line that says ignore malicious instructions. Bolt on a second model to scan the input for bad stuff.

All of it is behavioral. All of it is asking a gullible thing to please not be gullible this time. Under a real attacker it folds. A filter that catches 99 percent of attacks sounds great but the attacker only has to win one time to ruin your life. And you cannot fix it by adding more AI, because the guard model can be fooled the same way the first one was.

So what is being developed to solve this problem?

The move that actually works

Assume the model will be fooled. Create an infrastructure that can withstand a compromised model.

You stop defending the model and start constraining the system around it, so that even a fully hijacked model has nowhere to send the damage. The safety lives outside the model, in systems that are unaffected by prompt injections. That seems to be the current plan in this space.

Two versions are worth knowing. One more secure than the other.

Dual-LLM. The good habit.

Simon Willison sketched this back in 2023. Two models, two privilege levels.

One is the planner. It is trusted, it can use tools, and it never touches the dirty content. The other is the worker. It reads the untrusted stuff, the scraped page, the shady document, but it has no reach into your private data. It cannot do anything.

The trick is what passes between them. The worker never hands the poison back to the planner. It reads the nasty webpage and returns a sealed result, a variable, call it VALUE1. The planner shuffles VALUE1 around and acts on it without ever reading the contents. The poison stays sealed in the box.

Where the good habit cracks

It is not enough, and the CaMeL researchers say so plainly.

The worker can still be tricked. It reads an attacker’s document and hands back the attacker’s email address as the recipient. The planner never saw the poison. It still mails your secrets to the wrong person, because nobody controlled where that sealed value was allowed to go.

Splitting the models isolates them. It does not govern the flow between them.

CaMeL

CaMeL came out of Google DeepMind, last year. It keeps the two models and adds the piece the dual pattern was missing. Enforced data flow, outside the model.

Here is how it runs.

The privileged model sees only your request. It does not read the untrusted content at all. Instead it writes a little program, a plan, describing what should happen.

A second, quarantined model gets called only to pull specific values out of the dirty content. It has no tools and cannot act.

Then the important part. A locked-down interpreter runs the plan. It tags every piece of data with where it came from and what it is allowed to do. It tracks those tags through the whole flow. And before any real action fires, it checks the tags against a policy you set.

The model does not call the tool. The interpreter does, and only if the policy allows it. Say the rule is: an email can only go to an address the user typed, never one that came out of a scraped document. Now the model can be completely fooled and the forbidden email still does not send. The interpreter refuses, and the interpreter does not have feelings you can manipulate.

Think of the model as a cashier. You can hand a cashier a forged note. But the register only opens under fixed rules, and no amount of convincing changes that. Move the safety from the cashier to the register. That is CaMeL.

The part I actually care about

CaMeL has a side effect that lines up with everything I believe.

The planner only ever sees your request. It never touches your private data. Which means the planner can be a big cloud model, and the quarantined worker, the one that reads your actual private files, can run local, on your own hardware.

Lease the sports car but buy the Toyota.

That is not a manifesto. That is a design out of a DeepMind paper. The best current thinking on securing these agents puts the frontier model and your self-hosted private data on the same team on purpose. The privacy is not a compromise bolted on. It falls out of the architecture.

Where it still breaks

Architectural is not the same as bulletproof. It just moves the fight to a different surface.

You still have to write the policies, and you cannot forbid a gap you did not think of. The worker can still return a bad value that the policy happens to allow. Some leak channels are subtle enough to slip a ruleset. And if you make the rules too strict, user fatigue turns approval into a rubber stamp.

So no, this is not solved. But there is a difference between a fight you cannot win, which is arguing with the model, and a fight that is merely hard, which is getting a ruleset right. This beats doing nothing until a perfect system is invented.

What I am doing with it

Right now my own setup runs the crudest version of all of this. A human approval gate. Me, clicking yes before anything commits. I am the interpreter, and I am a slow, tired, distractible one.

CaMeL is the ceiling I am guiding toward. I am pulling these patterns apart one at a time so I can start wiring the real ones into the lab and then attacking them in the range the same way I attack everything else. Human first. Then a script. Then a local model off its leash. Whatever structure survives that is worth trusting. Whatever does not was never security.

You cannot argue a model into being safe. You can only build a room it cannot burn down.

That is the work.

Sources

Return to writing archive