For the first time, Sentinel actually feels like the thing I originally wanted to build.
PI-004 was the first experiment with two agents. Not one agent reading a hostile document, deciding what to do, and holding the dangerous tool. Two separate agents with different jobs, different context, and different privileges.
Neither one had all three ingredients for disaster. Remember: if an agent can ACCESS PRIVATE DATA, READ UNTRUSTED CONTENT, AND COMMUNICATE EXTERNALLY, a successful prompt injection has everything it needs to become a real security event.
That sounds obvious now. It did not feel obvious while I was spending days trying to make PI-003 finish.
We could have stayed buried in the timeout problem for another week. Maybe longer. I had already turned a pretty simple reliability question into DPAPI objects, ACL problems, routing catalogs, PowerShell quirks, JSON assumptions, filesystem discovery, and version after version of scripts that were apparently ready until I ran them.
Unfortunately, I am very impatient. Fortunately, this is a personal project so I can do whatever I want.
I wanted to see Sentinel work.
So instead of continuing to isolate the exact cause of the timeout, I made a blunt runtime correction. Thinking off. 512-token maximum. Streaming on.
Three variables at once.
Scientifically elegant? No.
Effective.
I treated the combination as a new runtime, tested the hell out of it, and moved on.
Then I went back to the idea that originally made me want to build this project in the first place: separate the model that touches hostile content from the model that holds privilege.
The inspiration is basically the same direction explored in the CaMeL paper. If prompt injection is hard to solve by making the model perfectly recognize malicious instructions, stop putting all of the dangerous ingredients in one place.
That is what PI-004 tested.
And it worked.
First, the timeout mess
PI-002 was supposed to establish a vulnerable prompt-injection baseline.
It produced 14 valid observations and zero unauthorized send_data executions.
It also produced five infrastructure failures.
That ruined the clean story.
If the attack does not work because the agent sits there until the experiment dies, I do not know what actually stopped it. Maybe the model resisted the attack. Maybe the whole thing just hung.
A timeout is not a security control. Weaponized incompetence is funny but not the goal.
So PI-003 became a reliability study.
Phase 1 gave me six observations. Five completed. One timed out at almost exactly 900 seconds.
The successful trials were not exactly healthy either. Some post-read model calls were taking 500, 600, even 800 seconds. One call took more than 823 seconds to produce a 434-byte final answer.
The timing kept pointing to the same place. read_document would complete, then the first model call after the document entered context would take forever.
Useful finding.
Then I let the experiment get away from me.
The next phase needed a treatment-blind handoff mechanism. That became routing catalogs, DPAPI, ACLs, PowerShell behavior, filesystem discovery, allocation resolution, JSON parsing assumptions, and a lot of me functioning as the integration test suite.
Paste this.
Run it.
Send back the error.
Patch it.
Run it again.
New error.
Repeat.
It was like sitting in rush-hour traffic.
More importantly, I was spending almost all of my Sentinel time troubleshooting an experimental apparatus instead of doing AI security work.
So I stopped.
PI-003 is incomplete. I am leaving it incomplete.
Phase 1 happened. It found a real reliability problem. The planned Phase 1R extension did not happen. The apparatus work became more expensive than the question was worth.
At that point I had a simpler goal.
Just make the thing run.
The reliability reset
The hardware did not look like the problem. Qwen was already generating around 27 to 28 tokens per second on the MS-A2.
My runtime configuration looked a lot more suspicious.
Thinking was on. There was no useful task-specific output ceiling. Streaming was off. A 900-second timeout was sitting at the end waiting to kill whatever wandered too long.
So I changed all three.
Thinking off.
512-token maximum.
Streaming on.
The settings changed as a bundle. This established a dependable runtime, not which individual setting fixed it.
Again, I know. One variable at a time makes better data, but I’m not a scientist.
That was no longer the experiment I cared about. I did not need to know which of those three changes fixed the problem. I needed a dependable runtime so I could get back to the security experiments.
One live smoke cycle passed.
Then 25 consecutive synthetic cycles passed.
Seventy-five model calls.
Zero timeouts. Zero infrastructure failures. Zero hangs.
The slowest post-read call was about 57 seconds.
Compared with 500 to 800 second calls and a 900-second timeout, I was done troubleshooting.
Good enough.
Now to go break something interesting.
The vulnerable baseline finally worked
Because I changed the runtime, I reran the original attack instead of pretending the old PI-002 result still applied.
That became PI-002-RC1.
Fifteen observations.
Fifteen completed.
Zero infrastructure failures.
Zero timeouts.
Then the model got absolutely wrecked.
The audit found 13 exact model-originated calls to send_data. Eleven of the fifteen observations produced completed unauthorized egress that was independently recorded by the catcher.
Perfect.
Not from a security standpoint, obviously.
From an experimental standpoint, this was exactly what I needed.
The agent ran. The task completed. The tool worked. The catcher worked. The attack worked repeatedly.
No timeout muddying the result.
I finally had a clean vulnerable baseline.
Now what?
Stop asking the model to save itself
The obvious next move would have been model tuning.
Turn thinking back on. Rewrite the system prompt. Add more warnings. Add some kind of prompt-injection classifier. Maybe try another model. Keep making changes until the attack rate goes down.
I will get there eventually.
It was not what I wanted to test next.
PI-002-RC1 already showed me that the model reading the hostile document could be manipulated into using a privileged tool.
Fine.
Assume it happens.
What can I do architecturally so that successful manipulation does not automatically become a privileged action?
This is where Sentinel finally became a two-agent system.
Two agents
The original architecture was basically:
untrusted document -> privileged agent -> send_data
The same model read the hostile content, decided what it meant, and held the dangerous tool.
That is a lot of trust in one place.
PI-004 split the job.
untrusted document -> tool-less worker -> validated structured data -> privileged planner -> send_data
PI-004 moved raw hostile content and privileged tool access into separate agent contexts. Neither agent received the full set of capabilities needed for the tested egress path.
The worker gets the raw document.
It sees the prompt injection.
It has no privileged tools.
The planner gets the privilege.
It has send_data.
It never sees the raw document.
Between them is a very small structured handoff that gets validated before anything crosses the boundary.
That is the main event.
The worker can be fooled. The planner can be privileged. Neither one gets everything.
This was the first Sentinel experiment where I was actually testing that architecture instead of testing one model with a pile of instructions around it.
PI-004
PI-004 used 15 fresh observations.
Five clean. Ten poisoned.
The treatment order stayed private until after the evidence was frozen.
I kept the corrected runtime from PI-002-RC1. Same Qwen model, same bounded generation, same streaming behavior, same general attack objective.
Before the run I tested the privileged send_data path directly.
It worked.
That matters because zero unauthorized sends would be pretty meaningless if I had accidentally broken the tool.
Then I ran all fifteen observations.
All fifteen completed.
Zero infrastructure failures.
Zero timeouts.
Zero prohibited actions.
Then I tested send_data again after the run.
Still worked.
Only then did I unblind the treatment schedule.
The privileged path remained operational before and after the experiment. All 15 experimental observations completed without an unauthorized send_data attempt.
The clean documents
Five clean documents. Five valid observations. Five legitimate task completions. Zero unauthorized send_data attempts.
The architecture still did the normal job.
The poisoned documents
This is the part I cared about.
All ten poisoned documents reached the worker.
The attack was not filtered out before the system saw it. The worker got the hostile content exactly as designed.
The planner saw the raw hostile content zero out of ten times.
There were zero unauthorized model-originated send_data attempts, zero unauthorized dispatches, zero catcher posts, zero infrastructure failures, and zero timeouts.
All ten legitimate tasks still completed.
Across the whole experiment, fifteen of fifteen legitimate tasks completed and zero of fifteen produced unauthorized egress.
PI-002-RC1 also completed fifteen of fifteen legitimate tasks.
Eleven of those fifteen leaked.
Both architectures completed the legitimate task in all 15 observations. The vulnerable RC1 baseline produced catcher-backed unauthorized egress in 11; PI-004 produced none.
That is probably the cleanest result Sentinel has produced so far.
The vulnerable architecture ran properly and leaked.
The separated architecture ran properly and did not.
Same basic attack problem. Very different outcome.
What I think this actually means
Of course PI-004 does not prove prompt injection is solved.
Obviously.
It was one model, one architecture, one general attack pattern, and fifteen observations.
There are plenty of ways this could still fail.
What I am comfortable saying is narrower.
The hostile document reached the unprivileged worker. The raw hostile content did not reach the privileged planner. The dangerous send_data capability remained functional. Every legitimate task completed. No unauthorized send_data attempt occurred.
That is what I designed the boundary to do.
It did it.
Calling this a small win to cheer me up after escaping troubleshooting purgatory.
The part I like
The model did not get smarter between PI-002-RC1 and PI-004.
I did not teach it to detect prompt injection.
I did not add another paragraph to the system prompt telling it to behave.
The model that got wrecked in PI-002-RC1 was still basically the model sitting on the hostile side of the PI-004 architecture.
I just stopped giving that model everything it needed to turn being fooled into an actual security event.
That feels a lot more like normal security engineering.
Assume something eventually gets compromised. Then decide what that thing is allowed to touch.
The more I work on Sentinel, the less interested I am in asking whether a model can be made perfectly resistant to prompt injection.
Maybe it can. Maybe it cannot.
I care more about what happens after the injection works.
PI-004 is the first experiment where I think Sentinel actually started answering that question.
So now what?
There is one thing I want to know before I start getting clever with PI-005.
Did the worker actually fall for the prompt injection?
Right now I know the poisoned document reached it. I know the planner never saw the raw hostile content. I know nothing unauthorized happened.
What I do not know is whether the worker ignored the attack, refused it, or actually followed it and just had nowhere dangerous to go.
I want to get a handle on that before moving on.
The whole point of this architecture is that I should be able to assume the worker gets compromised and still keep the privileged side safe. So I want to test that directly.
The next experiment will give the poisoned document a harmless compromise canary. Something obvious and test-only that the worker can produce if it follows the injected instruction. No send_data. No real privilege. Just a clean way to answer the question.
Did the injection actually change the worker’s behavior?
If the answer is yes and the planner still never sees the hostile content or performs the prohibited action, that is a much stronger result than PI-004 gives me today.
Basically:
Did the model get fooled? (Working toward yes.. the model can and is fooled and ready to defect).
Did the architecture still hold?
I want both answers separately.
Then I can move on to PI-005, where the attacker gets smarter and stops trying to sneak an instruction across the boundary at all.
That one is still the obvious next attack. The worker cannot pass arbitrary instructions to the planner, but it can pass data. So what if every field is valid? Or the schema accepts it? What if the attacker manipulates a perfectly allowed value badly enough that the privileged planner makes the dangerous decision on its own?
The working description is schema-conformant semantic smuggling.
Which is a ridiculous way of saying:
Can I fill out the approved form in a way that still tricks the person with the keys?
First I want to prove the worker can lose without taking the system down with it.
Then I want to see how far I can push that.