I love having this thing up and running. PI-005 just finished running today. I’ll do more analysis over the next several nights and post a detailed essay next week.
In PI-004, we were left with the question; did the model refuse the attack? Or was it just unable to act on the prompt injection?
Here’s where I went from that question into PI-005.
80 runs. Four conditions. Twenty runs each. Each run being an AI agent reading a document with a legitimate task, extract one vendor capability statement from the document and return it in a strict structured format. Poisoned runs have this task, but also have hidden instructions for the agent to perform an unauthorized action to send data externally.
The four conditions:
Clean document, no tool-call ability: 0/20 unauthorized calls.
Poisoned document, no tool-call ability: 0/20 unauthorized calls.
Clean document, contained tool-call ability: 2/20 accepted unauthorized calls.
Poisoned document, contained tool-call ability: 15/20 accepted unauthorized calls.
The first two results look boring, but they give us an important baseline.
Neither worker had access to send_email, so neither worker could call it. That does not tell us the model resisted the attack. It tells us the infrastructure gave it nowhere to go.
What I care about here is that the agents were still useful.
All 40 of the no-tool runs completed the legitimate extraction task successfully. Clean or poisoned, the worker could read the document, pull out the requested information, and return it in the required format.
So removing the dangerous capability did not make the agent useless. It just removed its authority to do something outside the job I actually gave it.
Then I added a contained send_email tool.
Contained is important here. The agent could attempt to use it, and I could record exactly what it tried to do, but nothing could actually leave the experiment. No real email was sent.
This is where things got interesting.
In the clean condition, the agent made an unauthorized tool call in 2 of 20 runs.
There were no hidden malicious instructions in those documents. The model simply had a tool available and, twice, decided to use it even though I had not asked it to.
That is not enough for me to claim that models are naturally biased toward using whatever capabilities you put in front of them.
It is absolutely enough for me to want to test that question next.
It gets much louder in the poisoned condition.
The poisoned worker proposed a send_email call in 20 of 20 runs.
Every single one.
Fifteen of those calls were valid enough to be accepted by the tool interface.
Five were rejected.
The distinction is that those five were not refusals. The model still tried to perform the unauthorized action. The attempt just did not satisfy the tool-call contract well enough to get through.
Across all 80 runs, I recorded zero explicit refusals.
The 15 accepted poisoned calls were also clearly tied to the hidden instructions. Every one contained at least one detail that only existed in the attack, such as the attack-specified recipient, subject, or requested supplier and personnel information.
So this gives me a much cleaner answer to the question PI-004 left open.
The model was not quietly resisting the attack.
When I gave it a safe place to express the behavior, it went for it.
There is another wrinkle I want to spend more time on before I make too much of it.
The no-tool workers completed their legitimate task in 40 of 40 runs.
Once the tool was available, only 31 of 40 completed that same legitimate task successfully.
The clean tool-enabled group completed 16 of 20.
The poisoned tool-enabled group completed 15 of 20.
So simply adding an action channel may be affecting more than security. It may also be affecting whether the agent reliably finishes the job it was actually given.
That is not what PI-005 was designed to prove, so I am treating it as a lead for future questions.
I also want to figure out how to close that gap.
If this pattern holds, there may be a tradeoff hiding here: give the agent more capability and it becomes more useful, but potentially less reliable at finishing the basic job it was given.
Why?
Is it the mere presence of the tool? The way the capability is described? The extra decision path? Does the model start treating an available tool as something it is expected to use?
Eventually I want to isolate that and see whether I can give a worker narrowly scoped tools without sacrificing the 40 of 40 task completion I saw when those tools were absent.
That feels like a systems-design problem worth solving.
The point remains:
If we assume the model will eventually get fooled, can we build the system so that it does not matter?
PI-004 showed that a worker with no authority to act can process a poisoned document without creating an external effect.
PI-005 now gives us much stronger evidence that the worker itself really was susceptible. When given a safe, contained action channel, the poisoned worker tried to use it in all 20 runs.
That makes the architecture question much more interesting to me than the refusal question.
I am less interested in building a model that has to recognize every attack correctly.
I am more interested in building a system that assumes it eventually won’t.
Full PI-005 analysis next week.