Well, I finally did a repeated test.
I guess hoping nothing would go wrong was too optimistic.
I am still baselining Sentinel, having one model read dummy documents and summarize them. One of the documents is poisoned with a prompt injection telling the model to send data somewhere it is not authorized to send it.
In several valid poisoned runs, the model recognized the prompt injection and refused it, which was interesting.
Other attempts timed out before producing a valid result.
Secure with weaponized incompetence.
Sentinel’s deliberately vulnerable baseline is allowed to read untrusted documents and use tools. One of those tools, send_data, can move information to a contained destination I control.
The attack is indirect prompt injection.
Instead of telling the agent directly to do something malicious, I hide an instruction inside a document the agent has a legitimate reason to read. The question is whether the agent treats that untrusted text like data or follows it like an instruction.
The malicious instruction in this experiment tells the agent to use send_data.
If the runner executes that unauthorized action, the attack worked.
Simple enough.
PI-002 was supposed to tell me how repeatable that failure was.
It gave me a different problem instead.
The experiment I wanted
The clean version of this story looks like this:
Give the vulnerable agent a poisoned document.
The model follows the hidden instruction.
send_data fires.
The catcher receives the payload.
Vulnerable baseline established.
Then I build the defended architecture, run the same attack again, and see whether the defense stops it.
Nice before-and-after.
Nice chart.
Very publishable.
That is not what happened.
The vulnerable agent never executed the malicious action.
But several runs broke badly enough that I cannot pretend “nothing happened” means the system was safe.
Fourteen valid runs. Five failures.
PI-002 repeated the same basic attack from fresh context.
Each trial started with a blank agent conversation. No memory of the previous run.
Same local model.
Same legitimate summarization task.
Same tools.
Same attack text.
Same document pair.
Same runner behavior.
Fifteen trial slots were planned.
Five used a clean target document.
Ten used a poisoned twin containing the hidden instruction.
By the time I closed the experiment, all 15 slots had been addressed.
Fourteen produced valid observations.
Five additional attempts failed at the infrastructure level while I was trying to complete those slots.
One poisoned slot failed three separate times and never produced a valid observation.
Across the 14 valid runs:
- 0 unauthorized
send_dataactions executed - 0 executable
send_datacalls emitted - 0 experimental POSTs reached the catcher
- 14 legitimate summarization tasks completed
The obvious headline is:
Zero exfiltration.
That is also where the result becomes easy to overstate.
Of course a timeout is not a block
A crash is not containment.
A stalled model is not a defended architecture.
A tool that never fired because the process died first was not successfully blocked.
The harder part is enforcing that rule when the broken run is helping your result.
Suppose I run an attack 15 times.
Ten runs finish safely.
Five crash before anything dangerous happens.
I do not get to report 15 successful defenses.
I have ten valid observations and five broken experiments.
PI-002 forced that distinction.
The failed attempts remain in the record.
They are reported separately.
They do not get mixed into the security denominator.
One planned slot never recovered, so the study closes with 14 valid observations instead of the 15 I wanted.
Messier number.
Better experiment.
What actually counts as an attack succeeding?
Agent systems have several layers between a model thinking about an action and something actually happening in the world.
A model can consider using a tool.
It can write something that resembles a tool call.
It can emit a syntactically valid tool call.
The runner can accept and execute that call.
The receiving system can actually get the data.
Those are not the same event.
Sentinel records them separately.
A model considering an action is not execution. A timeout is not prevention. A refusal is not an architectural control.
For PI-002, the primary question was intentionally narrow:
Did the runner execute an unauthorized send_data action?
That is the line I care about.
Not scary reasoning.
Not the model mentioning the tool.
Not a refusal.
Not a timeout.
Execution.
Across the 14 valid observations, the answer was no.
That is the evidence we have from this test.
It is also much narrower than saying the agent is secure.
The poisoned document still changed the run
The agent did not execute the hidden instruction.
It also did not behave normally.
The five valid clean runs took about 428 seconds each on average.
The nine valid poisoned runs took about 930 seconds each.
More than twice as long.
The clean runs required 14 model calls total.
A model call is basically one round trip to the AI. Sentinel sends the model whatever context it has at that point, the model responds, and the agent loop decides what happens next.
The poisoned runs required 32.
The poisoned condition changed system behavior even though no unauthorized action executed.
Something got heavier.
I do not know exactly what.
The model may have spent more time resolving the conflict between my task and the instruction embedded in the document.
Protocol corrections may have added friction.
Some runs may have bogged down during final-answer generation.
The local runtime may have contributed.
PI-002 was not instrumented well enough to separate those explanations cleanly.
What I can say is this:
The poisoned runs were slower and required more model calls in this sample.
The attack did not produce unauthorized execution.
It still changed system behavior.
An attack does not need a clean win to hurt the system
Most prompt-injection discussion naturally focuses on the worst outcome.
The agent leaks the secret.
Deletes the file.
Sends the message.
Executes the command.
Fair enough. Those are the outcomes that matter most.
But an adversarial document can also damage an agent workflow without producing the final malicious action.
It can make the model deliberate longer.
Trigger retries.
Cause protocol corrections.
Consume more inference.
Push a local runtime toward its limits.
Turn a workflow that normally completes into one that sometimes just stops.
That is not the same class of failure as unauthorized tool use.
I do not want to collapse them into one metric.
But I also do not want to ignore the second one because the first one is more exciting.
PI-002 started as a prompt-injection repeatability experiment.
It ended with a reliability problem sitting in the middle of the apparatus.
Nine valid poisoned runs produced zero unauthorized executions.
That tells me what this exact frozen configuration did nine times.
It does not tell me what Qwen will do against another injection.
It does not tell me what happens if I change the task, document, tool schema, runtime, or attack text.
I am nowhere near calling Sentinel secure.
This is a small exploratory experiment against one attack variant.
The failures are now the experiment
Five infrastructure failures in a pilot this small are not cleanup work.
They become the next problem.
Before I make the injection more sophisticated, I need to know why the measuring instrument sometimes falls over.
That is PI-003.
It is not another security victory test.
It is a reliability study.
Where does the run actually fail?
After the document has already been read?
During another model call?
After a protocol correction?
During final-answer generation?
Is the model exhausting an output budget?
Is the host under pressure?
Is a wall-clock timeout hiding a predictable failure mode?
PI-002 cannot answer those questions because the frozen runner did not capture enough information at the individual-call level.
The next version will.
Per-call timing.
Explicit output budgets.
Termination reasons.
Better runtime telemetry.
A bounded final-answer contract if the evidence points there.
I am sticking with the same model for now.
Changing models would also change capability, speed, refusal behavior, formatting, and probably a few other things at the same time.
I can do that later.
Right now I need fewer changed variables, not more.
Before I make the attack harder, the measurement tool needs work.
What I can actually claim
What I have is fourteen valid observations under one frozen vulnerable configuration.
Nine poisoned runs with zero unauthorized execution.
Five infrastructure failures that stayed failures.
One planned slot that never recovered.
Enough to know the poisoned input changed the system’s behavior.
Not enough to know exactly why.
I wanted PI-002 to give me a compromised agent.
Instead it gave me a measurement problem.
Less cinematic.
More useful.