I benchmarked vLLM Semantic Router's new Decision 3.0 models as a prompt-injection gate for my agent, all six sizes, running locally in MLX on an M2 Max.
The 27B model at 4 bits is the first local detector that beats Jev, the cloud model I use. It blocks 400 of 433 attacks (Jev 383) and 140 of 150 instructions planted in ordinary emails (Jev 138), and it wrongly blocks 7 benign items against Jev's 10. It also doesn't flag articles about prompt injection, which tripped earlier models. The price is speed: about 7 seconds per message and 17 GB of memory.
The smaller ones rank attacks almost as well, but they also score some harmless content high, so at a low false-positive rate they let many planted emails through. d3-lite, at 0.85B parameters, ranks as well as Nimble 9B in a fifth of a second.
Updated blog with current benchmarks:
https://juraj.bednar.io/en/blog-en/2026/09/28/a-prompt-injection-gate-for-my-ai-agent-what-worked-what-didnt-and-the-benchmark/#d3
Juraj / Juraj🏴💛🌘
npub1m2…lr8p9
2026-10-12 19:04:00 UTC
Author Public Key
npub1m2mvvpjugwdehtaskrcl7ksvdqnnhnjur9v6g9v266nss504q7mqvlr8p9Published at
2026-10-12 19:04:00 UTCEvent JSON
{
"id": "3bcd25c9ef7d02a76643edb67b5aae9aab911c5a15a94e9240fc141757ab1996",
"pubkey": "dab6c6065c439b9bafb0b0f1ff5a0c68273bce5c1959a4158ad6a70851f507b6",
"created_at": 1791831840,
"kind": 1,
"tags": [],
"content": "I benchmarked vLLM Semantic Router's new Decision 3.0 models as a prompt-injection gate for my agent, all six sizes, running locally in MLX on an M2 Max.\n\nThe 27B model at 4 bits is the first local detector that beats Jev, the cloud model I use. It blocks 400 of 433 attacks (Jev 383) and 140 of 150 instructions planted in ordinary emails (Jev 138), and it wrongly blocks 7 benign items against Jev's 10. It also doesn't flag articles about prompt injection, which tripped earlier models. The price is speed: about 7 seconds per message and 17 GB of memory.\n\nThe smaller ones rank attacks almost as well, but they also score some harmless content high, so at a low false-positive rate they let many planted emails through. d3-lite, at 0.85B parameters, ranks as well as Nimble 9B in a fifth of a second.\n\nUpdated blog with current benchmarks:\nhttps://juraj.bednar.io/en/blog-en/2026/09/28/a-prompt-injection-gate-for-my-ai-agent-what-worked-what-didnt-and-the-benchmark/#d3",
"sig": "f0d5946551e682db636d9c48688e027010abf901d752ce022b6e9a5969e553778e9406a0fb5ec629a79b837908fe39be9047dca7abf15d90d7218abf21a1f381"
}