UBAG was evaluated by Dixit Algorizmi on two public academic prompt-injection benchmark suites. The suites were developed independently by ETH Zurich and UIUC. The evaluations were not conducted by those institutions and are not a third-party audit.
Independent academic suites 01
LIVE AGENT
AgentDojo
ETH Zurich and Invariant Labs affiliations · banking suite · important_instructions attack · gemini-2.5-flash
Money plus sensitive-action grants32.6% to 0.0%46.5% to 31.2%
Run shape: 16 user tasks by 9 injection tasks per arm. The two baselines differ because they are separate model runs and the model is nondeterministic. The evaluator, policy configuration, and raw traces are retained for reruns. Identical gateway calls reproduce the same UBAG decision; end-to-end task rates can vary with the model trajectory.
Interpretation: The money policy leaves one password-change attack family outside its scope. Requiring a signed grant for sensitive account changes closes that residual in the measured run.
Utility: AgentDojo has no human approval or grant issuance path. Held first-time payees and refused password changes count as failed tasks, so the result measures a strict benchmark boundary rather than a complete production approval workflow.
STATIC REPLAY
InjecAgent
UIUC · Findings of ACL 2024 · 1,054 cases · 17 user tools · 62 attacker tools
What it proves: InjecAgent attacks require at least one tool outside the legitimate task scope. UBAG least-privilege confinement blocks that call deterministically.
Our disclosure: We report this as a replay of the benchmark's labeled tool calls through the shipped authorization engine, not a live-agent run and not an end-to-end task-completion measurement.
Boundary: When a harmful action can reuse an authorized tool, tool scope alone is insufficient. Destination, argument, value, state, and grant controls must carry the decision.
Internal regression suite 02
SUPPLEMENTARY ENGINE COVERAGE
Known failure modes stay executable as tests.
Our authored suite exercises tenant isolation, capability grants, state drift, cumulative plans, fabricated references, argument injection, circuit breakers, and benign controls.
The suite contains 574 total cases and recorded zero execution errors. Seven blind, model-authored corpora were generated from the same declared scenario using GLM 5.2, Gemini 3.5, DeepSeek, Microsoft Copilot, Claude Opus 4.8, GPT-5.6 Sol, and Claude Sonnet 4.6. The models proposed the cases; UBAG evaluated them deterministically. Thirteen authored expectation labels differed from the configured policy semantics and are reported separately rather than counted as attacks or benign controls. This is internal regression evidence, not independent validation.
Evaluation standard 03
01
Mechanism, not detection
The claim is not that UBAG recognizes every malicious instruction. The claim is that policy denies unauthorized execution even when the model is persuaded.
02
Conditions remain visible
Model, suite, attack family, policy point, case count, utility effect, and evaluation type are stated beside each result.
03
Residuals stay public
The AgentDojo money-only residual and the InjecAgent replay limitation remain part of the result, not hidden in fine print.
04
Evidence is reviewable
Run logs, policy configuration, evaluator code, and decision reasons are available during a technical diligence or pilot review.
TECHNICAL DILIGENCEReview the policy, run artifacts, and decision logs.
UBAGby Dixit AlgorizmiRequest technical review