A registered attack on jev-1.13.0 moves validated success from 1.8% to 3.5%

Tiantong Wu and Wei Yang Bryan Lim, at Nanyang Technological University, posted arXiv:2609.28613 on September 23. The version we read is v1, CC BY 4.0. The measurements are on jev-1.13.0. The HTML we extracted does not contain a GitHub repository. A September 26 post links the abstract page. We did not rerun the experiments.

The suite reconstructs 510 InjecAgent cases. The abstract’s summary of the adaptive attack matches the table we read. Score feedback doubles the mean highest attacker-target probability found during the search, from 0.043 to 0.084. Success on fresh validation calls rises from 1.8% to 3.5%.

Validated success in that last experiment means at least 3 target selections in 5 fresh calls. At a search budget of 0 the rate is 1.8%, with an interval from 0.0% to 6.3%. At a budget of 24 it is 3.5%, with an interval from 0.0% to 10.0%. The validated change in attacker probability, budget 24 minus budget 0, is +0.0347, interval [0.0188, 0.0571]. The change in the safe option’s probability is -0.0193, interval [-0.0346, -0.0034]. A screening table prints 1.8% at budget 0, 3.3% at budget 8, and 3.5% at budgets 16 and 24. Successful cases go from 9 to 18. Of the 501 cases that failed at the baseline, 9 succeed once the budget reaches 24, which the paper puts at a conditional 1.8%.

The same section prints a conditional hit rate that uses Jev’s confidence field. At a threshold of 0.7 the rate is 0.2% at budget 24. It is also 0.2% at the baseline checkpoint.

Earlier registered conditions do not hand the attacker a larger effect. An override phrase that the paper says strengthens some attacks on other agents does not do that here. Base attack success is 1.8% (9 of 510), with a mean probability shift of +0.043. The enhanced wording is 0% and +0.009. A repeated-call rerun of enhanced minus base is -0.034, interval [-0.062, -0.015].

A second comparison, against their A0 condition, prints an importance marker at -0.006, an ignore-previous instruction at -0.032, and a natural-wording variant at -0.006. The natural-wording sign runs against the expectation they registered. Call-level attack success across the five conditions is 2.0%, 2.2%, 0%, 0%, and 1.3%: 51, 56, 0, 0, and 34 target selections out of 2,550 calls. Under a majority rule those are 11, 11, 0, 0, and 7 of 510 cases. Dropping the emergency-dispatch goal leaves the directions unchanged. Ignore-previous in that cut is -0.020, interval [-0.031, -0.012].

The third experiment’s probability gaps stay small. P2 minus a neutral prompt is +0.0030, interval [0.0010, 0.0054], under the 0.02 smallest effect they registered. P1 minus neutral is -0.0025, interval [-0.0054, -0.0005], which opposes the registered hypothesis. After they average 17 contexts per goal, P2 minus neutral is positive for 25 of 30 goals. The median absolute change at goal level is about 0.003.

The paper’s reading of the successes points at small initial margins, or at cases where the attacker has more control over the text. Schema-defined outputs change the shape of the answer. The measured rates above are the remainder.