Published
Updated
jevgrep solves the same 8 of 10 SWE-bench tasks, and the bill including Jev is 25.8% lower
This topic was created 31 days ago, and the information it contains may have evolved or changed since then.
David posted dzhng/jevgrep, MIT, a CLI that asks Jev which files matter and returns excerpts. The September 26 post said about 40% lower coding-agent cost. The September 28 file for package 0.4.3, ten tuned Python tasks, keeps 8 of 10 official solves and prints $5.6564 against a saved $7.6221, Jev included. Five of the ten tasks cost more than that saved baseline. We did not rerun it.
David, @dzhng, posted dzhng/jevgrep on September 26 and a v0.4 note on September 28. The LICENSE file is MIT. The CLI is jg. It wants Node.js 22 or newer, on macOS or Linux, and a key for Vercel AI Gateway, TypeSafe, OpenRouter, or OpenCode Zen. A question goes out, Jev judges folders and files, and stdout comes back with paths and line-numbered excerpts. The coding agent still writes the patch. jg skill installs a skill for Claude Code, Codex, and OpenCode. Searches send source to the provider you picked at jg auth. We did not install it.
The September 26 post says the tool cuts coding-agent cost by 40% on SWE-bench. The September 28 post says v0.4 is at intelligence parity with Codex and Claude subagents and about 30% less cost, and that he traded some of the original saving for that parity. The files print two different bills.
The README describes a Sol-only comparison on ten tuned Python SWE-bench tasks. Both arms solve 8 of 10. Full Sol cost falls from $7.62 to $5.44, a 28.6% cut, including failed attempts and excluding Jev. The page rounds that to about 30%. The graphic on the repo is this comparison.
The later file, total-cost-2026-09-28.md, is a fresh run of @dzhng/[email protected]. Same 8 of 10 official solves. Combined cost is $5.6563991: $4.1779912 for Sol and $1.4784079 estimated for Jev, against a saved baseline of $7.6220690. That is 25.8% lower, failed attempts included. Per attempted task the totals are $0.5656 and $0.7622. Spend divided by the eight solves is $0.7070 and $0.9528. Matplotlib matplotlib-26466 and Pylint pylint-4604 are the two unresolved tasks on both arms.
Jev’s dollars are a list-price estimate. The note multiplies usage.input_tokens by $0.042 per million for jev-1.13.0 and treats output as free. The rate is TypeSafe’s model page as retrieved on September 28. Sol’s charges come from retained gateway metadata. Neither column is an invoice. The no-Jev baselines were reused and not run again. The agent may use ordinary tools after the required Jevgrep call. The note says the ten tasks are tuned, not a holdout, and that this Sol trajectory differs from the earlier one, so the 25.8% is a new measurement.
Five of the ten combined totals are higher than the saved baseline: pydata__xarray-3305 ($0.7009 against $0.4937), psf__requests-1142 ($0.3496 against $0.2685), astropy__astropy-13579 ($0.4938 against $0.4286), the unresolved Matplotlib task ($0.5966 against $0.3957), and the unresolved Pylint task ($0.2923 against $0.2836). The total still falls. Sphinx goes from $1.0595 to $0.5392, and pytest from $2.3445 to $0.5289.
This site's reading
Editorial notes evaluating claims against primary sources, contextualizing findings alongside related implementations, and defining technical terms.
Verify
dzhng/jevgrep, MIT, Node.js 22 or newer. Package in the September 28 total-cost file is @dzhng/[email protected]. Ten tuned Python SWE-bench tasks, one fresh first attempt each, openai/gpt-5.6-sol at medium effort, native TypeSafe retrieval, jev-1.13.0.
Official solves 8 of 10 on both arms. Matplotlib and Pylint stay unresolved. Combined cost $5.6563991, of which Sol is $4.1779912 and the Jev estimate is $1.4784079, against a saved no-Jev baseline of $7.6220690.
That is 25.8% lower. Per attempted task, $0.5656 against $0.7622. Per solved task, $0.7070 against $0.9528. The Jev estimate uses $0.042 per million input tokens, output free, from TypeSafe's docs retrieved that day.
It is not an invoice. Saved baselines were not rerun. Five combined totals exceed the saved baseline: xarray, requests, astropy, matplotlib, pylint. The README's Sol-only comparison, Jev excluded, is $7.62 to $5.44, 28.6%, which the page rounds to about 30%.
The September 26 post said 40%. We did not rerun the tasks.
Compare
astra-jev-harness measures Astra input tokens on one two-file fixture, 19,198 to 14,011, and does not grade SWE-bench. jev-gateway's chess note is a different coding harness. The 25.8% figure includes Jev's list-price estimate.
The 40% in the introduction does not match either file we read.
Terms
- 8 of 10
- Official SWE-bench solves for both the jevgrep arm and the saved no-Jev baseline, on ten tuned Python tasks. matplotlib-26466 and pylint-4604 stay unresolved.
- 25.8%
- Total cost including a Jev list-price estimate, $5.6564 against $7.6221. Sol alone in that file is $4.1780. An older Sol-only comparison, Jev excluded, is $5.44 against $7.62, printed as 28.6%.
- $0.042
- Per million input tokens, the rate the September 28 note uses for jev-1.13.0. Output tokens are free in that formula. The note says this is not invoice reconciliation.