Published
Updated
jgrep's test bench prints mean recall 0.926 across 60 commits, at $0.1112
This topic was created 31 days ago, and the information it contains may have evolved or changed since then.
kyu1204/jgrep, MIT, npm name jevgrep, command jgrep, is a different project from dzhng/jevgrep. A September 28 re-run of 60 commits prints mean recall 0.926, a selection ratio of 0.119, and a total cost of $0.1112. The post's 12%, 90%+, and $0.11 are those three figures rounded. The model field sent is jev-latest. We did not run it.
kyu1204/jgrep is an MIT package whose npm name is jevgrep and whose command is jgrep. The README says the repository is separate from dzhng/jevgrep, whose command is jg. MINT replied on September 30, in a Mario Nawfal thread, that on 60 commits from hono, zod, fastify, flask, and requests the tool picked 12% of test files and caught 90%+ of the tests each commit changed, at $0.11 total. The reply has no photo. The bench file is the tighter source.
That file is dated 2026-09-23 and was re-run on 2026-09-28 with jgrep 0.5.0 and a package-root rule, on the same 60 commits, 12 per repository. The model field sent is jev-latest. The CLI does not echo the provider’s response model id. Across all 60, mean recall is 0.926, code-only recall is 0.449, the selection ratio is 0.119, the total cost is $0.1112, and the mean time is 0.59 seconds. The post’s 12%, 90%+, and $0.11 are those selection, recall, and cost cells rounded.
Per repository, in the order recall, code-only recall, selection, cost, mean seconds: hono 1.000, 0.917, 0.104, $0.0294, 0.61 s. zod 0.882, 0.269, 0.190, $0.0368, 0.66 s. fastify 0.958, 0.208, 0.123, $0.0341, 0.67 s. flask 0.875, 0.350, 0.110, $0.0074, 0.52 s. requests 0.917, 0.500, 0.067, $0.0035, 0.51 s.
Ground truth is the test files the commit modified that still exist. Scoring reverts the tree to the parent first, so the commit’s own test edits cannot select themselves. A file counts when its probability is at least 0.5. The README says an extra selected file is not a false positive. The package-root rule makes no Jev call when the package entry file changes. It moved mean recall from 0.919 to 0.926 and code-only recall from 0.432 to 0.449, and cost from $0.1115 to $0.1112. zod’s mean moved from 0.899 to 0.882 with no package hit. The README attributes that move to Jev variance between runs. A wider re-export variant reached recall 0.946 and selection 0.208, and the page says that variant was rejected.
The misses with recall under 1 are named. flask 7203fea missed tests/test_basic.py. flask e82db2c is also listed. fastify af079bd is a stem bug on fastify.tst.ts. requests 27e0981 is listed. zod lists cc4cd4e, d6bc1e3, ad32d75, and dd9c36f. No commit in this run needed the retry-and-skip path.
A smaller note in the main README, dated 2026-09-21, covers a 142-file TypeScript suite, 3 commits, and 12 source files. The full suite is 142 files, 1,420 cases, and 18.9 seconds. jgrep --tests HEAD~3 is 47 files, of which 12 are by name, 27 by import, and 8 by Jev, then 536 cases and 12.6 seconds. The selection itself is 7 requests, 56k tokens, $0.0024, and 1.0 second.
The tool packs 16 chunks and 16 Noul questions. The price line is $0.042 per million input tokens, output free, which is the published Jev list price. The default search threshold is 0.7. JGREP_ENDPOINT examples include http://127.0.0.1:11434/v1/systemone with model nimble. That line is a configuration example. Later posts from the same account say a Nimble comparison will be posted once it has been tested, and they do not print one. We did not run the bench. The GitHub account is kyu1204. The September 30 reply is MINT’s, and the pages we read do not say MINT wrote the repository.
This site's reading
Editorial notes evaluating claims against primary sources, contextualizing findings alongside related implementations, and defining technical terms.
Verify
The X post is a reply by @MINTr5dw, September 30, 2026, 07:34 UTC, quoting a Mario Nawfal thread. It has no attached photo. The reply says that on 60 commits from hono, zod, fastify, flask, and requests the tool picked 12% of test files and caught 90%+ of the tests each commit changed, at $0.11 total. The bench README is tighter. Those three figures round selection ratio 0.119, mean recall 0.926, and total cost $0.1112.
The bench file is dated 2026-09-23 and was re-run 2026-09-28 with jgrep 0.5.0 plus a package-root rule, on the same 60 commits, 12 per repo. The model field sent is jev-latest. The CLI does not echo the provider's response model id. The README says this repository is not dzhng/jevgrep. That project's command is jg. This package's command is jgrep, and the npm name is jevgrep. The license is MIT.
Per repo, recall, code-only recall, selection ratio, cost, mean seconds. hono: 1.000, 0.917, 0.104, $0.0294, 0.61 s. zod: 0.882, 0.269, 0.190, $0.0368, 0.66 s. fastify: 0.958, 0.208, 0.123, $0.0341, 0.67 s. flask: 0.875, 0.350, 0.110, $0.0074, 0.52 s. requests: 0.917, 0.500, 0.067, $0.0035, 0.51 s. All 60: recall 0.926, code-only 0.449, selection 0.119, $0.1112, mean 0.59 s.
Ground truth is the test files the commit modified that still exist. The tree is reverted to the parent before scoring, so the commit's own test edits cannot select themselves. The threshold is probability at least 0.5. The README says an "extra" file is not counted as a false positive. The package-root rule skips the Jev call when the package entry file changes. It moved mean recall from 0.919 to 0.926 and code-only from 0.432 to 0.449. Cost moved from $0.1115 to $0.1112. zod's mean moved from 0.899 to 0.882 with no package hit, which the README attributes to Jev variance between runs. A wider re-export variant reached recall 0.946 at selection 0.208 and was rejected.
Misses with recall under 1 are listed: flask 7203fea missed tests/test_basic.py; flask e82db2c; fastify af079bd, a stem bug on fastify.tst.ts; requests 27e0981; zod cc4cd4e, d6bc1e3, ad32d75, and dd9c36f. No commit in this run needed the retry-and-skip path.
A smaller note in the main README, dated 2026-09-21, is a 142-file TypeScript suite, 3 commits, 12 source files. The full suite is 142 files, 1,420 cases, 18.9 seconds. `jgrep --tests HEAD~3` is 47 files (12 by name, 27 by import, 8 by Jev), 536 cases, 12.6 seconds. The selection itself is 7 requests, 56k tokens, $0.0024, 1.0 second.
The tool packs 16 chunks and 16 Noul questions. The price line is $0.042 per million input tokens, output free. The default search threshold is 0.7. The endpoint example includes an Ollama URL, http://127.0.0.1:11434/v1/systemone, model nimble. That line is a configuration example. Two later posts from the same account say numbers for Nimble against Jev will be posted once tested. Those posts do not print a comparison. We did not run the bench.
Compare
dzhng/jevgrep, command jg, is the file-and-excerpt tool already on this desk, with 8 of 10 SWE-bench solves and a 25.8% combined-bill note. The npm name jevgrep on this package does not make the two repositories one project. The bench here is test-file recall on 60 commits.
The $0.042 per million input tokens is the published Jev list price this desk already uses. The bench's $0.1112 is the author's total for those 60 commits, not a new price card.
Terms
- 0.926
- Mean recall on the September 28 re-run of 60 commits, jgrep 0.5.0 plus the package-root rule. The post's 90%+ rounds this figure. Code-only recall on the same run is 0.449.
- 0.119
- Selection ratio on that 60-commit run, the share of test files picked. The post's 12% rounds this figure. A wider re-export variant reached recall 0.946 at selection 0.208 and was rejected.
- $0.1112
- Total cost printed for the 60-commit run. The post's $0.11 rounds this figure. The package-root rule moved the earlier total from $0.1115.