Published
Updated
mizchi's fit notes record 0 illegal moves in 489 MOBA calls, and 6 to 27 wrong pages where a one-line rule was 16/16
This topic was created 31 days ago, and the information it contains may have evolved or changed since then.
mizchi posted a fit table for Jev on September 24. Enumerated moves: 489 MOBA decisions and 37 chess moves, 0 illegal. A one-line monitor rule is 16/16 with 0 wrong pages. Jev on that job is 6 to 27 wrong pages. Naming a code-criteria question still returns no at 0.09 to 0.22. Five agent components are 26 to 103 times faster, and none of those columns is a clear quality win. The linked criteria note, opened September 30, puts named hard bugs at 11/21 against held-out hard bugs at 3/15, 52.4% against 20.0%.
mizchi posted a fit table on September 24, 08:55 UTC, and linked docs/fit.md in mizchi/jev-playground. The page says the numbers are measurements, and that the thirteen write-ups, including a MOBA, live in case-studies.md. A request for LICENSE at the repo root came back 404. We read the fit page.
The table sorts jobs by shape. When the options are the legal moves, or names that already exist, the page marks the shape as a fit: 489 MOBA decisions and 37 chess moves, and zero of the answers was illegal. An ordered judgment, allow then confirm then block, goes from 19/24 to 23/24 if a Choice is followed by a Score. Several decisions on one state pack: a 5v5 covers 4.86 heroes in one request, at 53 ms per hero. Picking from a named list is 90.0% on the name alone and 100% after one line of description. A sweep of 1,000 functions costs $0.017 to $0.055. A hook in front of a command has a median of 371 ms on 391 commands, inside a 2,500 ms budget. The page’s unit price is about $0.000017 per function, and about 200 ms.
Independent yes/no questions are a weaker fit unless a general question sits beside them. On defect classes that were listed, atomic questions are 25/36 and a general question is 18/36. On classes that were not listed, the atomic set is 11/15 and the general question is 15/15. The page’s reading is that a list of named hazards leaves a hole where the list stopped, so the general question has to stay in the call.
Two shapes are marked as the wrong tool. If the answer is a function of a quantity you can compute, a one-line rule is 16/16 with zero wrong pages, and Jev on that monitor triage is 6 to 27 wrong pages. If the decision depends on an API behavior the snippet does not state, naming the behavior in the question still returns no at 0.09 to 0.22. The examples on the page are Array.sort with no comparator, splice through the end of the array, a regex without the g flag, and a returned promise that has already left the try. The same page says the easy side does not care whether you name the criterion (15/15 either way), and that naming changes the hard side of that code-criteria note by about 2.6 times.
On September 30, 2026 we opened docs/22-code-criteria.md. Section 11.1, three repeats at the fitted thresholds, puts easy bugs at 15/15 on the full rubric whether the criterion is named or held out. Hard bugs on that rubric are 11/21 when named, 52.4%, and 3/15 when held out, 20.0%. The page calls the gap about 32 points, about 2.6 times. It says the earlier 15/15 reading was easy bugs. The addendum splits held-out bugs into 5 easy and 5 hard.
Section 3, also three repeats, asks the matching criterion on the six misses. Two came back: Array.sort with no comparator at 0.32, and setUTCMonth overflow at 0.27. Four stayed no: a missed await at 0.22, splice with too few arguments at 0.12, a regex without the g flag at 0.09, and an LRU cache that does not update order at 0.11. Section 11.2 drops one criterion at a time. Seven of the eight lose nothing on their own class. api_default falls from 9/12 to 4/12. The addendum’s corpus is 78 functions in 15 files, with 22 bugs.
Spatial questions on one board split by how the board is written down: “is it the row below” 96%, “is it the column to the right” 74%, and “is an enemy adjacent” 49% when an enemy is adjacent. Decisions that sit on a threshold flip when the wording moves a little, and the page sends that band to a person. Inside five agent components, Jev is 26 to 103 times faster than the models it was compared with, and the page says none of those columns is a clear win on quality. The closing line of the action section is about moves, not tokens: one move is under a few cents, and one move changes a real app. A mechanism that cut tokens by 67% still had to be paid back. A mechanism that removed five moves did not. We did not re-run the cases.
This site's reading
Editorial notes evaluating claims against primary sources, contextualizing findings alongside related implementations, and defining technical terms.
Verify
The September 24 post, 08:55 UTC, links github.com/mizchi/jev-playground/blob/main/docs/fit.md. The table on that page: enumerated legal moves, MOBA 489 decisions and chess 37 moves, 0 illegal. Ordered judgment, choice then score, 19/24 to 23/24.
One state, many decisions: 4.86 heroes per request in a 5v5, 53 ms per hero. Names alone 90.0%, plus one line of description 100%. Enumerated defect classes: atomic questions 25/36 against a general question 18/36; classes that were not listed, 11/15 against 15/15.
A full pass of 1,000 functions, $0.017 to $0.055. A hook, median 371 ms on 391 commands, budget 2,500 ms. A one-line monitor rule, 16/16 and 0 wrong pages; Jev, 6 to 27 wrong pages.
A named question about an API behavior the snippet does not show, no at 0.09 to 0.22. Spatial, same board: "row below" 96%, "column to the right" 74%, "enemy adjacent" 49% when the enemy is there. Five agent components, 26 to 103 times faster, and no column where quality clearly wins.
The page also says about 200 ms and about $0.000017 per function. The repo root returned no LICENSE file when we requested it. We read fit.md. We did not re-run the case studies, and we did not call Jev.
On September 30, 2026 we opened docs/22-code-criteria.md. Section 11.1, three repeats at the fitted thresholds, puts easy bugs at 15/15 on the full rubric whether the criterion is named or held out. Hard bugs on that rubric are 11/21 when named, 52.4%, and 3/15 when held out, 20.0%. The page calls the gap about 32 points, about 2.6 times.
The same section says the earlier 15/15 reading was easy bugs. The addendum splits held-out bugs into 5 easy and 5 hard. The corpus there is 78 functions in 15 files, with 22 bugs.
Section 3, also three repeats, asks the matching criterion on the six misses. Two came back: Array.sort with no comparator at 0.32, and setUTCMonth overflow at 0.27. Four stayed no: a missed await at 0.22, splice with too few arguments at 0.12, a regex without the g flag at 0.09, and an LRU cache that does not update order at 0.11.
Section 11.2 drops one criterion at a time. Seven of the eight lose nothing on their own class. api_default falls from 9/12 to 4/12.
Compare
TypeSafe's jaggedness page already says to keep arithmetic in code and to filter unrelated state. This table is an independent set of counts for those two warnings: a one-line rule beats Jev on page triage, and a named API question still comes back no.
The 0 illegal moves are the same shape as Tetris and the macOS loop, where the host lists the actions. They are not a JevBench score. The 26 to 103 times speed line is against other models inside five of his components, and the page says quality does not follow.
Terms
- 0 illegal
- fit.md's count for jobs whose options are listed up front. 489 MOBA decisions and 37 chess moves, and none of the answers was an illegal move. The page treats that as what happens when an illegal answer cannot be written down.
- 0.09 to 0.22
- The probability of yes on questions about an API behavior the snippet does not state, after the question names the behavior. The page's examples are Array.sort with no comparator, splice to the end, a regex without the g flag, and a promise that leaves the try.