jevbench.app replays recorded Jev answers on labeled examples, and one test is 20 of 21

Yusuke, @yusukelp, posted a soft launch on September 28. The product is jevbench.app. A test pairs one Jev question with examples a person has already labeled, then shows where the recorded answer disagrees. The page says the playground stores real Jev runs and that opening a test makes no new model call. There is no signup. The posts and the two pages we opened do not link a GitHub repository. The name collides with Florian S’s JevBench, the 534-decision board. This one does not publish a composite, a sealed set, or a price.

The index prints one score. On lead qualification, Jev matched 20 of 21 human labels. The miss is quoted on the page: “I need a cheaper alternative to Klaviyo. In my article today I compare five options for readers.” The human label is no. Jev’s recorded answer is yes, score 0.51. The test page says the set is 21 examples, 10 labeled yes and 11 labeled no, recorded September 27, 2026. The question is whether the person is actively looking for help with an email marketing or deliverability problem that an email product could solve. Yes means a current problem or a shopping trip. No means commentary, promotion, or praise without buying intent. A visitor can move a decision line across the recorded scores. We did not move it, and we did not export the 21 rows.

Seven more tests are linked from the same index: support triage, content moderation, directory submissions, email triage, customer feedback, scraped-data relevance, and human-review routing. Each has a one-line definition of yes. The index does not print a match count for any of them. The only fraction we read off the site is the 20 of 21. A later measurement of sexual text, on a different set, is jevmod.