Published
Updated
Rox's chart keeps 96% of relevance at 200 chunks, against 84% in production
This topic was created 30 days ago, and the information it contains may have evolved or changed since then.
Rox posted on September 28 a chart titled Jev vs. our production reranker, a mean over 75 queries. At 200 chunks the printed callouts are Jev-Score 96% and production 84%. The post, and a TypeSafe quote on September 30, say 20 times faster, 10 times cheaper, and 12% more accurate. Those multiples are not on the chart. We did not open a methods note.
Rox posted on September 28 a chart titled “Jev vs. our production reranker.” The subtitle is “Mean over 75 queries.” The horizontal axis is chunks sent to the agent, from 50 to 200. The vertical axis is relevance kept, in percent. The series are Jev-Score, Jev-Noul, Production (GPT-5 Mini), GPT-6 Astra as a dashed line, and Random order.
At 200 chunks the printed callouts are Jev-Score 96% and production 84%. GPT-6 Astra is drawn along the top of the chart and has no printed number. Random order is the lowest series and rises as more chunks are sent. The chart does not label that series at the right edge.
The post says 20 times faster, 10 times cheaper, and 12% more accurate. TypeSafe quoted that sentence on September 30. The speed and price multiples are in the post, not on the chart. The 12 point gap at 200 chunks, 96 against 84, is the accuracy figure the chart prints. The post’s “12% more accurate” is the post’s wording of that gap. We did not open a methods note or a repository. The only query count in view is 75.
This site's reading
Editorial notes evaluating claims against primary sources, contextualizing findings alongside related implementations, and defining technical terms.
Verify
Primary post is @rox_ai, September 28, 2026, 18:41 UTC. The chart title is "Jev vs. our production reranker." The subtitle is "Mean over 75 queries." The x-axis is "Chunks sent to the agent," marked from 50 to 200. The y-axis is "Relevance kept (%)."
Series drawn: Jev-Score, Jev-Noul, Production (GPT-5 Mini), GPT-6 Astra as a dashed line, and Random order. At 200 chunks the printed callouts are Jev-Score 96% and production 84%. GPT-6 Astra is drawn along the top and has no printed number. Random order is the lowest series and rises as more chunks are sent. The chart does not label that series' endpoint.
The post text says 20x faster, 10x cheaper, 12% more accurate. TypeSafe quoted that line on September 30, 2026, 17:57 UTC. The 20 times and the 10 times are in the post, not on the chart. The 12 point gap at 200 chunks is 96 against 84. We did not open a methods note or a repository. The only query count in view is 75.
Compare
andre's file search moves nDCG@10 from 0.322 to 0.377 on NFCorpus, 623 queries, after Jev reads the top 128. McLaren's jevsearch prints Hit@1 of 83% against a keyword pass at 41% on 41 labelled docs queries. Rox's chart is relevance kept, mean over 75 queries, at a chunk count. It is not nDCG and it is not Hit@1.
The production series is labelled GPT-5 Mini. Jev's published list price on this desk is $0.042 per million input tokens, output free. The chart does not print a dollar figure. The 10 times cheaper line is the post's.
Terms
- 96%
- Jev-Score at 200 chunks on Rox's September 28 chart, mean over 75 queries, relevance kept. Production, labelled GPT-5 Mini, is 84% at the same point.
- 75 queries
- The subtitle of Rox's chart. We did not open a methods note, a query list, or a repository. The post's 20 times faster and 10 times cheaper are not drawn on the chart.