StreamDecisionBench puts Kev-27B at 62.15 and Jev at 59.63

This topic was created 28 days ago, and the information it contains may have evolved or changed since then.

Jacob Lin posted StreamDecisionBench on October 2, 2026. The chart's single-model leaderboard is normalized log-AUC over 0.5 to 8 seconds. Kev-27B is 62.15. The Jev API is 59.63. The live page's HTML did not render those numbers. The October 1 paper table is a different scoring.

Jacob Lin posted StreamDecisionBench on October 2, 2026. The chart asks what decision is actually in force while the application state keeps changing. The score is a normalized log-AUC over windows from 0.5 to 8 seconds. Higher is better.

The thread points at jacoblincool.github.io/StreamDecisionBench for the leaderboard and the code. Opening that URL returned the page shell and no numbers. The table below is the chart image on the post. The thread text matches the top of it: Kev-27B at 62.15, and the Jev API at 59.63. Lin writes that Kev-27B is the only open-weight model in the set that lands in the same range as Jev, and that the lab tested Kev-4B first, then went back for Kev-9B and Kev-27B.

The chart

Rank	Model	Kind	Score
1	Kev-27B	open weight	62.15
2	Jev	API	59.63
3	GPT-5.6-Terra, reasoning none	API	54.11
4	GPT-5.6-Terra, low	API	48.04
5	GPT-5.6-Luna, low	API	45.26
6	GPT-6-Astra, low	API	45.15
7	Cloudflare Clef	API	30.96
8	GPT-5.6-Luna, reasoning none	API	30.67
9	Kev-9B	open weight	28.76
10	Kev-4B	open weight	21.22
11	DJev / DiffusionGemma	open weight	20.89
12	Cloudflare Clef Flash	API	19.69
13	Qwen3.5-4B, direct logits	open weight	15.61
14	Bespoke Nimble-9B	open weight	14.43
15	Laya, typed decisions	open weight	1.46
16	Laya, English	open weight	0.42
17	Laya, multilingual	open weight	0.21

A second image in the thread plots hosted APIs as quality against the decision interval. It does not print numeric callouts. On that plot Jev is high at the short intervals, the chat models rise toward 8 seconds, and Clef and Clef-flash stay low. This note does not read extra values off the curves.

What this is not

Kev’s own new-source split put Kev-8B at 79.6% against Jev at 85.7%. The September 26 paired run is a third set, full JSON schema on 1,253 function-selection cases. The 62.15 here is log-AUC on a moving state. Lin’s order inside the Kev family on this chart is 27B, then 9B at 28.76, then 4B at 21.22.

Nimble-9B at 14.43 is not the Bespoke holdout of 292 of 324 against Jev at 302 of 324. Clef at 30.96 is not the latency row on the Clef page, and it is not the 61.2 on the GLiDE chart.

arXiv:2609.38612 names Jhen-Ke Lin and Chung Chun Wang at NYCU. Version 1 was submitted September 29, 2026. Version 2 is dated October 1. That paper’s table uses another scoring, including an older Jev line near 59.6% and a Kev number that is not 62.15. Those cells stay in the paper. They are not rows on the October 2 chart. The pages we read do not say that the X account and the paper’s first author are the same person.

We did not rerun the streams.

This site's reading

Editorial notes evaluating claims against primary sources, contextualizing findings alongside related implementations, and defining technical terms.

Verify

Jacob Lin, @JacobLinCool, October 2, 2026, 17:41 UTC, status 2106077196553199892. The first attached image is the chart saved for this page. Its title is "Single-model leaderboard." The subtitle says normalized log-AUC over 0.5 to 8 seconds, higher is better. The thread links https://jacoblincool.github.io/StreamDecisionBench/ for the leaderboard and the code.

Opening that URL returned the JavaScript shell and no table. The numbers in this note are the ones printed on the chart image. The thread text matches two of them: Kev-27B at 62.15, and Jev at 59.63, with Kev-27B called the only open-weight model in that range. The thread says the lab tested Kev-4B first, then Kev-9B, then Kev-27B.

The chart lists eight hosted APIs and nine open-weight rows. In order: Kev-27B 62.15, Jev API 59.63, GPT-5.6-Terra with reasoning none 54.11, Terra low 48.04, GPT-5.6-Luna low 45.26, GPT-6-Astra low 45.15, Cloudflare Clef 30.96, Luna none 30.67, Kev-9B 28.76, Kev-4B 21.22, DJev / DiffusionGemma 20.89, Clef Flash 19.69, Qwen3.5-4B direct logits 15.61, Bespoke Nimble-9B 14.43, Laya typed 1.46, Laya English 0.42, Laya multilingual 0.21.

A second image in the thread is a tradeoff plot of hosted APIs. It has no printed callout we are treating as a number. Jev sits high at the short intervals. The chat models climb toward 8 seconds. Clef and Clef-flash stay low on that plot.

arXiv:2609.38612 names Jhen-Ke Lin and Chung Chun Wang, NYCU. Version 1 was submitted September 29, 2026. Version 2 is dated October 1. That paper table is a different scoring. It is not the source of the chart cells above. We did not rerun the benchmark.

Compare

Kev-27B at 62.15 and the Jev API at 59.63 are this chart's log-AUC. They are not Jared Palmer's new-source split, where Kev-8B was 79.6% against Jev at 85.7%, and they are not the September 26 paired BFCL counts.

Bespoke Nimble-9B at 14.43 on this chart is not the 90.12% against 93.21% holdout, 292 of 324 against 302 of 324. Clef at 30.96 here is not the blog's median of 209.3 ms, and it is not the 61.2 on Fastino's Decision Index chart.

The paper's October 1 table includes an older Jev cell around 59.6% and a Kev line that is not this 62.15. Mixing those cells with this chart would cross two scorings. The pages we read do not say that Jacob Lin and Jhen-Ke Lin are the same person.

Terms

StreamDecisionBench: Jacob Lin's October 2, 2026 chart of the decision in force while the state keeps changing. The single-model score is normalized log-AUC from 0.5 to 8 seconds. The site HTML we opened did not render the table. The code link in the thread points at the same site.
0.42: Laya, English instructions, on the October 2 chart. The multilingual Laya row on the same chart is 0.21. Typed Laya is 1.46. These are log-AUC cells, not accuracy percents.