Updated
Topic
Evals
Measurements that print a sample size, a table, or the author's caveat. 117 stories.
OR-decision-4B prints JevBench 193 of 231, and Choice 79.7 against Jev 79.6 on alphaXiv's chart
alphaXiv posted OR-decision-4B on October 30, 2026, at 16:06 UTC. The Hugging Face card prints JevBench 193 of 231, 83.55%. The launch chart prints Choice 79.7 against Jev 79.6, with sarcasm at 53.6 against Jev 70.7. Apache-2.0 LoRA on Qwen3.5-4B. This desk did not run the weights.
MiniCPM5-2B-Jev prints 78.79% on JevBench, first among open models at or under 2B
OpenBMB posted MiniCPM5-2B-Jev on October 28, 2026, at 13:01 UTC. The project prints 78.79% on JevBench, first among open models at or under 2B, and 86.41% accuracy with ECE 0.0307 on decision-v7. Apache-2.0 on MiniCPM5-2B. This desk did not run the weights.
Opened October 26, Decision Index 0.4.1 Full score for d3-ultra is 70.18, second of 229
Xunzhuo Liu posted d3-ultra on October 27, 2026, at 03:01 UTC. Opened October 26, Decision Index 0.4.1 Full score is 70.18, second of 229, +3.03 over d3 at 67.15. The card's private SR-Text mean is 87.2. Apache-2.0. This desk did not run the weights.
Decision Model Factory prints DF-JA-Large at 67.63 on S1MB Japanese Task Avg against a public snapshot high of 60.05
Yuma Ichikawa posted Decision Model Factory on October 27, 2026, at 03:19 UTC. arXiv 2610.34956 prints DF-JA-Large at 67.63 on S1MB Japanese Task Avg against a fixed public snapshot high of 60.05. No public weights on the pages this desk opened. This desk did not run a build.
Abe's 1,774 Snowflake posts cost $0.055 on Jev Noul against $1.87 on AI_FILTER
ABE Masanori posted on October 24, 2026, at 11:20 UTC. On 1,774 of his 2025 X posts, Jev Noul cost about $0.055 against AI_FILTER at about $1.87, about 1/34. AUC against AI_FILTER was 0.9685. Of 70 splits where Jev scored under 0.2 and AI_FILTER was TRUE, he judged Jev right on 54. Date headers 08:49:12 GMT to 08:49:40 GMT October 26, this desk's jev-1.14.0 noul on his Lightdash example was 0.04.
Base Labs posts mammoth-1 at Decision Index 0.4 Full 71.24; live 0.4.1 prints 68.4
Base Labs posted mammoth-1 on October 21, 2026, at 17:14 UTC, and the card prints Decision Index 0.4 Full 71.24, first of 208. Opened October 23, the live space is edition 0.4.1, 229 models, mammoth-1 Full 68.4 rank 3. Date headers 02:23:21 GMT to 02:23:23 GMT, GET /v1/models with no key returned HTTP 401.
Sthānika's Sieve-31B card prints Decision Index 0.3.2 Full score 63.8
Cuth posted Sieve 31B on October 20, 2026. The Hugging Face card prints Decision Index 0.3.2 Full score 63.8, board data of October 14, and a public index of 62.52 on the authors' own 0.3 run. Apache-2.0 LoRA on Gemma 4 31B. This desk did not run the weights.
Opened October 26, Decision Index 0.4.1 Full score for d3-turbo is 67.31, and vision full is 72.46
Xunzhuo Liu posted d3-turbo on October 20, 2026, at 00:42 UTC. Opened October 26, Decision Index 0.4.1 Full score is 67.31 and vision full is 72.46, second of 56. The card's own 0.3.2 estimate is 70.2. Apache-2.0. This desk did not run the weights.
Datadog prints Jev Noul recall 82.90% to 3.88% under a counterfeit policy, and this desk's is_risky fell from 0.94 to 0.13
Datadog Security Labs' October 15 write-up, posted October 19, prints Jev Noul recall 82.90% on raw attacks and 3.88% with a counterfeit policy prepended. Date headers 08:50:46 GMT and 08:50:47 GMT October 20, this desk's jev-1.13.0 is_risky was 0.94 then 0.13.
Nace posts Drex Max at Decision Index 0.3.2 public 67.59, $0.06 per million input
Nace posted Drex Max on October 19, 2026. The product page prints Decision Index 0.3.2 public 67.59 against Jev 1.14 at 60.70. Docs price input at $0.06 per million tokens. Date headers 08:50:01 GMT and 08:50:02 GMT October 20, GET /v1/models with no key returned HTTP 401.
Qwen-Audio-Jev prints 78.70% on MMAU-mini and about 20x the base model's speed
Qwen-Audio-Jev prints 78.70% on MMAU-mini and a 20x speedup against Qwen-Audio-3.1-ASR-Flash-Next. The paper is arXiv 2610.21614, submitted October 15, 2026. This desk did not run the model.
ToddlerJev prints 68.70% on MMAU-mini, with weights under the Qwen Research License
ToddlerJev, posted October 18, 2026, prints 68.70% on MMAU-mini, 61.30% on MMSU, and 56.90% on MMAR. Weights are TeenageWorks/Toddler-Jev under the Qwen Research License. This desk did not run them.
Opened October 19, JevBench ranks basal-omni-1.0-ultra first of 184 open systems on Capability at 82.5
Opened October 19, 2026, JevBench ranks basal-omni-1.0-ultra first of 184 open systems on Capability at 82.5. Composite among open weights is 71.8, rank 3. The card prints Werdykt v1 at 0.879. This desk did not run the weights.
Opened October 17, JevBench API board v1.7.46 ranks kettei-fat first of 39 at composite 77.8
Opened October 17, 2026, JevBench API board v1.7.46 ranks kettei-fat first of 39 at composite 77.8. ImageJevBench v0.3.0 capability is 75.9. Docs price kettei fat at $0.06 per million input tokens. GET /v1/models with no key returned HTTP 401.
Opened October 16, Artificial Analysis Decision Index prints d1-ultra at 65 and Jev 1.14 at 57
Vals AI, October 22, 2026, 23:57 UTC, printed d1-ultra as the only decision model to get all 1,960 ClaimBench claims right. Opened October 16, Artificial Analysis Decision Index prints 65 against Jev 1.14 at 57. Liquid's blog prices input at $0.042 per million tokens, output free. This desk did not call the API.
On 492 tool calls, OpenRouter Armor's default Jev 1.13 scored 90.7%, 446 of 492
On 492 tool calls, OpenRouter Armor's default Jev 1.13 scored 90.7%, 446 of 492, at $0.0021 a call. On October 16, 2026, at 01:18 UTC, this desk's jev-1.13.0 noul was 0.98 on git push origin main and 0.01 on ls ./reports.
Span-02's card prints Decision Index 0.3 text index 63.62, and the blog prints thinking off at 61.2 against Jev 58.0
Hendrix Liu posted October 16, 2026, 23:16 UTC, that Span-02 had more usage in under 24 hours than Span-01 did in 20 days. The card still prints Decision Index 0.3 text index 63.62. The blog prints thinking off at 61.2 against Jev 1.13.0 at 58.0. This desk did not run the weights.
HiddenLayer prints Jev at 91% Confirm Price on a checkout ad, and this desk's jev-1.13.0 still chose place_order at 0.89
HiddenLayer's October 14, 2026 write-up prints TypeSafe Jev at 91% Confirm Price and 9% Place order on a checkout ad. Date headers 07:24:57 GMT and 07:24:58 GMT October 19, this desk's jev-1.13.0 still chose place_order, 1.0 then 0.89.
On October 26 TypeSafe's jev-latest named jev-1.14.0, and both aliases point at 1.14.0
Date headers 08:49:11 GMT to 08:49:40 GMT October 26, POSTs with jev-latest and jev-preview named jev-1.14.0. Docs say both aliases point at jev-1.14.0, and that there is no preview build. A named jev-1.13.0 POST still named 1.13.0. At 00:28 UTC October 16, jev-latest still returned jev-1.13.0.
Jeva-4B prints 0 flips on 600 Choice permutations, and BANKING77 at K=16 is 91.62 against Jev 82.50
Hugging Face DailyPapers posted Jeva on October 14, 2026, at 20:10 UTC. The paper prints 0 flips on 600 Choice items and BANKING77 at 16 candidates 91.62 against Jev-1.13.0 at 82.50. Apache-2.0 on Qwen3.5-4B. This desk did not run the weights.
On DeepEval's Paris weather example, jev-1.13.0 scored 0.254 when the answer invented a breeze
On October 14, 2026, at 01:58 UTC, two POSTs to jev-1.13.0 scored DeepEval's invented-breeze case at 0.254. The matching clean answer scored 0.901. Sumanth's October 13 post points at JevEval, DeepEval's Apache-2.0 metric that maps Jev probabilities through a fixed weighted mean. The docs walkthrough that prints 0.461 is illustrative. This desk did not install deepeval.
On 40 client requests, Jev and Opus both scored 37 of 40, at $0.0015 against $0.005 to $0.015
Akshay Pachaar's TrueForge agent scored 37 of 40 client-issue matches with Jev and 37 of 40 without it. The printed decision costs are $0.0015 and $0.005 to $0.015. Review caught 5 wrong Jev matches. On October 13 he wrote that 40 is too small to establish how reliably confidence routes errors to review. On October 14, 2026, at 02:34 UTC, this desk's jev-1.13.0 call picked the intended issue at confidence 1.0. A trap shortlist scored 0.51. This desk did not install TrueForge.
On eight Stripe passages, a 0-9 Score put Roll secret at 8.86 and the other six at 2.42 or below
On eight Stripe passages this desk sent to jev-1.13.0, a 0-9 Score put the Roll secret steps at 8.86 and the other six at 2.42 or below. Pairwise Choice ranked rotating the API key third, and the majority graph had a cycle.
SubQ Snap 1.1 prints 65 on Artificial Analysis Decision Index, $0.04 per million input
On October 23, 2026, Date headers 02:23:17 GMT to 02:23:20 GMT, three POSTs from this desk to SubQ Snap returned HTTP 200. Docs name subq-snap-1.1 at $0.04 per million input tokens. GET /v1/models listed subq-snap-1.1 and subq-snap-preview. Artificial Analysis Decision Index prints Snap 1.1 at 65, tied with d1-ultra and Elephant 9B. The October 12 blog prints Decision Index 0.2.1 at 61.7 against a Jev cell of 57.9.
Opened October 26, Decision Index 0.4.1 Full score for d3 is 67.15; d3-turbo and d3-ultra have their own pages
Opened October 26, 2026, Decision Index 0.4.1 Full score for d3 is 67.15. The free API aliases vllm-sr/d3 to d3-turbo. Sibling pages cover d3-turbo at 67.31 and d3-ultra at 70.18. The October 16 card reading of 66.3 on edition 0.3.2 stays.
Mizan 4B averages 78.7 on 3,000 Persian records
xmanii/Mizan-4B prints a mean accuracy of 78.7 on 3,000 real Persian records, against the card's Jev cell of 83.7. Clef 27B on that table is 83.4 and Laya is 62.4. The Q4 GGUF mean is 78.3 at 2.5 GB. The card says the weights are Apache-2.0, from Qwen3.5-4B, and text only. We did not run the weights.
THX-01 averages 98.4 on its 2,843 support tickets
HAL-X's THX-01 card averages 98.4 on 2,843 support tickets, against Jev 1.13 at 97.4 on the same four sets. Number lookup on a different set is 93.4, and Jev on that line is 96.6. The card says 322M parameters, Apache 2.0, on mmBERT-base. The October 11 post calls the API free. The docs page we opened does not print a price. We did not call the API.
PurpleMIST-Flash-1.0 scores 0.668 on 15,050 translated African-language decisions
PurpleMIST-Flash-1.0 scores 0.668 accuracy on 15,050 translated African-language decisions, against DeepSeek-V4-Pro at 0.669. KL on that set is 0.214 against 1.125. The card says the checkpoint is Apache-2.0, about 7.9B parameters, from Qwen3.5-9B-Base, and text only. A self-run of Decision Index edition 0.3 prints a public index of 46.7. We did not run the weights.
JevBench ranks Sage 1.3.0 first of 27 hosted systems at composite 74.0
Opened October 11, 2026, the JevBench API board ranks Sage 1.3.0 first of 27 hosted systems at composite 74.0. Levanto's pricing page lists Pro input at $0.042 per million tokens, and output at $10 per million for reasoning. Answers on that page are free. SageCraft's October 9 log is 4,573 decisions and level 5 at 2 cents a kill. We did not call the API.
TypeLLM scores Qwen3.8-27B at 228 of 231 public JevBench tasks when thinking is on
On October 16, 2026, at 05:44 UTC, TypeLLM posted that Jev 1.14 cannot return an array of vehicle boxes, including Tesla's flying Roadster. The cookbook at 14.00 s prints six boxes. The September 23 file still scores Qwen3.8-27B at 228 of 231 public JevBench tasks with thinking. This desk did not call the API.
Vega, the vegaml package, scores 75.4% on 800 phishing emails against Jev at 61.9%
Nandha Kishor M's Vega, the vegaml package, scores 75.4% on 800 phishing emails against Jev 1.13.0 at 61.9%. The default checkpoint is 0.8B on frozen Qwen3.5-0.8B, Apache 2.0, with a 4B sibling. Decision 2.0's Vega is a different checkpoint. We did not run the weights.
On WANDS, more Jev questions move the mean to 0.610 and leave the median at 0.561
Doug Turnbull's WANDS table prints mean NDCG@10 of 0.610 for a bag of Jev questions and 0.574 for one question. Both medians print 0.561, and the CSV stores one shared float. The score adds ten times the summed Nouls to BM25. This desk checked the CSV and did not rerun the searches.
Clef-omni scores a 21-second video in about 1.5 seconds, at $0.15 per million input tokens
Cloudflare's October 9, 2026 blog prices Clef-omni at $0.15 per million input tokens. The same post cuts hosted Clef-flash from $0.09 to $0.038 and from a 64k window to 24k. A 21-second video with sound scores in about 1.5 seconds. On that post's ten-benchmark table, Clef-omni's BANKING77 macro-F1 is 94.8. The weights are Apache 2.0, from Qwen3-Omni-30B-A3B-Instruct. We did not call the API.
Microsoft-Decision-1 lists $0.042 per million input tokens and calls itself 4.5 times quicker than Quyet-1.0-Large
Opened October 11, 2026, the JevBench API board ranks Microsoft-Decision-1 sixth of 27 hosted systems at composite 69.1. On October 9, 2026 Microsoft posted the model at $0.042 per million input tokens, with output free. The post says it was 4.5 times quicker than Quyet-1.0-Large and 35 times quicker than GPT-6 Sol, and it does not print an accuracy. The Foundry catalog, opened October 10, lists a 32,768-token window and says the weights are not distributed. A search of the October 10 API page found no Microsoft-Decision-1. We did not call the API.
OpenBles Firewall 1 catches 119 of 120 agent attacks and flags no benign look-alike
OpenBles Firewall 1 caught 119 of 120 agent attacks and flagged none of 120 benign look-alikes on its held-out test. The README prints classic-set recall at 55 of 60, with 2 of 56 benign prompts flagged. The adapter is MIT, 31.8 MB, on fastino/GLiNER2.5-Decide. We did not run the weights.
Step-Jev scores 19.6% when the reward grades every search step
On October 9, 2026 hosted jev-1.13.0 scored the repository's no-step test state incorrect at probability 1.0. The five-step fixture in the same test was correct at 0.05. On a Moon-landing question written here, a no-search answer of 1965 was incorrect at 1.0, and a repeated NASA search stayed positive at 0.53. real_jev_gate.py's rollout files are not in the repository.
Celeris-1 Decision prints 80.0% on 3,210 jev-bench items at a 67 ms median, and Artificial Analysis lists it at 52
Celeris-1 Decision, posted October 8, 2026, prints 80.0% on 3,210 jev-bench items at a 67 ms median, and $0.04 per million input tokens. Opened October 16, Artificial Analysis Decision Index lists it at 52. GET /v1/models with no key returned HTTP 401. This desk did not send a billed POST.
H2O-Lightning-4B v1.1 prints JevBench composite 72.5 against Jev at 71.5
H2O's October 7, 2026 blog prints H2O-Lightning-4B v1.1 at JevBench composite 72.5, against Jev 1.13.0 at 71.5. That reading is board revision v1.7.17 and 104 open-weight systems. The blog says the weights are Apache 2.0, from Qwen3.5-4B. We did not open the card, and we did not run the weights.
CLERC top-1 moves from 5% to 18% when jev-1.12 reranks the shortlist
TypeSafe's CLERC cookbook moves top-1 from 5% to 18% when jev-1.12 reranks a BM25 shortlist of 30 on 40 queries. Hindsight's listwise rerank later prints LoCoMo recall@1 0.950 against MiniLM at 0.800. aifabrice's best NFCorpus row is nDCG@10 0.451. We did not rerun them.
Arena scores typesafe/jev-router at +3.6% and $0.155 a task
Arena's October 7, 2026 thread scores typesafe/jev-router at +3.6% net and $0.155 a task. The same thread says similar performance to DeepSeek V4.1 Flash (Max) costs 38% more, with median latency 1.7 times higher, on more than 4,700 agent sessions. The model page FAQ says the router has no price of its own. We did not call it.
Open d1's d1-3B card prints 48.57 on a self-run of Decision Index 0.2.1
On October 7, 2026 Liquid AI posted Open d1, and the d1-3B card prints 48.57 on a self-run of Decision Index 0.2.1. Winnow-12B on that table is 50.02. d1-omni-600M is 15.95. The LICENSE file is LFM Open License v1.0, with commercial use under a $10,000,000 revenue line. We did not run the weights.
PolicyLM-1.7B scores each policy category from 0 to 1
PolicyLM-1.7B prints custom-policy accuracy 0.842 at a 0.5 cutoff, and it returns a score from 0 to 1 for each category, up to 16 in one pass. The weights are Apache-2.0. The table has no Jev column.
Vela 2.0's 9B card prints 41.63 on Decision Index 0.2.1
The Vela 2.0 9B card prints Decision Index 0.2.1 at 41.63 with Noul calibration on, and 41.09 with it off. The same evaluation file prints the Lux base at 46.23 and does not print a Jev row. Four Apache-2.0 checkpoints also return span offsets. We did not run the weights.
Nace posts Drex 1.1, and the card titles the weights Drex DLM
Nace posted Drex 1.1 on October 6, 2026, and the card scores Decision Index 0.2 at 52.31 against the kit's Jev cell of 51.67. Weights are CC BY-NC 4.0. Nace's code is MIT. The call is POST /v1/systemone. The hosted Drex 1.5 row on Decision Index 0.2.1 is 58.28.
Blink v0.3 prints Decision Index 0.2.1 at 57.48
Pixilab's Blink v0.3 card prints a self-run of Decision Index 0.2.1 at 57.48, against the board's Jev cell of 57.91. The card's license field is Apache-2.0. The base is Gemma 4 26B-A4B. We did not run the weights.
fsdecide scores 91.3% on ForkScape review, and Jev scores 56.3%
ForkScape posted fsdecide on October 5, 2026, a 149M ModernBERT trained on the app's own decisions. On 80 review items it scores 91.3% against jev-latest at 56.3%. On the authors' 1,248-request general set, jev-latest is 90.5% and fsdecide is 60.5%. That set is not Florian's JevBench.
Quyet-1.0-Large's homepage prints JevBench v1.6.0 at 71.39
Quyet-1.0-Large's homepage prints JevBench v1.6.0 official score 71.39, rank 1 of 92, and Capability 81.7. The card says Apache-2.0, 31.3B, from Gemma 4 31B. A later reading on this site still lists Capability 81.7 and official composite rank 3. We did not run the weights.
basal-1.5 scores 72.12% on Werdykt, and Jev 1.13.0 scores 81.64%
Remek Kinas posted basal-1.5 on October 4, 2026. Werdykt v1, exported that day, puts the 4.5B at 72.12% and Jev 1.13.0 at 81.64%. A closed set of 3,000 new Polish decisions is 93.1% against basal-1.0 at 87.4%. The site schedules the weights for October 5. Opened October 8, 2026, GitHub main titles the README basal 1.5 and says the models are on Hugging Face from 2026-10-05.
JEV-27B-VL finishes 15 of 20 pick-and-place scenes
AutoTrust posted a JEV-27B-VL follow-up on October 4, 2026. The model card's October 3 section prints 15 of 20 pick-and-place scenes, and 95% of 60 browser tasks when the element text is included. A six-benchmark mean from September 27 is 84.07 against a hosted Jev cell of 83.85. That JevBench column is not the official board.
AutoTrust scores GEV-26B-Decide at 62.48 on its own Decision Index run
AutoTrust posted GEV-26B-Decide on October 3, 2026, a Gemma 4 26B model with about 4B active. The card's own Decision Index 0.2.1 scoring prints an adaptive 62.48 against the September 28 board's Jev cell of 57.91. Adaptive thinking on Knowledge misses the kit's latency cap. A 1,754-question holdout moves from 73.3% to 83.4% when thinking is allowed. The project is not affiliated with TypeSafe.
NIRNAY 450M prints 0.8792 on 3,080 Banking77 rows and says the Jev cell was not measured here
Gautam Kishore posted NIRNAY 450m on October 3, 2026, an Apache-2.0 Laya fine-tune from Eulogik. The phase_b JSON is 0.8792 on 3,080 Banking77 test rows, Brier 0.208, fitted ECE 0.045. The post's headline 0.8656 is the phase_a file, set against a Jev figure of 0.64. The card also prints Jev at 0.803, and the post says that column was not measured in the repo. The eval JSON has no Jev field.
StartLux-Decision v1.1 27B Instant prints Full score 69.12, first of 174 on Decision Index 0.3.2
The StartLux-Decision README, opened October 20, 2026, prints v1.1 27B Instant at Full score 69.12, first of 174 on Decision Index 0.3.2. The October 3 self-run of edition 0.2.1 stays 63.88. Code is Apache-2.0. Weights are CC BY-NC 4.0.
WaterSheep fine-tunes ModernBERT and serves /v1/systemone with a multi-label question
Samrat Dutta posted WaterSheep on October 3, 2026. The Apache-2.0 repo fine-tunes ModernBERT-base and serves POST /v1/systemone on port 8766, with noul, choice, score, and multi. The README prints 77.8% on an in-distribution split, ECE 0.026, and 61.2% on held-out datasets, ECE 0.043. Those two rows do not print a question count, and the page has no Jev column. We did not run it.
Decision 2.0's Vega card prints 56.5 on its own Decision Index run
vLLM Semantic Router posted Decision 2.0 on October 3, 2026. Six Apache-2.0 checkpoints run from Kai at 0.6B to Vega at 27B. Vega's card prints 56.5 on a reproduction of Decision Index 0.2.1 and does not print a Jev row. Its JevArena cell is 74.0, which the card calls level with AutoJev-27B at 72.1. Vela 2.0, posted October 7, is a later fine-tune and has its own page. Nandha Kishor M's October 10 package named Vega is a different model. Decision 3.0, posted October 11, has its own page. d3's card prints 64.7 on its Decision Index 0.3.1 table.
A CUA Speedrun chart prints Jev at 9.8% on OSWorld-50, 125 seconds a task
Pranjal Aggarwal posted a CUA Speedrun chart on October 2, 2026. The printed callout is Jev at 9.8% success on 50 OSWorld tasks, 125 seconds a task. The chart's own annotation says that is as fast as Astra xhigh at 9x lower accuracy. The opening post says 1.5x slower than Opus-5.5-xHigh and 10x lower accuracy. Those two comparisons are not the same sentence.
OpenAI Decisions API is generally available; this desk's October 23 POSTs returned HTTP 200
On October 23, 2026, Date headers 02:23:12 GMT and 02:23:13 GMT, two POSTs from this desk to the Decisions API returned HTTP 200 from gpt-6-luna. OpenAI Developers posted general availability on October 22, 19:07 UTC. The guide prices uncached input at $0.10 per million tokens and cached input at $0.01, with no output charge. Requests now hold up to 400 questions. On October 8, 2026 ten urgency calls had a median clock of 668 ms on Jev and 387 ms on Decisions.
StreamDecisionBench puts Kev-27B at 62.15 and Jev at 59.63
Jacob Lin posted StreamDecisionBench on October 2, 2026. The chart's single-model leaderboard is normalized log-AUC over 0.5 to 8 seconds. Kev-27B is 62.15. The Jev API is 59.63. The live page's HTML did not render those numbers. The October 1 paper table is a different scoring.
Cloudflare posts Clef and Clef-flash, Apache 2.0 decision models on Workers AI
On October 9, 2026 Cloudflare's blog prices Clef-flash at $0.038 per million input tokens, down from the $0.09 on the October 1 card. Clef stays at $0.24. Clef-omni, posted the same day, is a separate model at $0.15. Cloudflare posted Clef and Clef-flash on October 1, 2026, two Apache 2.0 decision models hosted on Workers AI and released on Hugging Face. The October 1 blog's median latency row is 209.3 ms and 38.8 ms against a printed Jev cell of 524.1 ms. On October 3 a Fastino chart prints Clef at 61.2 and Clef-flash at 57.1, and captions those Cloudflare scores as self-reported. llama.cpp v0.6.0, published October 5, serves Clef on /v1/systemone, including images. The October 6 note leaves Clef's speed cell blank.
GLiDE v1.2 prints Decision Index 0.4.1 Full 71.5, first of 229, at $0.04
Fastino posted GLiDE v1.2 on October 22, 2026, at 16:36 UTC. Opened October 26, Decision Index 0.4.1 Full score is 71.5, rank 1 of 229. Docs price input at $0.04 per million tokens, output free. Date headers 08:49:09 GMT to 08:49:11 GMT, GET /v1/models returned HTTP 200 and omitted GLiDE, POST /v1/systemone returned HTTP 401. JevBench lists Fastino GLiDE (hosted API, 22 Oct 2026) at composite 76.4, rank 3 of 48. The October 1 self-run of 64.81 stays.
HydroJEV tests jev-1.13.0 as a one-second screen on simulated water-network alarms
Tianwei Mu and coauthors test Jev as a first screen on simulated water-network alarms, and a gate settles 69 of 200 and 76 of 200 windows.
A frozen Jev pipeline reports the lowest ten-corpus valence-arousal error in a SemEval table
Yiqun Zhang and coauthors report the lowest ten-corpus valence-arousal error among full-coverage systems, using frozen Jev and 488 fitted coefficients.
Perplexity posted pplx-decider-v1.2 on October 16, with Decision Index 0.3 overall 68.09 on joint requests against v1.1 at 62.30
Perplexity posted pplx-decider-v1.2 on October 16, 2026, with Decision Index 0.3 overall 68.09 on joint requests against v1.1 at 62.30, at $0.02 per million input tokens. Flash is $0.01. The October 1 panel still prints 85.71% for v1. Opened October 8, Decision Index 0.3 ranked v1.1 first at 62.8.
Strands Decider 2B scores 167 of 231 public JevBench tasks
Strands Labs posted Decider 2B on October 1, 2026, scoring 167 of 231 public JevBench tasks at a median 115 ms on an RTX 3090.
Nace posts Drex 1.5 at $0.04, and its chart shows 199 of 231 against Jev's 201
The October 9, 2026 weights card prints Drex v1.5 at Decision Index 0.3.1 public index 58.08, against Jev 1.13.0 at 57.96. Nace posted on September 30 that Drex 1.5 is live, with a 128k context window, and called it first on Decision Index 0.2.1 and on JevBench. The chart on the next post prints Jev 1.13.0 at 201 of 231 and Drex 1.5 at 199 of 231. Hard tier on that chart is 82 of 111 for both. The post's price is $0.04 per million input tokens. October 1 Benchmark Heaven text has no Drex 1. On October 6, 2026 Nace posted open weights and called them Drex 1.1. The card title is Drex DLM, and that card's Decision Index 0.2 cell is 52.31 against the kit's Jev cell of 51.67.
The Jeeves README prints 0.865 on the public JevBench hard tier
PostHog/jeeves, MIT, is a Qwen3.5-9B LoRA plus a pointer head. The README's thinking run, greedy, 2,560-token cap, prints 0.865 on 111 public JevBench hard items, against a Kev-published Jev cell of 0.730. On the transfer overall, Jev is 0.800 and Jeeves is 0.746. The sealed judge tier is not in the JevBench rows. We did not run it.
jgrep's test bench prints mean recall 0.926 across 60 commits, at $0.1112
kyu1204/jgrep, MIT, npm name jevgrep, command jgrep, is a different project from dzhng/jevgrep. A September 28 re-run of 60 commits prints mean recall 0.926, a selection ratio of 0.119, and a total cost of $0.1112. The post's 12%, 90%+, and $0.11 are those three figures rounded. The model field sent is jev-latest. We did not run it.
Maincode's Matilda card prints 59.26 on a self-run of Decision Index 0.2.1
Opened October 8, 2026, the Matilda card prints Decision Index 0.3 at 60.16, and the model API lists private false. Maincode posted Matilda on September 30, a 26.1B model under an Apache 2.0 line, and a self-run of the Decision Index 0.2.1 kit at 59.26 against the board's published Jev cell of 57.91. The card's median is 56.6 ms per request. The post's text says 56.8 ms. The run section still says the weights are private. We did not run the model.
Inception calls Mercury Decide first on OpenRouter's JevBench v1.4, and prints no score
Opened October 8, 2026, the JevBench API board ranks Mercury Decide third at composite 72.4. The free model page returned the same day, priced Free, with up to 14 decisions a second and no JevBench score. Inception posted on September 30 that Mercury Decide is the most intelligent decision model on OpenRouter, citing JevBench v1.4, at up to 14 decisions a second. The post does not print a score. The free model URL returned Not Found. OpenRouter's Mercury 2.5 page describes Mercury Decide as a System One endpoint with free output tokens and the same schema as Jev. An October 1 search of Benchmark Heaven found no Mercury Decide.
Upstage's Solar Decide chart prints 87.0% on the public 231, and a p95 of 15.7 seconds
Sung Kim posted Solar Decide on OpenRouter on September 29. The chart says the comparison is the public JevBench 231, easy 48, standard 72, hard 111. Solar Decide is 87.0% at a p50 of 0.14 seconds. Jev 1.13, measured through OpenRouter, is 86.1% at 0.30 seconds. The footnote puts p95 at 15.7 seconds against Jev at 0.40 seconds. We did not rerun the 231.
Violoop ran the same desktop harness with Jev and with Opus 5.5 on one QuickBooks bill
Violoop posted four live desktop runs on September 29. On one prepared QuickBooks bill, the same harness made 123 decisions with Jev in 127.3 seconds and with Claude Opus 5.5 in 978.9 seconds. Estimated model cost was $0.0027 against $1.38. The Tetris snapshot used a faster gravity setting for Jev. We did not rerun the tasks.
DiffusionGemma as Jev reads one denoising step, and a hand chart sums to 198 of 201
Google's Gemma account pointed at vLLM pull 57250 on September 29: seed a canvas, read yes/no, choice, and score probabilities in one denoising step. Matt Mastracci's September 17 chart adds to 198 of 201 against Jev at 191 of 201. A public 231-task replay is 196 against a published 200. On a 100-point routing file, djev-spark's joint score is 32.2% against Jev at 61.4%.
A v3 judge paper puts a frozen Jev cascade 0.9 points above GPT-6 on 1,610 pairs
Yubo Li, Yidi Miao, Ramayya Krishnan, and Rema Padman revised arXiv:2609.26550 to v3 on September 29. On 4,850 public judgments JEV is eighth of fifteen LLM judges, 1.7 points behind GPT-6, at $0.044 per 1,000 judgments. A frozen cascade is 0.9 points more accurate than GPT-6 on 1,610 held-out pairs, at 41% of its fee. We did not rerun it.
Liquid's chart puts d1 at 58.9 on an internal Decision Index 0.2.1
Liquid AI posted a chart on September 29 captioned as an internal reproduction of Decision Index 0.2.1. The index line is d1 58.9, Jev 1.13 57.9, difference +1.0. Arts, language, and retrieval are ahead on that chart. Tools and knowledge are behind. The docs name a hosted endpoint and the model id d1:free. We did not call it. On October 7, 2026 the JevBench API board ranks d1 second at composite 73.0, with capability 74.4. Opened October 8, 2026, d1 is still second at 73.0, and Jev 1.13.0 on that board is fourth at 71.5. The same day's open weights, d1-3B at 48.57, are a separate page.
OpenAI's Decisions API preview says results come back in 150 milliseconds
On October 22, 2026 OpenAI Developers posted the Decisions API generally available, and this desk's October 23 POSTs returned HTTP 200. The September 29 post had called it a limited preview. On October 6 the Developers post said public beta. The guide prices uncached gpt-6-luna input at $0.10 per million tokens and cached input at $0.01.
A 37-dataset paper scores frozen jev-1.13.0 against two open models
Tobias Deußer, Lorenz Sparrenberg, and Rafet Sifa submitted arXiv:2609.37647 on September 29, 2026. The abstract says jev-1.13.0, one frozen template per dataset, made 346,009 requests for under $10. Jev beats Qwen3.8-27B on 27 of 37 datasets and Gemma-4-E4B on all 37. This note uses the abstract. The HTML and the PDF were not opened.
Glean replayed four internal decisions on Jev, and 40 expert transfers were 8.1 times faster
Tony Gentilcore posted Glean's write-up on September 28. On 4,855 captured queries, Jev Choice cost about $0.00044 and took 0.195 seconds at p50, and tied about 37 of 41 candidates. Production order as a tie-break raised Recall@6 from 40.2% to 44.3%. A 40-entry transfer sample had a median speedup of 8.1 times. Several chart cells in the article did not come through as text. We did not rerun the sets.
One day of grant posts, 100 of them, hid 75 of 78 rejects after the questions changed
Deji labeled 100 funding posts himself, then scored Jev against that key. Seven were a real yes for the founders he serves. After he switched to full pages, an explicit 'doesn't say,' and one post at a time, the thread prints 75 of 78 nos hidden, 5 of 5 yeses shown, 6 of 6 duplicate programs caught, and deadlines on 33 of 41. The day of testing cost about 2 cents. There is no repository. We did not relabel the posts.
imajev-4b: Hugging Face weights and JevBench 70.4
Opened October 8, 2026, the JevBench v1.6.1 open-weights list puts imajev-4b at capability rank 31 and 61.9. Image JevBench v0.3.0, opened the same day, lists capability 68.0 at rank 1 and official composite 63.7 at rank 2 of 52. On October 7, 2026 the open-weights list put imajev-4b at capability rank 17 and 61.9. imajev-4b is Mohit Garg's Apache-2.0 LoRA on Qwen3.5-4B, with Hugging Face weights, JevBench v1.5.6 rank 10 at 70.4, and Image JevBench rank 2 at 76.4 on October 4. The September 27 changelog scores the same checkpoint 67.37 at rank 1, and Jev 1.13.0 stays 63.29 on that board. A September 26 adapter moved ImajevBench from 82.4% to 83.9%. We did not rerun the boards.
jevbench.app replays recorded Jev answers on labeled examples, and one test is 20 of 21
Yusuke posted jevbench.app, a playground of unit tests for questions you have already labeled. The lead-qualification set is 21 examples, 10 yes and 11 no, recorded September 27. The index says Jev matched 20 of 21, and the miss is a yes at 0.51 on a post a human marked no. Opening a test makes no new model call. This is not Florian S's JevBench. We did not rescore the file.
jevgrep solves the same 8 of 10 SWE-bench tasks, and the bill including Jev is 25.8% lower
David posted dzhng/jevgrep, MIT, a CLI that asks Jev which files matter and returns excerpts. The September 26 post said about 40% lower coding-agent cost. The September 28 file for package 0.4.3, ten tuned Python tasks, keeps 8 of 10 official solves and prints $5.6564 against a saved $7.6221, Jev included. Five of the ten tasks cost more than that saved baseline. We did not rerun it.
A 400M encoder named Kev scores 79.96% against Jev on 17,476 of its own rows
Sandipan Kundu posted skundu42/kev, a 400M encoder with Choice, Noul, and Score. On his held-out mixture Jev 1.13 scores 73.54% and this Kev scores 79.96%, ECE 0.015 against 0.098, median 132.7 ms against 388.4 ms. On a phishing set he did not retrain for, the same checkpoint flags 1,995 of 2,000 emails and accuracy is 50.25% against a published Jev figure of 62.6%. The root listing has no LICENSE file. This is not Jared Palmer's Kev. We did not rerun it.
Polylane moved five on-call decisions to Jev and prints a 39.1% lower bill per 1,000 calls
Vi Tran posted Polylane's write-up on September 28. The page, dated September 25, replaces five fixed-answer steps with jev-1.13.0. The rollup is p90 latency 4,752 ms to 508 ms, and $0.76199 to $0.46369 per 1,000 calls. The page does not print accuracy or a call count. We did not rerun the traffic.
toolgate 0.15 separates writing a script from running it, 19 of 20 on the paired set
Risk Averse Technology Company posted toolgate 0.15, MIT, a Claude Code hook and MCP proxy that asks Jev before a risky tool call. Frozen set 6, ten matched pairs, scores 19 of 20 with every pair separated and no permissive allow. A 1,369-decision week on version 0.9.1, with 455 items labeled again blind, found no permissive miss. Laptop p50 on 1,368 calls was 1,060 ms. We did not rerun the sets.
JevAdvBench's abstract says one appended opinion flips 12.1% of jev-1.13.0 decisions
Jianyi Hu and coauthors submitted arXiv:2609.31142 on September 25. The abstract describes 812 typed questions over 66 scenarios and 9,744 single-edit variants, run on jev-1.13.0. Rewording stays within 1.2 percentage points of a re-run baseline. One unverified opinion appended to the state flips 12.1% of decisions. We read the abstract and a September 28 post, not the 19 tables.
Rox's chart keeps 96% of relevance at 200 chunks, against 84% in production
Rox posted on September 28 a chart titled Jev vs. our production reranker, a mean over 75 queries. At 200 chunks the printed callouts are Jev-Score 96% and production 84%. The post, and a TypeSafe quote on September 30, say 20 times faster, 10 times cheaper, and 12% more accurate. Those multiples are not on the chart. We did not open a methods note.
A DTap chart puts a Jev 1.13 tool selector at 70.1% attack success, and a second Jev call at 36.1%
Zhaorun Chen posted a DTap chart on September 27 for Jev 1.13 used as a tool selector. Direct misuse attack success is 70.1%. A second Jev check on the filled tool call brings that to 36.1%, and indirect prompt injection from 43.5% to 21.6%. Benign success moves from 60.9% to 55.8%. The chart does not print a task count. We did not rerun it. The linked repository, opened September 30, is x402check, a pre-payment risk check. Its sample model field is jev-wallet-risk/v6.
Plumb-4B reaches 89 of 111 public hard items, and the official board scored it 65.84
Christopher House published crh225/plumb, Apache-2.0, starting from JevK5 v0.2. His runner, 231 public items on one RTX 4080 Super, puts v5 at 0.802 on the hard tier, 89 of 111, against the starting checkpoint at 0.739. The JevBench changelog for v1.4.2.1 scores the submitted model 65.84. On the v1.5 roster addendum it would place fourth at 71.6, outside the frozen ranked order. The v1.5.4 page opened September 30 numbers that same 71.6 as official rank 5, still tagged roster addendum A1. We did not rerun either table.
astra-jev-harness cuts Astra input 27% on one two-file fixture
Louis posted Oranquelui/astra-jev-harness, MIT, version 0.5.0. On a completed coding run dated September 24, two files and six checks, Astra input tokens fall from 19,198 to 14,011, and the checks stay 6/6. An older three-task note cut files from 28 to 4 while Astra input fell 1.2%. The dollar column is the maintainer's formula, one trial per arm, not a Codex invoice. We did not rerun it.
A registered attack on jev-1.13.0 moves validated success from 1.8% to 3.5%
Tiantong Wu and Wei Yang Bryan Lim, NTU, posted arXiv:2609.28613 on September 23. On 510 reconstructed InjecAgent cases, an adaptive attack with score feedback lifts validated attack success on jev-1.13.0 from 1.8% to 3.5%. The mean highest attacker-target probability found during search goes from 0.043 to 0.084. Several registered phrasings do not help the attacker. The paper has no code repository in the text we read. We did not rerun it.
jevos-1b prints 0.815 on 2,000 yes/no questions, against Jev at 0.927
Federico Elia released feder-cr/jev, a local server called jevos-1b, built with Loris Salsi. The README's image alt is the printed bench: on 2,000 yes/no questions from unseen policies, accuracy is 0.815 for jevos, 0.927 for Jev, and 0.489 for Laya. Short-prompt latency is 54 ms, 344 ms, and 104 ms. The server accepts Noul and returns 422 for Choice and Score. pyproject says MIT. The root listing has no LICENSE file. We did not rerun it.
RLCDAlignBench puts a generic Noul at median AUROC 0.886, and median ECE at 0.168
Ruoqi Guo, Yi Liu, and seven coauthors posted RLCDAlignBench on September 24. The set is 10 failure types, 44 benchmarks, and 7,193 instances. Every run in the paper we read uses jev-1.13.0. A generic Noul has median AUROC 0.886 on 31 benchmarks. Median ECE is 0.168 against a null of 0.074, so a threshold fit on one benchmark does not travel. Code is MIT. The dataset is CC BY-NC 4.0 and the Hugging Face copy is gated. We did not rerun it.
Decision Index 0.2 prints Jev at 51.67, and a later post names another first place
Opened October 8, 2026, the Decision Index space still headers v0.3, 115 models, updated 2026-10-07, and Perplexity Decider v1.1 is still first at 62.8. Opened October 7, 2026, the Decision Index space was edition 0.3, 112 models, and its full-score table ranked Perplexity Decider v1.1 first at 62.8. Jev 1.13.0 is a reference at 60.1. Rank 2 is Fastino GLiDE no-thinking at 60.2, and rank 3 is Torchcast Decision 27B at 59.9. apolinario/decision-index reproduces Decision Index 0.2, dated September 24: 40 benchmarks, chance-corrected, five equal areas. A test in the kit locks Jev at 51.67, AutoJev-27B at 50.94, Hopper at 30.01, and Verdict at 1.82. Posts the next day say Drex leads at 51.73. That 51.73 is not in the test we read. The board file generated September 28 is edition 0.2.1 and lists Jev 1.13.0 at 57.91 among 71 names, with no Drex. Nace's page prints Drex 1.5 at 58.28. Maincode's September 30 card prints a self-run of the 0.2.1 kit at 59.26 for Matilda, against this board's Jev cell of 57.91. The card says that run is being submitted, so 59.26 is not a row in the September 28 file. Nace's September 30 post also calls Drex 1.5 first on JevBench. The chart on the next post prints 201 of 231 for Jev and 199 of 231 for Drex 1.5. Liquid's September 29 chart is a separate reproduction at 58.9 against 57.9. Fastino's October 1 blog prints GLiDE at 64.81 on a self-run of the 0.2.1 scorer, against this board's Jev cell of 57.91. AutoTrust's October 3 card prints GEV-26B-Decide at 62.48 on another self-run. Neither figure is in the September 28 file. StartLux's October 3 README prints 63.88 for StartLux-Decision-27B on a self-run of the same scorer, higher than that Jev cell on 31 of 38 benchmarks, and says the run is not on the board. vLLM Semantic Router's Decision-2.0-Vega-27B card, opened October 5, prints 56.5 on its own 0.2.1 reproduction. The card does not print a Jev row, and 56.5 is not in the September 28 file. Pixilab's Blink v0.3 card, opened October 12, prints 57.48 on a self-run of the 0.2.1 kit, against the board's Jev cell of 57.91, and says the run is not a board submission.
Deel's charts show 6 of 8 quality checks at parity, and two losses
TypeSafe posted four charts on September 25 and attributed them to Deel. Six of eight quality checks are at parity or better. Messy metric questions fall 16.6 points. Three-level ticket tags fall 3.6. The live-shadow speed bar is 4 times. The tweet's 50% to 86% and 70% to 97% are not printed on the chart. We did not rerun it.
Eikos-27B prints 82.9 on its JevBench hard row, and 2.4% error once confidence is 0.90
Caio Vicentino released Eikos-4B and Eikos-27B under MIT. His harness, same items for every system: public JevBench hard tier 82.9, 72.1, Jev 73.0, Laya 35.1. At confidence 0.90 or higher, error on 7,140 items is 2.4% for both Eikos sizes and 5.9% for Jev. A later chess screenshot has Eikos at 53.9% over 388 games. A paper-trading file opened at 09:20 UTC on September 26, cycle 626 of 864, has Jev equity at $10,051.22 and Eikos at $9,653.19. Opened September 30, the model card's RuleArena row is 0.50, 0.50, and a blank Jev cell. The repo's only JSON file is a list of eleven label ids.
mizchi's fit notes record 0 illegal moves in 489 MOBA calls, and 6 to 27 wrong pages where a one-line rule was 16/16
mizchi posted a fit table for Jev on September 24. Enumerated moves: 489 MOBA decisions and 37 chess moves, 0 illegal. A one-line monitor rule is 16/16 with 0 wrong pages. Jev on that job is 6 to 27 wrong pages. Naming a code-criteria question still returns no at 0.09 to 0.22. Five agent components are 26 to 103 times faster, and none of those columns is a clear quality win. The linked criteria note, opened September 30, puts named hard bugs at 11/21 against held-out hard bugs at 3/15, 52.4% against 20.0%.
On Ámbar Reflex the 4B logit player went further than the more accurate 9B
Abel Coronado's local server plays an original run-and-gun with six typed questions and no generated text. Qwen3.5-4B at 8-bit on an RTX 3060 advances 24.9% of the level in 408 ms a decision. The same size model writing JSON advances 10.3% in 1,041 ms. Offline, Qwen3.5-9B is right on 82.7% of 554 labelled decisions and reaches 17.5% when the clock is running.
jev_local's Sarashina row beats LFM on two JGLUE tests, and the README says that is not a ranking
あるごす added Sarashina2.2 Vision 3B to a local /v1/systemone server. On JGLUE test the Q4 checkpoint is 34% on JNLI and 86% on JCommonsenseQA, against LFM at 17% and 69%. ModernBERT-Ja, trained on the JGLUE train split, is 93% and 92%. The post times the 3B model at about 100 ms when it is fast and about 500 ms as the usual case, on a Radeon AI Pro R9700.
Jevre's test split is 0.9323, and held-out sources fall to 0.8366
Di Zhang released a 0.85B checkpoint that speaks the TypeSafe Python SDK from localhost. On 12,000 evaluation rows the test split is accuracy 0.9323 and ECE 0.0079. Held-out sources are 0.8366 and ECE 0.0944. Replacing the state drops both near 0.43. The served path is 0.9267 at 77 ms median. The post calls the training RLCD. The README names Plackett-Luce ranking and a Brier term.
Jevsus ran a true/false diode on jev-1.13.0, and 45% of statements move with the wording
2nd1st published every raw answer from a September 20 run of jev-1.13.0. The post counts 3,608 true/false statements, 40 forced picks, 521 names, 190 sentence templates, 209,628 calls, and $3.13. The README says 45% of statements move by 0.20 or more across three wordings, and that eight anchors stay at 0.00 and 1.00.
Rizzo Flow is 49 ms on an RTX 5060 Ti, and six missing-evidence rows stay confident
Simone Rizzo's Apache-2.0 server reads Spark-X2.5 logits through llama.cpp and speaks POST /v1/systemone. On Windows with an RTX 5060 Ti, the 4B Q8_0 checkpoint is 49 ms at the median and uses 5.6 GiB. On SemIf's authored144 fixture the balanced accuracy is 0.812 against SemIf's published 0.819. Where the right answer is insufficient evidence, 6 of 36 rows are still confident and wrong.
Jev keeps the same direction on 28,074 classical-Korean sentences
vase.park pointed at a Jev rerun of 28,074 translated sentences in hw725/dansa-research. One call per sentence, jev-latest, O at probability 0.5 or higher. Jev's target-minus-control gaps are 38.3, 11.3, and 36.8 points. Adding Jev as a fourth judge leaves the odds ratios above 1. The note prints no gold-label accuracy.
One post scores Jev at 100 and Laya's English checkpoint at 92, on 100 items
码农暖爸 posted a 100-item English judgment set. Laya's English checkpoint is 92 correct and 8 wrong. Jev is 100 correct. The post names four scene types and says the set is not a full benchmark. The item list is a reply image.
Five runs leave a 3.3-point lead on 18 claim checks
Jamie Watters reran a 42-claim support check five times on September 22. On the hard 18, Jev's mean is 73.3% and GPT-5.4's is 70.0%. Exact McNemar on the majority labels gives p = 1.000. The September 23 page says eighteen claims cannot show the models are equal. He calls the repo JevBench. The 534-task table on this desk is a different project.
JEV Search moves NFCorpus nDCG@10 from 0.322 to 0.377
andre posted assadiandre/jev-search, a macOS app that ranks files with no saved index. On 623 queries the README's nDCG@10 goes from BM25 0.322 to JEV 0.377 on NFCorpus, and from 0.687 to 0.745 on SciFact. The repo root has no LICENSE file. The post's cost line is under $0.001 a query.
A 100,000-line needle takes 18.0 minutes with Jev and 24.7 without
El Asombroso Nosequien posted elalber2000/jev-rlm, a fork of a minimal recursive-language-model loop. On one 100,000-line file both runs returned 2867825. results.json records 1,081 seconds with 82 Jev calls and 1,480 seconds with none. The post's dollar figures are an estimate. The file has no charge field.
One Jev pass scores 1,072 translations for $0.3165
七誌 rebuilt a translation bake-off so Jev 1.13.0 scores each language once. The run note counts 16 models by 67 languages, 7,536,547 input tokens, $0.3165, and 5 minutes 23 seconds. An earlier check put this evaluator's run-to-run range at 1.12 points. The repository is CC0.
CLM-8B caches the action list, and a fine-tuned head scores 31 of 38 DeepSWE holdouts
Jacky Kwok posted CLM-8B on September 23. A frozen Qwen3-8B plus two 20M projection heads. The README says zero-shot latency is up to 9 times lower than Jev. As a verifier, 31 of 38 held-out DeepSWE tasks are correct, and Terminal-Bench 2.1 is printed as 87.6% of 30. Jev, in that setup, sits below pass@1. Apache 2.0.
jevmod scores sexual text at 0.984 AUROC, and ZeroNSFW keeps the pictures local
jevmod's sexual AUROC on OpenAI's moderation eval is 0.984. ZeroNSFW scores pictures on the host and asks Jev only about text.
Together prices Tev1 input at $0.04, and Ollama serves Choice, Score, and Noul
Opened October 7, 2026, Together's Tev1 page lists input at $0.04 per million tokens and free output. The September 23 post listed $0.042. The chat call returns one letter. Ollama serves tev1 on /v1/systemone.
ASO Workbench ranks App Store terms with a model trained on Jev's answers
Mustafa Kendigüzel posted ASO Workbench, a Mac app that estimates Apple search popularity. The method note trains on 44,658 rows. On 578 held-out terms that have autocomplete clues, Spearman rank agreement with Apple is 0.74 and the mean error is 10.4 points. Jev, through OpenRouter, supplies the brand, demand, and language-fit features. Without those answers the app does not guess.
A 612-row banking-feed test puts Jev under two Gemini rows on accuracy
Peter posted a categorization run on 612 sanitized rows from a private banking-feed set of about 33,000. Jev 1.13 scored 36.1% in 6 seconds at $0.023. Gemini 2.5 Flash scored 37.3% in 95 seconds. Gemini 3.7 Flash scored 42.5% in 78 seconds. The post names no repository and no category list.
meraGPT posts Decider 1 ahead of Jev on a teacher-labelled set
Latent Node posted Decider 1 (sd-1), a hosted model that answers noul, choice, and score questions in one call. The blog's table is accuracy 0.768 against Jev 1.13.0 at 0.727. The labels are an average of teacher samples. A September 23 thread runs the same model through a harness called jev-eval and prints one task in full: yes/no sentiment, Jev 0.970 at $0.021 per 1,000 calls, Decider 1 0.943 at $0.0078. The dataset card, read September 30, says that table is the test split, 400 cases and 2,000 decisions. The viewer's 1,600 rows count train as well.
hotdog-27B leads a 1,631-decision holdout and trails on 74 public items
Michael posted hotdog-27B, a rank-16 LoRA on Qwen3.8-27B that reads two logits and returns yes or no. On an independent holdout it is 1,603/1,631 against Jev 1.13.0 at 1,587/1,631. On 74 native-binary public JevBench items it is 60/74 against 62/74. The second interval includes zero.
A Japanese table has Jev at 96.0% on one set and 56.0% on JamC-QA
t.toda posted a three-column table. Jev is 96.0% on JCommonsenseQA, 90.2% on JMMLU, and 56.0% on JamC-QA, mean 80.8%. GPT-5.6 Luna is 97.2%, 82.5%, and 76.5%, mean 85.4%. Bonsai 2, read as a one-token logit, averages 67.2%. The image prints no item counts and no prompt.
A chart puts Jev's reading of 25 probability words next to 99 people
Mike Taylor posted a chart of 25 phrases from a 2026 survey of 99 people. The caption says Jev 1.13.0 answered one Choice per phrase, from 0 to 100 in steps of 5, and that rank agreement with the human order is 0.99. The median gap is 3 points. We read the image. The follow-up links a Claude artifact. The Hails essay, opened September 30, is that survey of 99 people. Probable is 70 in its median table. The essay has no Jev column.
rev reads a frozen Qwen and posts 0.784 on public hard items
Steven Yu posted 54yyyu/rev, an MIT harness that reads option logits from a frozen open model. On the public JevBench tiers, Qwen3.8-27B through sglang scores hard 0.784. The same README prints Jev 1.13.0 at 0.730, and 0.721 on a September 22 measurement. A laptop Qwen3.5-2B scores hard 0.550.
spark-s1-4b-v6 swaps in Qwen3.5 and prints a public-tier 83.1
Abhishek Rai posted spark-s1-4b-v6, an Apache-2.0 LoRA on Qwen3.5-4B. The README's public JevBench proxy is Intelligence 83.1, hard 0.595 of 111. On the v1.5 page opened September 29 the same checkpoint is official rank 7 at 68.2, capability 65.9. The v1.5.4 page opened September 30 keeps 68.2 and lists official rank 13. The proxy is not that composite.