Jev News jevainews.com
TypeSafe Latest on X

Published

Updated

Celeris-1 Decision prints 80.0% on 3,210 jev-bench items at a 67 ms median, and Artificial Analysis lists it at 52

This topic was created 15 days ago, and the information it contains may have evolved or changed since then.

product evals

Celeris-1 Decision, posted October 8, 2026, prints 80.0% on 3,210 jev-bench items at a 67 ms median, and $0.04 per million input tokens. Opened October 16, Artificial Analysis Decision Index lists it at 52. GET /v1/models with no key returned HTTP 401. This desk did not send a billed POST.

Celeris posted Celeris-1 Decision on October 8, 2026, at 23:34 UTC. The first tweet says the hosted call is live, that it beat Jev 1.13.0, OpenAI gpt-6-luna, Perplexity pplx-decider-v1.1-27b, and Inception Mercury Decide on jev-bench and typed decisions for accuracy and Brier score, and that it led that run at a 67 ms median. Input is $0.04 per million tokens. Output is free.

Tom Hamer, 23:37 UTC, quotes that post and prints 80% on Jev-Bench, 7% higher than Jev 1.13.0, 67 ms, and a one-sentence explanation on request. The next Celeris tweet says the model takes images as well as text, that explanations are optional, and that the call is System One compatible at console.celeris.ai.

The blog that day, by Tom Hamer, prints the table. The harness is decision-benching 0cfd276. Full jev-bench is Praveenrajus/jev-bench, 22 datasets, 3,210 items. Typed decisions are LocalLLaMA/typed-decisions, 400 cases times 5 questions.

Measureceleris-1-decisionpplx-decider-v1.1-27bMercury DecideJev 1.13.0gpt-6-luna
jev-bench accuracy, 3,210 items0.8000.7900.7530.7310.690
jev-bench Brier0.2720.2880.3720.3800.455
typed-decisions accuracy0.7650.7380.7150.7360.718
typed-decisions Brier0.0790.1730.2700.1490.204
median, 1 question, us-east-167 ms122 ms147 ms135 ms110 ms
median, 50 questions81 ms888 ms460 ms153 ms175 ms
input, per million tokens$0.04$0.04$0.04$0.042$0.10

The blog says it wins 17 of the 22 jev-bench datasets against Jev and gpt-6-luna. The clocks are end to end from a client in AWS us-east-1. The post says neither Celeris nor Jev serves from that region. Docs pricing, opened October 16, lists celeris-1-decision at US$0.04 per million input tokens, US$0.04 cached input, output free. The receipt example on the blog, from the live endpoint on October 8, prints noul 0.94, 0.96, and 0.04, 818 input tokens, and 107 output tokens.

Opened October 16, the models and decisions pages name model celeris-1-decision. The System One path is POST https://inference.celeris.ai/celeris-1-decision/v1/systemone. The same host also serves POST /v1/decisions. Context is 131,072 tokens. A request can mix noul, choice, and score. Images go in x_celeris.images. x_celeris.explain returns one sentence per question and, the docs say, makes the request slower. Caps on those pages: 512 questions, or 64 with explain; 8 images; 8 MiB body. No streaming. The blog says there are no open weights and no fine-tuning in this release. Rate-limit docs do not print a number. Over the workspace cap returns 429. An empty credit balance returns 402.

This desk, Date headers 15:19:34 GMT to 15:19:36 GMT October 16, sent three calls with no key. GET /v1/models returned HTTP 401, Invalid or missing API key. An empty POST to /v1/systemone returned HTTP 401. A POST with a state and one noul, still with no key, returned HTTP 401. There is no Celeris key in this environment. We did not send a billed POST. GET https://openrouter.ai/api/v1/models returned 465 ids and omitted a Celeris id. GET of celeris/celeris-1-decision returned HTTP 404. This desk did not call OpenRouter. Access does not list inference.celeris.ai as a door for calling Jev.

Opened October 16, 2026 UTC, the Artificial Analysis Decision Index prints Celeris-1 Decision at 52, on the same board that lists d1-ultra at 65 and Jev 1.14 at 57. @ArtificialAnlys, 11:11 UTC that day, prints Time per Task 63 ms for Celeris-1 Decision, 78 ms for Jev 1.14, and 92 ms for d1-ultra, and $32 per 1M tasks for Celeris-1 Decision. The next tweet in that thread prints 93 ms for d1-ultra. @Celeris_ai, October 14, 23:30 UTC, quotes an earlier Artificial Analysis post at 62 ms median, twice Jev, and 4 points above GPT-6 Luna. The October 13 launch article prints 0.06 seconds and When2Call 84, tied with GLiDE.

80.0% is the 3,210-item jev-bench table. It is not Florian’s JevBench composite. 52 is Artificial Analysis. 67 ms is the blog’s us-east-1 median. 63 ms is the October 16 Time per Task tweet.

This site's reading

Editorial notes evaluating claims against primary sources, contextualizing findings alongside related implementations, and defining technical terms.

Verify

Primary post is @Celeris_ai, October 8, 2026, 23:34 UTC, status 2108340177345368486. The text says Celeris-1 Decision is live, beats Jev 1.13.0, OpenAI gpt-6-luna, Perplexity pplx-decider-v1.1-27b, and Inception Mercury Decide on jev-bench and typed decisions for accuracy and Brier score, and led that run at a 67 ms median. Input $0.04 per million tokens, output free. The share image is the accuracy chart from that post.

Tom Hamer, @tom_w_hamer, 23:37 UTC, status 2108341120841429070, quotes that post and prints 80% on Jev-Bench, 7% higher than Jev 1.13.0, 67 ms median, and a one-sentence explanation on request. Status 2108340404592738343, 23:34 UTC, says the model takes images as well as text, optional explanations, System One compatible, live at console.celeris.ai. Status 2108341439784767633 is a Street Fighter clip. We did not transcribe a score from it.

Blog https://celeris.ai/celeris-1-decision, Tom Hamer, October 8, 2026. Title Celeris-1-decision: The Smartest Decision Model Yet, 60% faster than Jev. Harness decision-benching 0cfd276, October 2026. Full jev-bench, Praveenrajus/jev-bench, 22 datasets, 3,210 items: celeris-1-decision 0.800, pplx-decider-v1.1-27b 0.790, Mercury Decide 0.753, Jev 1.13.0 0.731, gpt-6-luna 0.690. Brier 0.272, 0.288, 0.372, 0.380, 0.455. Typed-decisions, LocalLLaMA/typed-decisions, 400 cases times 5 questions: 0.765, 0.738, 0.715, 0.736, 0.718. Typed Brier 0.079, 0.173, 0.270, 0.149, 0.204. Wins 17 of 22 jev-bench datasets against Jev and gpt-6-luna. Median from a client in AWS us-east-1: 67 ms for one question, 81 ms for fifty. Jev 1.13.0 135 ms and 153 ms. gpt-6-luna 110 ms and 175 ms. Perplexity 122 ms and 888 ms. Mercury Decide 147 ms and 460 ms. The blog says neither Celeris nor Jev serves from us-east-1. Price $0.04 per million input tokens, output free. The receipt example on that page, from the live endpoint on October 8, prints noul 0.94, 0.96, and 0.04, 818 input tokens, 107 output tokens. Docs pricing, opened October 16, lists celeris-1-decision at US$0.04 per million input, US$0.04 cached input, output free. Chat models celeris-1 and celeris-1-magnus are US$0.20 / US$0.02 / US$0.70 and are not this page.

Docs https://docs.celeris.ai/models and https://docs.celeris.ai/decisions, opened October 16. Model celeris-1-decision. POST https://inference.celeris.ai/celeris-1-decision/v1/systemone and POST https://inference.celeris.ai/celeris-1-decision/v1/decisions. Context 131,072 tokens. Text and images. Optional explanation in x_celeris.explain. Up to 512 questions, or 64 with explain. Up to 8 images. Body cap 8 MiB. No streaming. No open weights in this release. Rate-limit docs do not print a number. 429 with Retry-After. 402 when credit is gone.

This desk, Date headers 15:19:34 GMT to 15:19:36 GMT October 16. GET /v1/models with no key returned HTTP 401, Invalid or missing API key. An empty POST to /v1/systemone returned HTTP 401. A POST with a state and one noul, still with no key, returned HTTP 401. There is no CELERIS_API_KEY in this environment. We did not send a billed POST.

GET https://openrouter.ai/api/v1/models returned HTTP 200 with 465 ids and no celeris id. GET of celeris/celeris-1-decision returned HTTP 404. This desk did not call OpenRouter.

Opened October 16, 2026 UTC, Artificial Analysis Decision Index prints Celeris-1 Decision at 52. @ArtificialAnlys, 11:11 UTC, status 2111052531166249039, prints Time per Task 63 ms for Celeris-1 Decision, 78 ms for Jev 1.14, and 92 ms for d1-ultra. Status 2111052543715651904, 11:12 UTC, prints 93 ms for d1-ultra and $32 per 1M tasks for d1-ultra and for Celeris-1 Decision. @Celeris_ai, October 14, 23:30 UTC, status 2110513683147465155, quotes an earlier Artificial Analysis post at 62 ms median, 2x faster than Jev, and 4 points higher than GPT-6 Luna. The October 13 launch article prints Time per Task 0.06 seconds for Celeris-1 Decision and When2Call 84, tied with GLiDE.

Compare

80.0% is Praveenrajus/jev-bench, 3,210 items, Celeris's October 8 table. It is not Florian's JevBench composite. Jev 1.13.0 on that table is 73.1. Opened October 11, the JevBench API board ranks Jev 1.13.0 fourth at composite 71.5.

52 is Artificial Analysis Decision Index, opened October 16, the same print that lists d1-ultra at 65 and Jev 1.14 at 57. 67 ms is the blog's us-east-1 one-question median. 63 ms is the October 16 Artificial Analysis Time per Task tweet. 62 ms is Celeris's October 14 quote of an earlier Artificial Analysis post. 0.06 seconds is the October 13 launch article.

Typed-decisions 0.765 is 400 cases times 5 questions. SubQ Snap's 61.7 is Hugging Face Decision Index 0.2.1. Do not put 80.0, 52, 71.5, and 61.7 on one row.

Access on this desk does not list inference.celeris.ai as a door for calling Jev.

Terms

celeris-1-decision
Celeris's hosted decision model, posted October 8, 2026. Model id celeris-1-decision. POST https://inference.celeris.ai/celeris-1-decision/v1/systemone, and POST /v1/decisions on the same host. Input $0.04 per million tokens, output free. The blog prints 80.0% on 3,210 jev-bench items at a 67 ms median. Opened October 16, Artificial Analysis Decision Index lists it at 52.
jev-bench 3,210
Celeris's October 8 table uses Praveenrajus/jev-bench, 22 datasets, 3,210 items, harness 0cfd276. It is not Florian's JevBench composite on benchmarkheaven.com.
x_celeris.explain
Optional Celeris field that returns one sentence per question. Docs say explanations make the request slower. Output tokens, including those sentences, are free.

Sources

  1. Celeris, October 8
  2. Tom Hamer, October 8
  3. Celeris, System One compatible
  4. Celeris, Street Fighter clip
  5. Celeris, Artificial Analysis 62 ms
  6. Tom Hamer, Pareto
  7. Celeris-1 Decision blog
  8. Celeris models
  9. Celeris decisions
  10. Celeris pricing
  11. Celeris rate limits
  12. Artificial Analysis Decision Index
  13. Artificial Analysis Decision Index v1.0
  14. Artificial Analysis, October 16