Jev News jevainews.com
TypeSafe Latest on X

Published

Updated

Base Labs posts mammoth-1 at Decision Index 0.4 Full 71.24; live 0.4.1 prints 68.4

product evals open-source

Base Labs posted mammoth-1 on October 21, 2026, at 17:14 UTC, and the card prints Decision Index 0.4 Full 71.24, first of 208. Opened October 23, the live space is edition 0.4.1, 229 models, mammoth-1 Full 68.4 rank 3. Date headers 02:23:21 GMT to 02:23:23 GMT, GET /v1/models with no key returned HTTP 401.

Base Labs posted mammoth-1 and elephant-1 on October 21, 2026, at 17:14 UTC. The text calls mammoth-1 the number one ranked decision model, and elephant-1 the best under 15B. It says the weights are open. The share image on this page is that post’s still.

Baseten, one minute earlier, says both models are free for one week on Model APIs, with zero data retention by default. The free week ends October 27, 2026, at 11:59 PM PT. The post says mammoth-1 is number one on the Jev Decision Index, and elephant-1 is tied for number one on the Artificial Analysis Decision Index.

The Hugging Face card, opened October 23, prints Decision Index 0.4 Full 71.24, first of 208. Perplexity Decider v1.2 is 69.42. The same table prints Jev 1.14 at 62.2, nine points back, and Cloudflare Clef at 55.7. elephant-1 is 62.65, tied eighth, best under 15B. A replica of Artificial Analysis Decision Index prints mammoth-1 65.9 and elephant-1 65.2 against d1-ultra at 65. License is MIT. The bases are Qwen3.5-27B and Qwen3.5-9B. Sequence length is 16,384 tokens. Training used 6.21 million examples. The vision encoder is frozen. Full-attention layers use a block-bidirectional mask. Self-hosting needs one CUDA GPU. This desk did not run the weights.

The technical report walks the same mix. Mix 8 is 58.3% core decision skills, 16.1% safety and abstention, 16.1% judging and expert domains, and 9.5% public decision data. A 9B ablation on an earlier mix prints causal full-attention at 44.8 and block-bidirectional at 48.6. Charlie O’Neill’s post the same minute says scaling the data 16 times added 20 index points, and cleaning it added 8.

The library page names POST https://inference.baseten.co/v1/systemone. The hosted id is decision-models/baseten-mammoth-27b-latest. elephant-1 is decision-models/baseten-elephant-9b-latest. The header is Authorization: Api-Key. Vercel’s changelog, dated October 21, lists baseten/mammoth-1 and baseten/elephant-1 free through October 27, at a 16k window.

Opened October 23, the Decision Index space headers v0.4.1, 229 models, Jev jev-1.14.0, updated 2026-10-22. Full score rank 1 is Fastino GLiDE 1.2 at 71.5. Rank 2 is Jeffan-31b-1710 at 69.8. Rank 3 is baselabs mammoth-1 (27B) at 68.4. The public cell in that row is 66.0. elephant-1 Full on that chart is 60.3. Jev 1.14 Full is 60.2. 71.24 is the card’s edition 0.4 print. It is not this 0.4.1 cell.

This desk, Date headers 02:23:21 GMT, 02:23:22 GMT, and 02:23:23 GMT October 23, sent a GET and two POSTs to https://inference.baseten.co with no key. GET /v1/models returned HTTP 401, “No Authorization header provided.” An empty POST to /v1/systemone returned HTTP 400, “Invalid request body.” A POST with model decision-models/baseten-mammoth-27b-latest, a short state, and a Noul returned HTTP 403, “please check the api-key you provided.” There is no BASETEN_API_KEY here. GET of OpenRouter ids baseten/mammoth-1 and baseten/elephant-1 returned HTTP 404. We did not send a billed POST.

71.24 is the card. 68.4 is the live 0.4.1 bar. 65.9 is the company’s Artificial Analysis replica. Those prints stay on their own rows.

This site's reading

Editorial notes evaluating claims against primary sources, contextualizing findings alongside related implementations, and defining technical terms.

Verify

Base Labs, @baselabs, October 21, 2026, 17:14 UTC, status 2112955587059450035. The text names mammoth-1 as the number one ranked decision model, and elephant-1 as the best under 15B, beating Jev. It says the weights are open, with a technical report. The share image on this page is that post's still. Baseten, @baseten, 17:13 UTC, status 2112955524207825053, says both models are free for one week on Model APIs, mammoth-1 number one on the Jev Decision Index, elephant-1 tied for number one on the Artificial Analysis Decision Index. The free week ends 10/27/26 at 11:59 PM PT.

The Hugging Face card, opened October 23, prints Decision Index 0.4 Full 71.24, first of 208, ahead of Perplexity Decider v1.2 at 69.42, Jev 1.14 by 9.0, and Cloudflare Clef by 15.5. elephant-1 is 62.65, tied 8th, best under 15B. A replica of Artificial Analysis Decision Index prints mammoth-1 65.9 and elephant-1 65.2 against d1-ultra at 65. License MIT. Base Qwen3.5-27B and Qwen3.5-9B. Max sequence 16,384 tokens. Training mix 6.21 million examples. The vision encoder is frozen. Block-bidirectional attention on full-attention layers. Self-hosting needs one CUDA GPU. This desk did not run the weights.

The Baseten library page names POST https://inference.baseten.co/v1/systemone. Model id decision-models/baseten-mammoth-27b-latest. elephant-1 is decision-models/baseten-elephant-9b-latest. Authorization header is Api-Key. Vercel changelog, October 21, lists baseten/mammoth-1 and baseten/elephant-1 free through October 27, 16k context.

Opened October 23, Hugging Face Decision Index space header: v0.4.1, 229 models, Jev jev-1.14.0, updated 2026-10-22. Full score rank 1 is Fastino GLiDE 1.2 at 71.5. Rank 2 is Jeffan-31b-1710 at 69.8. Rank 3 is baselabs mammoth-1 (27B) at 68.4. The public cell in that row is 66.0. elephant-1 Full on that chart is 60.3. Jev 1.14 Full is 60.2. 71.24 is the card's edition 0.4 print. It is not this 0.4.1 cell.

Date headers 02:23:21 GMT, 02:23:22 GMT, and 02:23:23 GMT October 23. GET https://inference.baseten.co/v1/models with no key returned HTTP 401, No Authorization header provided. An empty POST to /v1/systemone returned HTTP 400, Invalid request body. A POST with model decision-models/baseten-mammoth-27b-latest, a state, and a noul, still with no key, returned HTTP 403, please check the api-key you provided. There is no BASETEN_API_KEY in this environment. GET of OpenRouter ids baseten/mammoth-1 and baseten/elephant-1 returned HTTP 404. We did not send a billed POST.

Compare

71.24 is the Hugging Face card's Decision Index 0.4 Full score, first of 208, opened October 23. 68.4 is this desk's opening of edition 0.4.1 the same day, rank 3 of 229. Do not merge those two cells. 69.42 is Perplexity Decider v1.2 on the card's 0.4 table. GLiDE 1.2 at 71.5 and Jeffan-31b-1710 at 69.8 are 0.4.1 neighbors.

62.65 is elephant-1 on the card's 0.4 table. 60.3 is elephant-1 Full on the 0.4.1 chart. 65.9 and 65.2 are Base Labs' replica of Artificial Analysis Decision Index. Artificial Analysis's own October 22 thread prints Elephant 9B at 65, tied with d1-ultra and SubQ Snap 1.1. The replica 65.2 and the official 65 stay on separate rows.

16,384 tokens is the trained length and the Vercel 16k window. Jev's default request cap stays 64k. SubQ Snap's window stays 384K.

HTTP 401, 400, and 403 are this desk's unauthenticated GET and POSTs. They are not scored calls.

Terms

mammoth-1
Base Labs 27B decision checkpoint, MIT, from Qwen3.5-27B. Card Decision Index 0.4 Full 71.24. Live space 0.4.1 Full 68.4 rank 3, opened October 23. Hosted POST https://inference.baseten.co/v1/systemone, model decision-models/baseten-mammoth-27b-latest.
elephant-1
Base Labs 9B sibling, MIT, from Qwen3.5-9B. Card Decision Index 0.4 62.65. Live 0.4.1 Full 60.3. Artificial Analysis replica 65.2. Hosted model decision-models/baseten-elephant-9b-latest.
71.24
mammoth-1 Full score on Decision Index 0.4, first of 208, on the Hugging Face card and the technical report. The live space opened October 23 is edition 0.4.1 and prints 68.4.

Sources

  1. Base Labs, mammoth-1 and elephant-1
  2. Base Labs, block-bidirectional attention
  3. Base Labs, 6.21M examples
  4. Base Labs, weights
  5. Baseten, free week
  6. Charlie O'Neill, data loop
  7. Baseten blog
  8. Base Labs technical report
  9. Hugging Face, mammoth-1
  10. Hugging Face, elephant-1
  11. Baseten library, mammoth-1
  12. Baseten library, elephant-1
  13. Vercel changelog, October 21
  14. Decision Index space