Perplexity's pplx-decider-v1-27b scores 85.71% on a 7,210-row panel

Perplexity posted pplx-decider-v1-27b on October 1, 2026. The chart on that post is a fixed panel of 7,210 rows, dated September 2026. Overall accuracy is 85.71% for pplx-decider-v1-27b, 84.51% for TypeSafe jev-1.13.0, and 74.76% for the open-weight Qwen3.8-27B.

The Hugging Face card says the weights are Apache-2.0, a fine-tune of Qwen/Qwen3.8-27B, about 49 GiB, with a 250k context. We did not load them.

What the eleven rows say

Benchmark	Rows	Jev	Qwen3.8-27B	pplx-decider
WinoGrande	1000	90.70%	73.10%	83.30%
FinancialPhraseBank	999	76.98%	75.68%	84.18%
RAGTruth	1500	77.27%	61.53%	88.80%
JudgeBench	350	78.57%	68.86%	78.29%
BBH	750	94.27%	72.80%	82.80%
JevBench public hard	101	73.27%	72.28%	70.30%
TabFact	500	89.80%	78.60%	90.60%
Circa	500	84.60%	87.00%	89.20%
Belebele	500	95.00%	93.20%	94.00%
TruthfulQA binary	500	92.00%	82.80%	85.40%
ContractNLI	510	77.45%	80.78%	80.78%
Overall	7210	84.51%	74.76%	85.71%

Jev is ahead on WinoGrande, JudgeBench, BBH, the public-hard slice, Belebele, and TruthfulQA. pplx-decider is ahead on FinancialPhraseBank, RAGTruth, TabFact, and Circa. On ContractNLI the open base and the fine-tune print the same 80.78%, and Jev is at 77.45%. JudgeBench is close: 78.57% against 78.29%.

That public-hard row is 101 items. The Open-Jev hard tier is 111 tasks on another slice of public JevBench. The 85.71% overall is also not the September 28 Decision Index cell. That file lists Jev at 57.91 and AutoJev-27B at 56.40. This card does not say the weights are the checkpoint behind that AutoJev row.

What the hosted call costs

Opened October 1, the quickstart said the route is POST https://api.perplexity.ai/v1/decisions with model pplx-decider-v1-27b. Input on that reading was $0.04 per million tokens. Output was free. There is no per-request fee. The header it accepts is a bearer token. An x-api-key header is a 401. The org limit is 10 requests a second.

A request can hold 1 to 128 questions. A choice stops at 255 options. A score stops at 10 levels. Input has to stay under 262,144 tokens, and the body under 32 MiB. Images go in as base64 data URLs, cut into 32 by 32 tiles. The page says a request past 2,048 tiles, or about 60 seconds, returns 504.

The same page prints example latencies from a test on September 30, 2026. A few hundred tokens finished in under 2 seconds. About 90k tokens took 5 seconds, about 190k took 14, and a prompt near the cap took 23. The JSON sample next to those lines is an example payload. It is not a score on the panel.

A 4-bit timing, and a second index

On October 3 David Hendrickson wrote that a ramgpt EXL3 build of this model fits in 15.35 GB at 4.00 bits per weight. The medians he prints are 135 ms, 278 ms, and 430 ms at 128, 512, and 1,024 tokens. He says the card he is quoting has no accuracy test of that build against BF16, and that the official BF16 weights want about 49 GiB. He also restates the panel: 85.71%, 84.51%, 74.76%, and Jev still ahead on 6 of the 11. The download link was in the image alt text. We did not open it.

The same morning, a Fastino chart prints pplx-decider at 56.39 on Decision Index 0.2.1 and says Fastino ran that suite. The caption puts Jev at the public-board 57.91. That 56.39 is not a cell on the 7,210-row panel.

On October 4, 2026, at 10:10 UTC, Aravind Srinivas posted a Pokémon FireRed run. The post says the Elite Four and the Champion were cleared in one shot, with decision-making from Perplexity’s Decisions API. The stats in the post are a 592 ms median API response, a 987 ms p95, 96.4% of responses under one second, an estimated inference cost of $0.028, and 137 live API calls. The post does not say whether the median was measured on the client or on the server. It prints no win rate against another model, and it prints no Jev cell. The attachment is a video. We did not rerun the game.

v1.1, October 6

Perplexity posted pplx-decider-v1.1-27b on October 6, 2026, at 17:13 UTC. The post says the model is highest on Decision Index 0.3 and costs half as much as v1, at $0.02 per million input tokens. A follow-up the same minute says the context is 250k, the inputs are text and image, and links the weights and the docs. Aravind Srinivas wrote that Perplexity wins on the Hugging Face Decision Index and that the weights are open.

Opened October 7, the quickstart prices the Decisions API at $0.02 per million input tokens. Output is free. There is no per-request fee. The page names both pplx-decider-v1.1-27b and pplx-decider-v1-27b. The route is still POST https://api.perplexity.ai/v1/decisions. The page prices the API. It does not split the rate by model. The caps from the October 1 reading are still on the page: 10 requests a second, 1 to 128 questions, 255 choice options, 10 score levels, input under 262,144 tokens, a body under 32 MiB, images as base64, and 2,048 tiles of 32 by 32. The JSON sample on the page is an illustration. We did not send it.

The v1.1 card says the checkpoint updates v1 on the same backbone. It says a Decision Index score moves from 56.4 to 61.56, and that 61.56 is more than 3.5 points above Jev. The table on the card, in the order Jev, v1, v1.1:

Area	Jev	v1	v1.1
Knowledge	51.4	40.9	48.18
Language	62.0	63.5	69.45
Retrieval	55.4	54.9	61.26
Tools	75.1	79.3	78.88
Arts	37.7	39.4	44.66
Overall	57.9	56.4	61.56

The card says the overall is a suite weighting. The five v1.1 area scores average to 60.486. The printed overall is 61.56. It attributes the gain to lifting the causal mask and to tasksource data. The native checkpoint is a Qwen3_5Model plus readout.safetensors, BF16, shape [255, 5120]. The base is Qwen/Qwen3.8-27B. The card says about 49 GiB. A model-size line says 26B parameters. The model name says 27B. The Decision Index 0.3 row labels the same name 28B. The card says full-attention layers use a saved noncausal mode. The card we read has no license line. It says the repository is private and that access is authenticated. The October 6 post says the weights are open. A request to the Hugging Face model API from this host returned a rate limit, so this page does not add a second check of that gate. We did not load the weights.

Opened October 8, 2026, the same README frontmatter lists license: apache-2.0. The usage section still says authenticated Hugging Face access to this private repository. The model API lists private false and gated false. The safetensors total is 26,085,330,160 parameters. lastModified on that response is 2026-10-05. The score table is still 61.56 against Jev at 57.9. We did not load the weights.

Opened October 8, 2026, the Decision Index space still says edition v0.3. The header says 115 models, Jev jev-1.13.0, reproductions on one NVIDIA RTX PRO 6000, updated 2026-10-07. The lede says 114 open reproductions. The head-to-head panel still lists this model first at 62.8, with Jev as a reference at 60.1. The full-results table prints =2 on the GLiDE row, the Jev row, and the Torchcast row. The v1.1 row still prints 62.8, then 45.8, 62.3, 64.7, 80.6, and 51.4, then 28B, 100.0%, and 104 ms. We did not rerun the index.

The Decision Index space, opened October 7, 2026, headers edition v0.3, 112 models, Jev jev-1.13.0, reproductions on one NVIDIA RTX PRO 6000, updated 2026-10-06. The full-score summary lists Jev at 60.1 as a reference, Perplexity Decider v1.1 (27B) at 62.8 in rank 1, Fastino GLiDE no-thinking (28B) at 60.2 in rank 2, and Torchcast Decision 27B at 59.9 in rank 3. The GLiDE row’s button says open weights are coming soon. The model id on that row is decision-27b-ckpt-nothink. The v1.1 ranking row prints full score 62.8, then 45.8, 62.3, 64.7, 80.6, and 51.4. Those five match the area tabs in the order Knowledge, Language, Retrieval, Tools, Arts. The same row prints an empty cell, 28B, 100.0%, and 104 ms. We did not keep the column titles for the empty cell, the percent, and the milliseconds. The row labels the model a full fine-tune of Qwen/Qwen3.8-27B, autoregressive.

On the Language tab, Jev’s reference is 58.3 and v1.1 is rank 1 at 62.3. On Retrieval, Jev’s reference is 61.4 and v1.1 is rank 1 at 64.7. The Knowledge list we read starts with Quyet-1.0-Large at 47.8, then v1.1 at 45.8. Tools starts with GLiDE no-thinking at 82.7, then v1.1 at 80.6. Arts starts with GLiDE at 51.7, then v1.1 at 51.4. We did not copy a Jev reference for Knowledge, Tools, or Arts. A search of the space for “Decider v1 (27B)” and for “Drex” returned no row. The formula on the page is 0.20 times public, plus 0.50 times same-skill private, plus 0.30 times new domains. The public part is 100 times the weighted mean of the area scores, with GSM8K rebuilt and ForecastBench retired.

The card’s 61.56 against 57.9 is the table above. The space’s 62.8 against the reference 60.1 is the edition 0.3 full score. The October 1 panel’s 85.71% is a third measurement. We did not call Perplexity, and we did not rerun these tables.