Published
Updated
PurpleMIST-Flash-1.0 scores 0.668 on 15,050 translated African-language decisions
This topic was created 21 days ago, and the information it contains may have evolved or changed since then.
PurpleMIST-Flash-1.0 scores 0.668 accuracy on 15,050 translated African-language decisions, against DeepSeek-V4-Pro at 0.669. KL on that set is 0.214 against 1.125. The card says the checkpoint is Apache-2.0, about 7.9B parameters, from Qwen3.5-9B-Base, and text only. A self-run of Decision Index edition 0.3 prints a public index of 46.7. We did not run the weights.
On October 10, 2026 at 10:56 UTC, @olaverse_ posted PurpleMIST-Flash-1.0. The attachment is a video. A reply in the thread says about 60 ms per decision on one GPU, Apache-2.0, and links the model card. The card, opened the same day, lists license apache-2.0. The post calls the model 9B. The card says about 7.9B parameters, and the safetensors count is 7,936,684,544. We did not run the weights.
The card calls it a decision model. A state goes in with any number of typed questions, and one forward pass returns a probability for each answer. Types are choice, noul, and score. The backbone is the text model of Qwen3.5-9B-Base, with the vision tower and the language-model head removed. The card counts about 7.9B parameters, 32 layers, hidden size 4096, and a pointer head of 8.4M. LoRA rank 32 is merged into the weights. The file is bf16, 15.9 GB. Context is 65,536 tokens. serve.py answers POST /v1/systemone. The card says the answer fields match Ollama’s decision API, and that PurpleMIST is not in the Ollama library yet.
The lead table is African Typed Decisions, eight languages, one harness for every model. The translated set is 3,010 cases and 15,050 decisions, the 400 LocalLLaMA/typed-decisions test cases carried into those languages. PurpleMIST-Flash-1.0 scores 0.668 accuracy. DeepSeek-V4-Pro, listed at 1.6T parameters and prompted for JSON, scores 0.669. KL on that set is 0.214 against 1.125. Brier is 0.115 against 0.199. ECE is 0.049 against 0.121. English on the original cases is 0.720 for both, so the drop is 0.053 and 0.051. OpenDecider-small drops 0.171, from 0.648 to 0.477.
| Model | Size | Translated accuracy | KL |
|---|---|---|---|
| DeepSeek-V4-Pro, prompted | 1.6T | 0.669 | 1.125 |
| PurpleMIST-Flash-1.0 | 7.9B | 0.668 | 0.214 |
| Qwen3.8-Flash, prompted | undisclosed | 0.643 | 0.862 |
| OpenDecider-small | 4B | 0.477 | 0.365 |
| PurpleMIST-Mini-1.0 | 1.9B | 0.416 | 0.399 |
| laya-multilingual | 0.32B | 0.322 | 5.053 |
By language, PurpleMIST leads Pidgin at 0.710, Somali at 0.683, Swahili at 0.690, and isiZulu at 0.671. Amharic is 0.649 against DeepSeek’s 0.650, and Yorùbá is 0.644 against 0.651. Hausa trails by 0.026 and Igbo by 0.019. The card marks the native-language column as partly scored against the models that wrote those cases. It also says PurpleMIST trained on native cases from that pipeline, and that those training cases are not the benchmark cases.
LocalLLaMA/typed-decisions is a different table. The test split is 400 cases and 2,000 decisions. PurpleMIST scores 0.720 accuracy, KL 0.189, Brier 0.098, ECE 0.056. The card copies a published Jev 1.13.0 row of 0.727, KL 1.442, Brier 0.148, ECE 0.144. meraGPT Decider 1 leads that copied list at 0.768. The card says PurpleMIST did not train on that benchmark’s train split. A by-type note says yes/no accuracy is 0.822, and that the evaluation script’s overall is 0.717. That 0.717 is not the 0.720 in the table, and neither number is the African 0.668.
The card also prints a self-run of Decision Index edition 0.3, the maintainers’ kit, on one RTX PRO 6000. Public index 46.7. It says all 140,620 requests were answered, none truncated, and the Full score, which adds private tests, is still pending. Among 13 models it places in the 7B to 10.5B band, it says PurpleMIST is ahead of 8, and 4.8 points above their median of 41.9. The peer table prints Bespoke Nimble 9B v3 at 57.2 and Cloudflare clef-flash at 56.1. Arts and taste is the weak column, 25.5. Speed on this run is median 63 ms and mean 94 ms, self-measured. The reply’s about 60 ms is the other sentence. 46.7 is not the African 0.668, and Nimble’s 57.2 on this table is not the 292 of 324 holdout.
The training note says 300,000 states were sampled from 1.71 million decisions, including every African state. African decisions in that mix are 86,134. The run is 1,501 steps at learning rate 1.5e-4 on 4 H100s. Temperatures fitted afterward, on 2,751 states the training did not use, are choice 0.938, noul 1.035, and score 0.907. The card says corpora distilled from the Jev API were left out.
A second script, reading the training-data format, prints African accuracy 0.674 rather than 0.668. The same script’s other held-out lines include Open-Jev test 0.787 on 6,972 decisions and Open-Jev out-of-distribution 0.767 on 8,325. Those lines are not the harness table.
The card’s limits are specific. Reference labels come from language-model teachers. The African text was machine-written and machine-checked, not reviewed by native speakers. Images in the state are not read. The English demo on the card is a lane-runner game of 60 moves, 0 crashes, and 24 coins. That clip is not one of the tables.
PurpleMIST-Mini-1.0 is the 1.9B row in these tables: translated accuracy 0.416, English typed-decisions 0.601. We did not open that card.
This site's reading
Editorial notes evaluating claims against primary sources, contextualizing findings alongside related implementations, and defining technical terms.
Verify
@olaverse_, October 10, 2026, 10:56 UTC, status 2108874321477120394. The text calls PurpleMIST-Flash-1.0 a 9B open decision model for African languages and English, and says one forward pass returns a calibrated probability for every answer. The attachment is a video of 54.3 seconds. A reply in the same thread, status 2108874679821697346, says about 60 ms per decision on one GPU, above the median of 7B to 10B models on the Jev Decision Index, Apache-2.0, and links the model card.
Card https://huggingface.co/olaverse/PurpleMIST-Flash-1.0, opened October 10. Hugging Face API cardData.license is apache-2.0. base_model is Qwen/Qwen3.5-9B-Base. safetensors total is 7936684544 BF16 parameters. createdAt is 2026-10-07T23:47:03.000Z. The card text says about 7.9B parameters, 32 layers, hidden size 4096, a pointer head of 8.4M parameters, LoRA rank 32 merged into the weights, and bf16 weights of 15.9 GB. Context is 65,536 tokens. Question types are choice, noul, and score. serve.py exposes POST /v1/systemone. The card says PurpleMIST is not in the Ollama library yet. We did not run serve.py.
African Typed Decisions table on that card. Translated set: 3,010 cases, 15,050 decisions. PurpleMIST-Flash-1.0 accuracy 0.668, KL 0.214, Brier 0.115, ECE 0.049. DeepSeek-V4-Pro 0.669, KL 1.125, Brier 0.199, ECE 0.121. English original is 0.720 for both of those two, and the drop to the translated set is 0.053 and 0.051. PurpleMIST leads Pidgin 0.710, Somali 0.683, Swahili 0.690, and isiZulu 0.671. It trails Hausa by 0.026 and Igbo by 0.019. The native-language column is marked as partly scored against the teachers that wrote it. The card says PurpleMIST trained on separate native cases from the same pipeline.
LocalLLaMA/typed-decisions, a different table on the same card. Test split 400 cases, 2,000 decisions. PurpleMIST 0.720 accuracy, KL 0.189, Brier 0.098, ECE 0.056. The card's published Jev 1.13.0 row is 0.727, KL 1.442, Brier 0.148. A by-type note says the script's overall for PurpleMIST is 0.717, and yes/no is 0.822. We did not rerun either table.
Jev Decision Index, edition 0.3 kit, one RTX PRO 6000, serve.py with max_len 65536. The card says all 140,620 requests were answered and none were truncated. Public index 46.7. It says this is ahead of 8 of 13 models in the 7B to 10.5B band, 4.8 points above their median of 41.9, and that the Full score is still pending review. Bespoke Nimble 9B v3 is 57.2 and Cloudflare clef-flash is 56.1 on that printed peer table. Speed on this run, self-measured: median 63 ms, mean 94 ms. The reply's about 60 ms is a separate sentence from that 63 ms.
Limitations printed on the card: reference answers come from language-model teachers, the African text was machine-written and machine-checked, and images in the state are not read. We did not run the weights.
Compare
0.668 is accuracy on the translated African set, 15,050 decisions. 0.720 is LocalLLaMA/typed-decisions, 2,000 decisions, and the card's Jev cell there is 0.727. 46.7 is a self-run of Decision Index edition 0.3, public suite only, and the card says the Full score is pending. Nimble's 57.2 on that peer table is not the 292 of 324 holdout, and it is not Decision Index 0.2.1 at 56.88.
The training script's African accuracy is 0.674, which the card says comes from reading questions in the training-data format. The harness figure used above is 0.668. Open-Jev test 0.787 on 6,972 decisions is another held-out set in that script.
PurpleMIST-Mini-1.0, 1.9B, is a row in the Flash card's tables: translated accuracy 0.416 and English typed-decisions 0.601. We did not open the Mini card. The reply's about 60 ms and the index run's median 63 ms are both self-reported, on different sentences.
Terms
- PurpleMIST-Flash-1.0
- Olaverse's Apache-2.0 decision model, about 7.9B parameters, from Qwen3.5-9B-Base. On the card's African Typed Decisions harness, translated accuracy is 0.668 on 15,050 decisions. Text only.
- African Typed Decisions
- olaverse/african-typed-decisions. The translated comparison is 3,010 cases and 15,050 decisions in eight languages. KL, Brier, and ECE on the card are for that translated set.
- PurpleMIST public index
- 46.7 on a self-run of Decision Index edition 0.3, public suite, 140,620 requests, not yet the board's Full score. Separate from the 0.668 African accuracy.