Mizan 4B averages 78.7 on 3,000 Persian records

Mani posted Mizan 4B on October 11, 2026, at 13:50 UTC. The thread is in Persian. It describes an open-weight decision model for Persian text, a typed answer plus a probability, and a GGUF that needs 2.5 GB of RAM. The next two posts name the 78.7% figure, Apache 2.0, and the card at xmanii/Mizan-4B. The share image is the first still on the opening post.

The card, opened the same day, says Mizan takes a record and typed questions and returns a probability for every allowed answer in one forward pass. Question types on the example are choice, score, and noul. The request body matches /v1/systemone. The card says questions can be Persian or English, including questions the model has not seen. License is Apache-2.0, the same line it gives the Qwen3.5 base. The base it names is Qwen/Qwen3.5-4B. The card text says 4B parameters. The Hub file listing says 5B params. The head is described as a Clef joint-schema head, 4 layers, width 512, trained from scratch, with LoRA merged after training.

The evaluation table is 3,000 real, human-labeled Persian records from seven public datasets. The card says none of them was in training, and that every model saw the same rows and questions.

Task	Mizan 4B	Mizan Q4	Clef-flash 9B	Clef 27B	Jev	Laya 322M
SnappFood sentiment	87.0	86.0	88.0	85.8	86.0	80.8
Digikala recommend	74.8	73.2	76.2	75.2	74.8	44.5
ParsiNLU aspect	71.5	72.2	80.0	78.5	80.0	67.8
Pars-OFF moderation	80.8	81.2	82.2	86.8	87.5	72.2
MASSIVE fa-IR, 60 intents	72.5	71.7	87.7	85.7	78.2	27.2
Digikala Mag topics	80.8	81.5	77.0	81.0	82.2	62.5
FarsTail claim	83.5	82.5	90.0	90.8	97.5	81.8
Mean	78.7	78.3	83.0	83.4	83.7	62.4

The card’s own reading of that mean is that Mizan sits within about 5 points of models it calls 2 to 7 times its size, level with them on review and feedback, and behind on assistant intents, claim checks, and per-aspect questions. Laya is the other open decision model the card names for Persian. It also says the datasets are public, so the larger models may have seen some of them in training. A confidence note says the most sure 60% of records land near 90% accuracy, and that real-data ECE is about 0.10. Holdout ECE from the temperature fit is 0.057.

Two other prints stay off that mean. English held-out on the card is BANKING77 77.0%, CLINC150 83.0%, and typed-decisions 71.7%. Synthetic Persian held-out is 91.2%. The card says that synthetic set sits close to the training distribution, and that the real-data mean is the number to quote. Cloudflare’s BANKING77 macro-F1 of 94.8, on the Clef-omni page, is a different table. Bespoke Nimble’s 292 of 324 stays the synthetic holdout.

Training on the card is one epoch, 1,004 steps, batch 16, max length 4,096, one H100, 27 minutes, about $2 on Modal. The data line is 16k decisions, about 76% English and 24% Persian. The Persian portion is described as synthetic records in 10 business workflows, labeled by a hosted decision model and a second review when that model disagreed with the intended answer. The card says the main question trains on the reviewed intended answer.

The limitations section says the Persian training text is synthetic, and that a user should score 200 to 300 of their own labeled examples before automating. It also says the vision tower from Qwen3.5 is still in the weights and was not trained for decisions. The October 11 post says image input is planned for a later update. The GGUF path asks for llama.cpp b11443 or newer and serves /v1/systemone. The Q4 file the card names is mizan-4b-Q4_K_M.gguf, 2.5 GB, mean 78.3. The Python path names Unsloth FastDecisionModel, tested with unsloth==2026.10.3. A plain Transformers path on the card returns uncalibrated probabilities. We did not run either path.