Vega, the vegaml package, scores 75.4% on 800 phishing emails against Jev at 61.9%

On October 10, 2026 at 08:25 UTC, @Nandakishorm1 posted Vega. The attachment is a video. The README that the post’s GitHub link opens, NandhaKishorM/vegaml, says Apache 2.0. An earlier post the same morning links that repo and a Hugging Face space. We did not open the space, and we did not open the shortened blog URL. We did not run the weights.

Decision 2.0’s Vega is a different checkpoint. That card, Decision-2.0-Vega-27B, prints 56.5 on its own Decision Index run. This package is the Laya author’s, default 0.8B, with a 4B next to it.

load() with no argument returns the 0.8B. The frozen backbone is Qwen3.5-0.8B. The trained engine is 14.3 million parameters, 57 MB. The 4B uses frozen Qwen3.5-4B and an engine of 30.9 million parameters, 124 MB. Context is 73,728 tokens. The README says a state of at least 4,096 tokens is encoded once and reused across questions, and that an input which does not fit is refused. decide_image swaps a picture in for the state text. The printed benchmark figures are text. The image path is described as working and is not part of those figures.

Answer types are choice, score, and boolean. Score is an integer level on ordered criteria. Boolean is the probability that a statement holds. The README says version 0.5.0 renamed that yes/no type from a coined word, and it does not print the old word. A question that still uses the old name is rejected. Older checkpoints store thresholds under the old name, and the loader remaps them.

The default mode is "engine", zero-shot. The published figures use that mode. "ttt" needs at least six labeled examples, and the README says that path answers even when it should not. fit returns a cross-validated accuracy per question.

The paired table is live Jev 1.13.0 on the same items, with no tuning on the evaluation data. The README says the table is the rows where this model leads, and that the reference leads on most other areas, in particular reranking and multi-step reasoning.

Task	Vega	Jev 1.13.0
Phishing, 800 emails	75.4 accuracy, 252/400 caught, recall 0.63	61.9, 99/400, recall 0.25
Dates and quantities	46.7	20.0
enron-spam	1.000	0.920
ag-news	0.955	0.806
Typed scores	0.438	0.395
Calibration error, product relevance	6.0	22.0
Calibration error, 5,096 paired	9.5	9.3
Median latency	267 ms on a T4, fp16	591 ms, hosted

The README prints McNemar p = 3e-11 on the phishing catch counts, 252 against 99. Overall calibration error on the 5,096 paired decisions is the row Vega does not lead, 9.5 against 9.3.

A second table is both sizes on LocalLLaMA/typed-decisions, 2,050 decisions, each with its task adapter. The 0.8B scores 0.763, soft accuracy 0.680, calibration error 0.026, 22 ms. The 4B scores 0.803, 0.725, 0.019, 28 ms. With the adapter the 4B leads five of six workflows: apibank 96.5 against 89.5, when2call 94.6 against 90.4, appliances 89.1 against 85.1, banking77 74.9 against 71.4, esci 64.4 against 56.4. The 0.8B leads clinc150, 88.0 against 86.7. The 0.763 cell is this adapter run. The 0.438 cell is the typed-scores row in the paired table above. The 22 ms line is this size table. The 267 ms line is the paired latency.

Images are two more tests. On RVL-CDIP-N, 1,002 document pages, Vega 0.8B zero-shot reads the image and scores 0.793, ECE 0.207, median 1,989 ms. Apple Vision OCR plus Jev 1.13.0 reads the text and scores 0.896, ECE 0.045, median 1,049 ms. The README assigns 649 ms of that median to Apple Vision and 400 ms to the API call. DiT, trained on RVL-CDIP, scores 0.786. The README says Vega chose among the 12 categories present in the file, while a classifier trained on the original set chooses among 16. On MMMU-Pro, 300 items, chance 0.120, Vega scores 0.240 with ECE 0.052. The same questions with the image withheld score 0.157. Jev 1.13.0, text only because the hosted model takes no image, scores 0.337 with ECE 0.160. The card’s best-reported Gemini 3.1 Pro line is 0.839.

The README describes the engine as a particle in a 64-dimensional space. Option text is inside the prompt. One forward pass of the frozen language model pools the situation, the question, each option, and the last token. Those vectors set the particle and a well per option. A fixed number of damped steps lands on a well, and a temperature read from the leftover motion scales the probabilities. The language-model head does not run. The trained piece the README emphasizes is the 14.3M engine, plus rank-32 adapters and a per-question gate.

A laptop smoke test of the adapter, 96 rows, moved accuracy from 0.313 to 0.542 on 48 held-out items. The README calls that a smoke test. The full 16k run is the one it points at Kaggle for, and this page did not see a printed score for that run.

Laya, from the same author, stays the JevBench 70.1 page. This README does not print a JevBench composite.