Plumb-4B reaches 89 of 111 public hard items, and the official board scored it 65.84

crh225/plumb is Christopher House’s training pipeline for Plumb-4B. The NOTICE file names him and copyrights the work 2026. LICENSE is Apache-2.0. The starting weights, scripts, and runtime come from JevK5 v0.2, also Apache-2.0. The one-pass readout is the one JevK5 adapted from SemIf, which is MIT. The base model is Qwen3.5-4B. A Qwen3.8-27B teacher writes the hard questions. Weights are published as crh225/plumb-4b. Loading them wants the JevK5 runtime and about 10 GB free on a CUDA GPU. We did not load the weights.

His own runner is JevBench’s 231 public items, one RTX 4080 Super, in process, batch size 1. The first row is the starting JevK5 v0.2 checkpoint, re-run on that machine: easy 1.000, standard 0.972, hard 0.739, hard ECE 0.077, and a held-out check at 0.812. v5, the round he submits, is easy 1.000, standard 0.972, hard 0.802, hard ECE 0.094, held-out 0.846. The README prints that hard cell as 89 of 111 against the start’s 82, with 8 items fixed and 1 broken. The exact McNemar p is 0.039. Temperature is 2.07, fit on held-out teacher questions. The recipe is 5,026 teacher questions, hard-mined down to 3,204, plus a matching replay of public datasets, 3 epochs at learning rate 3e-5.

The family counts inside that hard move are uneven. Long policies go from 11 to 14, probability from 7 to 10, trade-offs from 5 to 6. Dates and numbers stay at 7 of 15. v4, before the long documents, was 0.775 on public hard and 0.051 ECE, with a held-out check of 0.844. v5 wins the held-out check and loses the hard-tier calibration comparison to v4. v1, trained on public datasets alone, dropped hard accuracy to 0.694. The README says those sets pulled the model toward short styles.

The held-out check is his, not Benchmark Heaven’s sealed set. It mixes teacher questions from domains the training run never sees, held-out splits of the public datasets, and a hand-written hard set. He picks the round on that check. An overlap scan drops training rows that share more than two 8-word sequences with a public JevBench item. An exact-text audit of 12,143 records found no training state equal to, or containing, a public one. The hits the scan did catch are generic phrases. He says no public item is used to pick a temperature.

The official composite is a second number. The changelog entry for v1.4.2.1, dated September 27, adds Plumb-4B at 65.84 on the live v1.4.2 scorer and reprices the estimated input at EmpirioLabs’ bookable Qwen3.5-4B rate, $0.04 per million tokens. An earlier $0.03 per million reference was retired. Existing v1.4.2 measurements stay. The capability view we opened on September 28 lists Plumb at 64.2, Intelligence 53.0, Calibration 75.5, an estimated $0.030 per 1,000 decisions, median 0.18 s, and official rank 2. The next changelog entry, the same day, places Imajev-4B first at 67.37. House’s README says the official score includes a private sealed set that only the maintainer runs, and his table does not print a Jev cell. Florian S posted both names on September 27 and said a larger test set was still incoming.

That larger set is v1.5.0. Plumb-4B is a roster addendum on the page we opened September 29, not an official rank. The placement against the frozen scores is fourth, at 71.6, interval 69.2 to 72.7. The capability card prints 71.6 as well, from Intelligence 55.8 and Calibration 87.4, at about $0.017 per 1,000 decisions. The 65.84 above is the v1.4.2.1 composite. We did not rerun v1.5.

The page we opened September 30 is v1.5.4. It numbers that same 71.6 as official rank 5 and still tags the row roster addendum A1. JevK5 v0.3 is rank 4 at 71.9 on that page, so the September 29 “fourth, outside the ranked order” sentence describes the earlier page. We did not rerun v1.5.4.