Published
GitHub Next ships LocalJev, a prompted stand-in that speaks /v1/systemone
GitHub Next published LocalJev, a Bun server that turns Jev questions into JSON-probability prompts for models running on oMLX. A 1,200-request bake-off on an M5 Max compared five 4-bit checkpoints. The authors say the probabilities are self-reported, not read from logits.
GitHub Next posted LocalJev on September 19. The note is short: not everyone on the team has a Jev key yet, so they spent a morning wiring a local stand-in on oMLX. The code is at githubnext/localjev.
The server speaks POST /v1/systemone. Point TYPESAFE_BASE_URL at http://127.0.0.1:8080 and the official SDK keeps working. jev-latest and jev-preview are accepted aliases. The default upstream is oMLX at port 8000, model diffusiongemma-26B-A4B-it-4bit.
A project they cite as OpenJev reads DiffusionGemma logits in one structured step, using vLLM request fields oMLX does not expose. LocalJev turns the typed questions into a prompt, asks the model to write a JSON probability vector, retries if the JSON is broken, then normalizes. Those probabilities are generated by the model. Evaluate calibration on your own workload, the README says, before you trust them.
They published a bake-off dated September 18: 1,200 requests, about 23.5 minutes, M5 Max with 64 GiB, five installed 4-bit models, 120 gold labels (AG News, BoolQ, SST-5), two input lengths. Short-input macro accuracy they report:
- Qwen3.6-35B-A3B: 76.7% (p50 0.889 s)
- Gemma 4 26B-A4B: 75.0% (0.675 s)
- DiffusionGemma 26B-A4B: 74.2% (1.207 s)
- Gemma 4 E4B: 63.3% (0.543 s)
- Gemma 4 E2B: 45.8% (0.522 s)
Paste 2,048 unrelated words around the same target and every model’s macro accuracy drops. Sentiment takes the hit: E4B falls from 50% to 12.5%. The authors refuse a ranking among the three larger models. Differences are a couple of items in 120. DiffusionGemma, the serving default, did not win on speed for these one-question prompts.
JSON validity is not semantic validity. E2B retried 21 of 120 long requests and exhausted retries on three all-zero distributions. The other four models completed every request.
This tests the prompted pipeline, not hosted Jev, and not OpenJev’s logit read. We did not run it.
This site's reading
Editorial notes evaluating claims against primary sources, contextualizing findings alongside related implementations, and defining technical terms.
Verify
The September 19 post links githubnext/localjev. The README describes a Bun TypeScript server that implements POST /v1/systemone, talks to an OpenAI-compatible Chat Completions endpoint (default oMLX on 127.0.0.1:8000, model diffusiongemma-26B-A4B-it-4bit), and accepts jev-latest as an alias so the TypeSafe SDK can point TYPESAFE_BASE_URL at it. It translates state and typed questions into a classification prompt, asks for a JSON probability vector, retries malformed output, then normalizes. The authors contrast this with OpenJev's one-step DiffusionGemma structured read, which needs unmerged vLLM extensions they do not have on oMLX. The published bake-off is 1,200 measured requests, about 23.5 minutes, Apple M5 Max 64 GiB, oMLX 0.6.4, Bun 1.4.0: five 4-bit models, 120 gold labels, two input lengths. Short input macro accuracy: Qwen3.6-35B-A3B 76.7%, Gemma 4 26B-A4B 75.0%, DiffusionGemma 26B-A4B 74.2%, Gemma 4 E4B 63.3%, Gemma 4 E2B 45.8%. Short-input p50: 0.522 s (E2B) to 1.207 s (DiffusionGemma). The report says not to treat the outputs as calibrated probabilities. We did not run the server or the eval.
Compare
jeff is a local /v1/systemone server on GLiFormer with a 1,600-item table against live Jev. Bespoke Nimble is a 9B LoRA scored on 324 synthetic labels. Verdict is a 151M encoder in a browser tab. LocalJev is the first of these from a GitHub research group, and the first that says out loud that prompted JSON probabilities are not the same as a logit read. It does not compare those local models with hosted Jev on the same 120 labels. The 40-example-per-task cells are smaller than Kumar's 300 or Bryo's 1,565.
Terms
- LocalJev
- GitHub Next's local HTTP server that accepts TypeSafe's System One request shape and answers with a model running behind an OpenAI-compatible Chat Completions API.
- Prompted probability
- A probability vector the generative model writes as JSON, then LocalJev validates and normalizes. The README says this is not mathematically equivalent to reading logits.
- Structured read
- OpenJev's path, as LocalJev describes it: a one-step DiffusionGemma pass that needs seeded canvases, read-only denoising, and selected-token logprobs, which stock oMLX does not expose.