Published
LlamaIndex's Logan Markewich ships jeff, a local System One API on GLiFormer
Logan Markewich released jeff, a self-hosted drop-in for TypeSafe's /v1/systemone endpoint backed by Knowledgator's GLiFormer. On 1,600 public items it trails Jev, especially on irony and reading comprehension, and comes in cheaper to serve on a GPU.
Logan Markewich, head of open source at LlamaIndex, posted jeff on September 19. The code is at logan-markewich/jeff.
The server implements POST /v1/systemone with the three TypeSafe question types. Point TYPESAFE_BASE_URL at it and the official SDK keeps working. Model aliases include jev-latest. Markewich wrote that Jev’s API is simple enough to copy, and that the model itself is a classifier, “just like GliFormer,” with similar latency, lower cost, and a mild hit in accuracy.
The README’s comparison, measured on 1,600 labeled items across eight public datasets:
- Sequential p50 from a laptop: 151 ms for jeff on an L4 over HTTP, 129 ms for Jev
- Cost per million single-question requests: about $2.6 versus about $15.6
- AG News topic accuracy: 75.5% versus 90.5%
The longer RESULTS.md says jeff is close on binary sentiment, tied on six-way emotion (both near 47%), and well behind on irony and BoolQ. Averaged across tasks, Jev leads on choice accuracy (0.69 vs 0.61), score MAE (0.48 vs 0.61), and noul AUROC (0.975 vs 0.84). Jev’s answers do not move when extra questions are packed into the same request. jeff’s nouls stay put only if each one gets its own encoder pass, which is the default.
A probe of 24 unambiguous yes/no items found that sharing one prompt could swing a noul by as much as 0.98. Isolation removes that spread and costs extra tokens. Probabilities are temperature-scaled at 3.2 because the raw encoder is over-confident; score still uses the untempered distribution.
The README recommends deploying the Modal GPU script on an L4 for the HTTP API. Measured throughput behind Modal’s web ingress caps at about 50 requests per second per container. An 8-core CPU container was slower and more expensive than calling Jev. On a Mac, the notes say to use MPS, not the ONNX path.
The Hugging Face card for knowledgator/gliformer-large-v1 lists 575.6 million parameters. The jeff README still says 400M. We did not train or host the model.
This site's reading
Editorial notes evaluating claims against primary sources, contextualizing findings alongside related implementations, and defining technical terms.
Verify
The September 19 post and the repository README agree: jeff serves POST /v1/systemone with choice, score, and noul, and the official typesafe-sdk can point TYPESAFE_BASE_URL at it. The README's headline table is 1,600 labeled items across eight datasets: sequential p50 from a laptop 151 ms on an L4 versus Jev at 129 ms; about $2.6 versus about $15.6 per million single-question requests; AG News 75.5% versus 90.5%. RESULTS.md adds averages of choice accuracy 0.61 versus 0.69, score MAE 0.61 versus 0.48, noul AUROC 0.84 versus 0.975. The README calls GLiFormer 400M parameters; the Hugging Face card for gliformer-large-v1 lists 575.6 million. We did not run the server or the eval.
Compare
Bespoke Nimble is a 9B generative LoRA scored on 324 synthetic labels. Verdict is a 151M encoder in a browser tab on 337 TypeSafe public cases. jeff is the first of the three that copies the TypeSafe HTTP API, so an existing SDK client can swap the base URL. It is also the first with a public multi-dataset comparison against live Jev. Isolation of each noul is the default because sharing one prompt moved answers by as much as 0.98 in a probe; that sits next to TypeSafe's warning that jev-1.13 does not guarantee identities across separate questions.
Terms
- jeff
- A self-hosted server that speaks TypeSafe's System One API and answers with Knowledgator's GLiFormer instead of Jev.
- GLiFormer
- Knowledgator's encoder with task heads at inference time. jeff loads knowledgator/gliformer-large-v1.
- Question isolation
- Running a separate encoder pass per noul so other questions in the same request cannot change that probability. jeff defaults to isolating nouls.