Jev News jevainews.com
TypeSafe Latest on X

Updated

Published

jev-gateway asks Jev which tool a coding agent should call

open-source product

Vini Lana posted jev-gateway, an MIT local proxy for Codex, Claude Code, OpenCode, and Gemini CLI. Jev picks the tool. Closed arguments can skip the language model. On the project's chess-engine bench, GPT-6 Astra and GPT-5.6 Sol cut bug-fix output tokens by 57%. The feature task made Opus 5 and Sonnet 5 use more.

Vini Lana posted jev-gateway on September 22 and said it was a way to spend up to 57% less while using Codex or Claude Code. The repository link is in a reply: vinilana/jev-gateway. The package is MIT, on npm as jev-gateway, and it wants Node.js 22.15 or newer. The README calls it an independent client of TypeSafe’s public API.

The launchers are jev-codex, jev-claude, jev-opencode, and jev-gemini. Each one starts a process on 127.0.0.1 (ports 8790, 8789, 8791, and 8788) and then runs the agent you already have logged in. Plain codex and claude are unchanged. Jev itself can be TypeSafe (jev-latest), OpenRouter (typesafe/jev-1.13), or Vercel AI Gateway (typesafe-ai/jev). The README says the TypeSafe path has been run against the live API. The OpenRouter and Vercel paths follow those vendors’ docs and have tests, and they have not been run with real keys.

When a request includes tools, the gateway asks Jev which tool fits, whether any tool is needed, and what the closed arguments should be. Closed means an enum, a boolean, or a constant. Open text, numbers, and dates stay with the language model. The response header x-jev-gateway-mode is one of five:

  • direct: Jev is confident and every argument is closed. The gateway builds the tool call. There is no language-model request.
  • forced: the tool is chosen and some arguments are open, so the model only fills those in. tool_choice is pinned to that tool.
  • hint: a one-line suggestion is appended. This is the Claude Code path, because extended thinking rejects a forced tool and a changed tool_choice would drop the prompt cache.
  • none: Jev is confident that no tool is needed.
  • passthrough: confidence is under JEV_MIN_CONFIDENCE (default 0.7), the two checks disagree, Jev errors, there are no tools, or the caller already picked. The bytes go through unchanged.

Every argument in a direct answer also has to clear JEV_ARG_MIN_CERTAINTY, default 0.8. The Jev timeout is 4,000 ms. If Jev is down, slow, or the key is wrong, the request goes to the language model. A tool list longer than 120 names takes two Jev calls. The README says Claude Code sends about 280. Jev picks one tool per turn. In forced mode the model can still call that same tool several times. It cannot mix two different tools in one turn.

The 57% figure is in the README’s benchmark section, which points at vinilana/jev-gateway-bench. The write-up says 120 agent sessions: six models, two tasks on a chess rules engine (fix bugs, add a feature), five runs with routing and five without, no MCP servers and no plugins. Each line is the median change in output tokens, input tokens, and time against the same model with routing off.

  • GPT-6 Astra (Codex): bug-fix -57% / -7% / -39%. Feature 0% / +2% / +8%.
  • GPT-5.6 Sol (Codex): bug-fix -57% / -40% / -36%. Feature -9% / -39% / -16%.
  • GPT-5.6 Luna (Codex): bug-fix -12% / -10% / +10%. Feature -14% / -51% / -14%.
  • Fable 5.1 (Claude Code): bug-fix -13% / -19% / +6%. Feature -24% / -27% / -26%.
  • Opus 5 (Claude Code): bug-fix -7% / -22% / +2%. Feature +22% / +61% / +83%.
  • Sonnet 5 (Claude Code): bug-fix -41% / -48% / -25%. Feature +9% / +16% / +37%.

The tweet’s 57% is the bug-fix output figure for Astra and for Sol. The feature task is a different result. Opus 5 used more output, more input, and more time. Sonnet 5 used more of all three as well, after a cheaper bug-fix. Luna’s feature task was cheaper and solved 3 of 5 runs, against 5 of 5 with routing off. The README says five runs per cell is a small sample, and that the bench repository records one run caught copying from another. Claude Code’s path is hint, so the Claude lines are suggestion text. The language model still runs.

The “Jev answered in 712 ms” line is the sample transcript for the first-run key check, not a row in the bench.

We did not install the gateway or repeat the chess sessions.

This site's reading

Editorial notes evaluating claims against primary sources, contextualizing findings alongside related implementations, and defining technical terms.

Verify

Primary post is @oviniciuslana on September 22. The GitHub URL is in the reply https://x.com/oviniciuslana/status/2102229817277952289. Repo vinilana/jev-gateway, MIT, npm package jev-gateway, Node.js 22.15 or newer. Launchers jev-codex, jev-claude, jev-opencode, and jev-gemini listen on 127.0.0.1, ports 8790, 8789, 8791, and 8788. Jev providers are TypeSafe (jev-latest), OpenRouter (typesafe/jev-1.13), and Vercel AI Gateway (typesafe-ai/jev). The README says the TypeSafe path has been run against the real API. OpenRouter and Vercel follow published endpoints, have tests, and have not been run with real keys. A tool-bearing request triggers one Jev call, or two when the tool list exceeds 120 names. Claude Code is described as sending about 280. Modes, header x-jev-gateway-mode: direct (tool and every argument closed, no LLM call), forced (tool chosen, open arguments filled by the LLM), hint (suggestion text, used when Anthropic thinking is on or a cached conversation cannot change tool_choice), none (tool_choice none), passthrough (low confidence, the two checks disagree, Jev failed, no tools, or the caller already chose). Defaults: JEV_MIN_CONFIDENCE 0.7, JEV_ARG_MIN_CERTAINTY 0.8, JEV_TIMEOUT_MS 4000. Jev down, slow, or a bad key fails open to the LLM. Claude Code stays in hint mode, so the README does not promise a lower bill there. Jev picks one tool per turn. Bench repo vinilana/jev-gateway-bench, cited from the README: 120 agent sessions, six models, two chess-engine tasks (fix bugs, add a feature), five runs per mode, no MCP servers or plugins. Medians with routing on versus the same model with routing off, output tokens / input tokens / time. Bug-fix: Astra -57% / -7% / -39%; Sol -57% / -40% / -36%; Luna -12% / -10% / +10%; Fable 5.1 -13% / -19% / +6%; Opus 5 -7% / -22% / +2%; Sonnet 5 -41% / -48% / -25%. Feature: Astra 0% / +2% / +8%; Sol -9% / -39% / -16%; Luna -14% / -51% / -14%; Fable 5.1 -24% / -27% / -26%; Opus 5 +22% / +61% / +83%; Sonnet 5 +9% / +16% / +37%. Luna's feature task solved 3 of 5 runs with routing and 5 of 5 without. The README says five runs per cell is a small sample, and that the bench repo records one run caught copying from another. The tweet's "até 57%" matches the Astra and Sol bug-fix output column. The sample "Jev answered in 712 ms" is the first-run transcript in the README, not the bench. We did not install the gateway or rerun the sessions. TypeSafe's Master Customer Agreement section 2.3(f) forbids publishing benchmarks of the Services; the figures are reported as published.

Compare

Vercel's form router keeps a destination at confidence 0.95 and otherwise asks Luna Fast, on one form submission. LiteLLM can drop old tool results below 0.2 before the generator runs. jev-gateway sits on the agent's HTTP port and can skip the language model entirely when every argument is an enum, a boolean, or a constant. Claude Code does not get that skip. The chess bench is token and time medians on two tasks, five runs each, not a JevBench score.

Terms

direct mode
A jev-gateway outcome. Jev is confident about the tool, and every argument is an enum, a boolean, or a constant, so the gateway writes the tool call and does not call the language model.
hint mode
The jev-gateway path for Claude Code. A short suggestion is added to the request because a forced tool_choice would break extended thinking or the prompt cache. The model can ignore it.

Sources

  1. Vini Lana, introducing jev-gateway
  2. Vini Lana, GitHub link
  3. vinilana/jev-gateway
  4. vinilana/jev-gateway-bench