astra-jev-harness cuts Astra input 27% on one two-file fixture

Oranquelui/astra-jev-harness is a Python harness Louis posted on September 26. The LICENSE file is MIT, copyright 2026 Oranquelui. Version 0.5.0. It wants Python 3.10 or newer and names no third-party packages at runtime. The benchmark notes pin Codex CLI 0.153.2, gpt-6-astra, and jev-1.13.0. The September 26 post quotes his September 24 note about files. We did not rerun the arms. Each arm in the file is one trial, and the dollar column is his formula, not a bill from Codex.

The older number is a run dated September 22, three synthetic tasks. Files go from 28 to 4, which is 85.7%. Characters in the harness prompt go from 4,309 to 2,581, which is 40.1%. Astra input tokens go from 42,714 to 42,199, which is 1.2%. Output tokens stay at 175. Elapsed time goes from 18.51 seconds to 21.53 seconds. Jev’s side of that run is 4,094 input tokens and 1.62 seconds of API time. Both arms pass 22 of 22 checks. The file-count drop and the token drop are different sizes.

The completed coding run is dated September 24, source commit 9eb422976c42d66dba4f92596a43b2e89a6897c8, on a build he marks as v0.3.0. The fixture is two files: a 75-byte main.py, and 27,278 bytes of gardening prose. Six checks are held out. main.py is pinned, so the model is not being asked to remember the file. On the Extra High setting, Astra input tokens go from 19,198 to 14,011, a 27.0% cut. Output goes from 193 to 126. Jev is called twice, 6,784 input tokens and 76 output tokens. Elapsed time goes from 12.778 seconds to 11.851 seconds. His API-equivalent price goes from $0.201630 to $0.146694928. The formula in the note is (14011 × 10 + 126 × 50 + 6784 × 0.042) / 1e6. Rates he recorded that day: Astra input $10 per million tokens, cache read $1, cache write $12.50, output $50. Jev input $0.042 per million, output $0. All four Astra calls in the run report zero cache tokens. Checks stay at 6/6.

Medium, which the note says is still the distributed CLI default, cuts input from 19,204 to 14,013, also 27.0%. The dollar figure goes from $0.195190 to $0.143564928. Elapsed time goes from 9.074 seconds to 12.217 seconds.

A selection-only pair, v0.2.0 against a v0.3.0 candidate, keeps 75 bytes where the older build kept 27,353. That is a 99.73% cut in retained bytes. Jev input on that pair rises from 392 tokens to 6,784, and the estimated Jev price rises from $0.000016464 to $0.000284928. Astra’s price for that pair is not in the file. Auto mode uses a threshold of 12,000 source bytes and makes no Jev call on the 75-byte file.

Version 0.5.0 adds a progress-log smoke. Stdout falls from 19,885 bytes to 7,601, which is 61.8%, with 5 of 5 required facts still present. Jev is two calls, 7,172 and 195 tokens. The note says this smoke is not a measurement of Astra tokens or of task time. A later context-only pass retains all 4 files it was given. Replaying a batch against a per-file pass moves 34,852 bytes to 34,634, which is 0.63%, and 15 files to 14, with no new API calls. The README says the full offline suite is 141 tests. The benchmarks note says 119 after a follow-up, and 114 at the selection stage. A Claude Code 2.1.281 check on September 24 uses --mode local, makes zero Jev calls, and is not a savings run.