Published
Updated
A Pi extension asks Jev which skills to load, and the 20-skill demo drops about 1,000 tokens
This topic was created 31 days ago, and the information it contains may have evolved or changed since then.
Arnav Gupta posted a Pi skill picker on September 29. The MIT repo, pi-skill-picker-jev 0.1.0, asks a decision model which skill names belong in the prompt. The README's 20-skill storefront demo cuts the initial system prompt by about 1,000 tokens. The post's 250-skill work repo, about 10,000 tokens and under a second, is not in the README. We did not boot Pi. On September 30, 2026 we scored the 20 published skill names, and a 250-name catalog of those names plus 230 fillers written here. Necessary-skill recall was 16 of 19 on both catalogs. Five of eight tasks had every required skill selected. Median latency was 951 ms and 2,322 ms. The scoring note, opened September 30, uses a five-level Score and a default threshold of 0.625.
Arnav Gupta posted a Pi extension on September 29 that asks Jev which skills the current task needs. Pi then sees those skills, and loads a skill’s full instructions only when it uses them. The repository is championswimmer/pi-skill-picker-jev, MIT, package 0.1.0. It wants Pi 0.87.1 or newer and an OpenRouter key, or another decision server. A reply in the thread says an M3 or newer Pro or Max can run Kev locally instead.
The README’s measured demo is a storefront project with 20 skills. Selecting skills for the request, instead of sending the catalog, cuts the initial system prompt by about 1,000 tokens. The screenshot shows five skills added on one turn, with ratings 0.760, 0.555, 0.468, 0.345, and 0.345, next to a 0.300 line. The post adds two figures that the README does not: a repository at work with more than 250 skills, where the same cut is about 10,000 tokens, and a note that batched scoring of that catalog stays under a second. The image on the post is the README page, including the install line pi install git:github.com/championswimmer/pi-skill-picker-jev.
The picker runs on each new prompt and, by default, again after tool text arrives. Skills it has already picked stay for the session. If the decision service has no credentials or does not answer, the extension does not fall back to the full catalog. Already selected skills, and skills marked always allowed for the project, remain available. The privacy note says the default call sends the request, recent conversation, tool text, and the skill names and descriptions. It does not send the skill files. A scoring-research note is linked from the README.
On September 30, 2026 this desk scored those 20 demo names, and a 250-name catalog made of the same names plus 230 fillers written here, with jev-1.13.0. Pi was not started. The gate was the extension’s: raw score divided by 4 at least 0.625, at most six new skills, batches of 50. Eight tasks had required skills named in advance. Necessary-skill recall was 16 of 19 on both catalogs. Five of eight tasks had every required skill selected. Median wall time was 951 ms on 20 names and 2,322 ms on 250. The 250-name run did not stay under a second. Gross skill-body text on the 20-name demo is about 698 tokens by a four-characters-per-token estimate. Mean Jev input on that catalog was 2,834 tokens, so the call cost more tokens than the bodies it held back. List price for the eight 20-name tasks was $0.000952, and for the eight 250-name tasks $0.010821, at $0.042 per million input tokens. Tax-and-shipping missed shipping-policy and selected pricing-rules. Privacy missed release-checklist. API pagination missed catalog-data and selected observability. A connection refused from 127.0.0.1 port 9 left the selected set empty and did not load the catalog. A bad key returned HTTP 401. The private 250-skill work repository was not in the clone.
On September 30, 2026 we opened docs/scoring-research.md. Production asks Score on five levels, 0 through 4: Unrelated, topically adjacent, optional support, directly useful, and Essential. The default threshold is 0.625, a raw score of 2.5. The note calls that a starting point taken from the rubric.
On six synthetic tasks, Noul and Score both ranked the obvious match first, 4 of 4, for Kev 4B and for Kev 9B. At the old 0.75 cutoff, normalized Score admitted eight extra candidates on 9B, and Noul admitted none of those. The note says the fixture does not establish general accuracy. The suite passed against Kev 4B with no auth and Kev 9B with a token, 51 tests each.
A question-format section in the same note, on hosted Jev, says shorter task-rubric text cut input tokens by 10 to 17 percent and mean latency by 10 to 16 percent. The note says that finding does not show the layouts are equal on accuracy. A separate paired report is linked for speed and includes hosted Jev. We left that report unopened.
This site's reading
Editorial notes evaluating claims against primary sources, contextualizing findings alongside related implementations, and defining technical terms.
Verify
Post by @championswimmer, September 29, 2026, 10:22 UTC. The thread says a 20-skill demo saves about 1,000 tokens, a 250-plus skill repo at work saves about 10,000, and batched scoring of 250-plus skills stays under a second. A reply says local Kev on an M3 or newer Pro or Max can replace the network call.
The image is the README. Repository github.com/championswimmer/pi-skill-picker-jev, MIT, copyright 2026 Arnav Gupta, package pi-skill-picker-jev 0.1.0. README: Pi 0.87.1 or newer, an OpenRouter key or another decision server, and about 1,000 tokens off the initial prompt on the 20-skill storefront demo.
The 10,000-token work repo and the one-second claim are not in that README. The screenshot in the README shows five skills added, with ratings 0.760, 0.555, 0.468, 0.345, and 0.345 beside a 0.300 line. If the decision service is down, the picker does not load the full catalog.
The privacy note says the default path sends the request, recent conversation, tool text, and skill names and descriptions to OpenRouter, and does not send full skill files. docs/scoring-research.md is linked. We did not boot Pi.
On September 30, 2026 this desk scored skill names with jev-latest. Responses named jev-1.13.0. The rule copied from the extension: Score levels 0 through 4, keep a skill when the raw score divided by 4 is at least 0.625, at most 6 new skills, batches of 50. Eight tasks named the required skills ahead of time. Recall is how many of those names landed in the selected set. Task success here means every required name was selected. Pi itself did not run, so this is a selection gate, not a finished storefront edit.
The 20-name catalog is the repository's demo skills. Necessary-skill recall was 16 of 19. Tasks with every required skill selected: 5 of 8. Wall times were 867, 880, 933, 950, 952, 1,066, 1,096, and 2,007 ms. The median is 951 ms. The 250-name catalog is those 20 skills plus 230 fillers written on this desk, including a few near-miss names. It is not Gupta's private work repository. Recall was again 16 of 19, and again 5 of 8 tasks cleared the gate. Wall times were 1,325, 1,517, 2,166, 2,285, 2,360, 2,689, 4,377, and 4,469 ms. The median is 2,322 ms. That catalog did not stay under a second.
Skill-body characters on the 20-name demo are 3,264 in the catalog and 471 in the mean selected set. Dividing the difference by 4 is an estimate of 698 tokens kept out of a prompt. Mean Jev input on that catalog was 2,834 tokens per task, 22,668 across the eight tasks, $0.000952 at $0.042 per million input tokens. The call spends more tokens than the short demo bodies save. The 250-name catalog used 257,652 input tokens across the eight tasks, about 32,207 per task, $0.010821 at list price. Output tokens are free on that price. The README's about 1,000 tokens is in the neighborhood of the gross body cut. Net of the Jev call, this 20-skill run did not save tokens. The private catalog behind the post's 10,000-token figure was not available.
Wrong selections on the 20-name catalog: low-stock also took catalog-data; migration also took release-checklist; runbook also took checkout-flow and observability; tax-ship missed shipping-policy and took pricing-rules; privacy missed release-checklist and took api-design; api-perf missed catalog-data and took observability. Coupon and the empty-cart task matched the required names exactly. The 250-name catalog made the same misses. Its extras were catalog-data, release-checklist, checkout-flow, pricing-rules, and observability. Privacy on that catalog missed release-checklist and added no extra name. No filler name cleared 0.625.
When the decision call was pointed at 127.0.0.1 port 9, the error was connection refused, in under a millisecond. The harness then applied the extension's failure policy: the selected set stayed empty, and the full catalog was not loaded. A call with a bad key returned HTTP 401, authentication_error. extension.ts clears the catalog before the try, and the catch does not put it back. Pi was not booted, so this is that policy on the recorded error, not a click in the editor.
On September 30, 2026 we opened that note. Production asks Score on five levels, 0 through 4: Unrelated, topically adjacent, optional support, directly useful, and Essential. The default threshold is 0.625, a raw score of 2.5. The note calls that a starting point taken from the rubric.
On six synthetic tasks, Noul and Score both ranked the obvious match first, 4 of 4, for Kev 4B and for Kev 9B. At the old 0.75 cutoff, normalized Score admitted eight extra candidates on 9B, and Noul admitted none of those. The note says the fixture does not establish general accuracy.
The suite passed against Kev 4B with no auth and Kev 9B with a token, 51 tests each. A question-format section in the same note, on hosted Jev, says shorter task-rubric text cut input tokens by 10 to 17 percent and mean latency by 10 to 16 percent, and that the finding does not show the layouts are equal on accuracy. A separate paired report is linked for speed and includes hosted Jev. We left that report unopened.
Compare
JevScout also uses Jev to drop text before a coding agent reads it, on oversized MCP results, with a printed seven-task check. This picker ranks skill names for Pi. The only number in the README is about 1,000 tokens on a 20-skill demo.
It does not print whether the agent still solved the task.
Terms
- about 1,000 tokens
- The README's figure for the 20-skill storefront demo: initial system prompt with the picker, against the same prompt carrying the whole catalog.
- about 10,000 tokens
- The post's figure for a 250-plus skill repository at work. The README does not describe that repository.
- 0.300
- The line next to the five ratings in the README screenshot: 0.760, 0.555, 0.468, 0.345, and 0.345. The screenshot labels the group as five skills added.