Published
Updated
JEV Search moves NFCorpus nDCG@10 from 0.322 to 0.377
This topic was created 10 days ago, and the information it contains may have evolved or changed since then.
andre posted assadiandre/jev-search, a macOS app that ranks files with no saved index. On 623 queries the README's nDCG@10 goes from BM25 0.322 to JEV 0.377 on NFCorpus, and from 0.687 to 0.745 on SciFact. The repo root has no LICENSE file. The post's cost line is under $0.001 a query.
andre posted on September 23 a macOS file search that does not keep an index. You describe what you want, the app walks the folder, and Jev reorders the best local matches. The repo is assadiandre/jev-search. The root listing has no LICENSE file. The post’s video shows the desktop window. Its numbers are rounded: hard set 0.32 to 0.38, science set 0.69 to 0.75, under $0.001 a query.
The README names the sets. NFCorpus is 3,633 documents and 323 queries. SciFact is 5,183 documents and 300 queries. Together that is 623 queries. The ranking score is nDCG@10, BM25 then JEV: 0.322 to 0.377 on NFCorpus, and 0.687 to 0.745 on SciFact. A relevant result appears in the top five on 70.9% of NFCorpus queries and 80.3% of SciFact queries. Relevant documents found in the top 100 are 25.5% and 92.9%. Those two rows are one cell per set. Only the nDCG row is printed as a before and after.
Average API cost is 0.084¢ on NFCorpus and 0.130¢ on SciFact. Average time, including the JEV call, is 0.82 seconds and 1.10 seconds. Peak search-worker memory is 55.8 MiB and 57.8 MiB. The header says 66.3¢ for the 623 queries and no saved index. Dividing 66.3 cents by 623 is about $0.0011 a query. The post’s line is under $0.001. The README’s SciFact cell, 0.130¢, is $0.0013.
Every search starts from the folder you chose. The app always checks the file name. For text and code it counts words. PDFs and Office files get a short sample. Pictures match by name only. If an OpenRouter key is set, JEV reads the top 128 of those matches and moves the ones that fit the sentence. The rest stay where the local rank left them. Without a key the search stays on the machine.
A separate desktop line is local only: 21,595 entries, 3.88 seconds, 48.75 MiB of worker memory, no API call. That probe streamed 7,566 text files, sampled 13 documents, and searched 14,015 entries by name. The README says it was one Mac, files may have been cached, and the memory figure leaves out the UI process.
The privacy note says queries, paths, and excerpts go to OpenRouter and TypeSafe when the key is set. Credential filtering is described as best effort. Settings keys stay in memory, and ~/Desktop/jev.txt is also read. There is no Keychain access. Requirements are macOS, Python 3.12, and Node.js 22. We did not run a query.
This site's reading
Editorial notes evaluating claims against primary sources, contextualizing findings alongside related implementations, and defining technical terms.
Verify
Primary post is @AssadiAndre, display name andre, September 23, 2026, 09:32 UTC. It says hard set 0.32 to 0.38, science set 0.69 to 0.75, under $0.001 a query, and links assadiandre/jev-search. The README table names those sets NFCorpus and SciFact.
Documents and queries: 3,633 / 323 and 5,183 / 300. nDCG@10, BM25 then JEV: 0.322 to 0.377, and 0.687 to 0.745. Relevant result in the top 5: 70.9% and 80.3%. Relevant documents found in the top 100: 25.5% and 92.9%.
Average API cost: 0.084¢ and 0.130¢. Average time including JEV: 0.82 s and 1.10 s. Peak search-worker memory: 55.8 MiB and 57.8 MiB. Header: 623 queries, 66.3¢ total, no saved index.
Dividing that total by 623 is about $0.0011 a query. JEV reads the top 128 local matches. A desktop probe: 21,595 entries, 3.88 seconds, 48.75 MiB, no API call. macOS, Python 3.12, Node.js 22.
OpenRouter key. The repo root listing has no LICENSE file. We did not run a search.
Compare
kylemclaren/jevsearch measures Hit@1 on 41 labelled TypeSafe-docs queries, 83% against keyword search at 41%. This app measures nDCG@10 on NFCorpus and SciFact after a local BM25 pass, and it does not print a Hit@1 on that docs set. The top-100 row is one cell per set.
Only the nDCG row is written as BM25, then JEV.
Terms
- nDCG@10
- The README's ranking score. Higher means the relevant documents sit closer to the top of the first ten. NFCorpus moves from 0.322 to 0.377. SciFact moves from 0.687 to 0.745. The footnote says these are complete text-retrieval sets and that desktop timing varies.
- top 128
- After the local word match, JEV reads at most 128 candidates and moves the ones that fit the query. Files it does not check stay in the list as local matches. A search with no API key never makes that call.