Published
Jev macOS Loop clicks native Mac windows; Finder sorted nine files in 7.39 seconds
Josh C. Simmons open-sourced a computer-use loop for Apple silicon that keeps pixels on the Mac and asks Jev only over observed controls. A Finder demo moved nine dummy files in three group drags in 7.39 seconds, with a separate verifier checking locations and hashes.
Josh C. Simmons posted Jev macOS Loop on September 19. The repository is jcpsimmons/jev-macos-loop.
The loop is for Apple silicon. A persistent Swift process captures the selected window, runs OmniParser and OCR locally, merges accessibility labels, and asks Jev a choice over the observed element IDs. Clicks are guarded for focus, occlusion, and a fresh frame. Pixels and coordinates stay on the Mac. The provider sees observed text, the goal, and recent actions.
Bring your own token. JEV_PROVIDER selects Vercel AI Gateway (default), OpenRouter, or TypeSafe. A missing or rejected token stops the run. The README says Vercel and OpenRouter have passed live checks; the TypeSafe adapter has offline contract tests.
The Finder demo uses real Finder, nine dummy text files, and three empty folders. Jev classifies all nine names in one request. The runner selects each group through accessibility and moves it with one native drag. A separate verifier checks locations and content hashes. The README reports 7.39 seconds through Vercel, down from 22.69 seconds when files moved one at a time. Both timings include Finder settling and the verifier.
performance.md records a six-task native GUI suite, tested September 17 in Arizona (September 18 UTC) on an M4 Max: 6/6 on Vercel and 6/6 on OpenRouter. Median Jev round-trip 301.9 ms versus 293.5 ms; full observe-decide-act cycle 394.6 ms versus 367.2 ms. Task times ran from about 1 to 2.5 seconds. Apple Calculator computed 32 x 14 = 448 in 2.940 seconds and 17 x 23 = 391 in 3.268 seconds, with a separate display reader. An earlier Calculator run entered an extra digit and produced 3,968; the verifier caught it.
The notes call these small functional samples. Icon-only controls and OCR-only state remain weaker than labeled accessibility trees. General drags, free-form typing, and multi-window work are out of scope. License is AGPL-3.0.
This site's reading
Editorial notes evaluating claims against primary sources, contextualizing findings alongside related implementations, and defining technical terms.
Verify
The September 19 post points at jcpsimmons/jev-macos-loop. The README and performance.md agree on the Finder demo (nine files, three group moves, 7.39 seconds through Vercel, 22.69 seconds for one-file moves) and on the six-task native suite (6/6 on Vercel and 6/6 on OpenRouter). Median Jev round-trip on that suite is 301.9 ms (Vercel) and 293.5 ms (OpenRouter). Apple Calculator cases 32 x 14 = 448 and 17 x 23 = 391 are in the same notes, with a separate display reader. An earlier Calculator run produced 3,968 and is recorded as a failure caught by the verifier. Direct TypeSafe console tokens have contract tests, not a live-token measurement. We did not run the loop.
Compare
Browser Use and Stagehand rank page elements from a browser DOM or an accessibility tree. This loop ranks native macOS controls after local OmniParser, Vision OCR, and accessibility fusion. Screenshots stay on the machine; only text, the goal, and a finite choice go to the provider. Vercel is the default route, with OpenRouter and TypeSafe as alternatives, which matches the Gateway free window in the adoption story. DONE is a model decision; the included benches check the app state separately, the same split Malis used when Postgres planned first.
Terms
- Native computer use
- Clicking and dragging controls in a real macOS window rather than driving a browser DOM.
- Local perception
- ScreenCaptureKit, OmniParser CoreML, Apple Vision OCR, and accessibility labels run on the Mac. Pixels and coordinates are not sent to Jev.
- Independent verifier
- A check of the app's actual state (file locations and hashes, Calculator display) that the decision policy cannot read.