Published
Updated
One builder ran Jev on 1,500 of his own emails
This topic was created 31 days ago, and the information it contains may have evolved or changed since then.
Ryan Vogel posted a long clip of Jev classifying his mail. The sample size is real; the accuracy number is not in the post. Parallel calls were the part viewers argued about. On September 30, 2026 this desk scored 30 fictional messages on jev-1.13.0. Precision and recall were 1.0 in each of five classes. That set is not his mailbox.
The morning after launch, Ryan Vogel posted a video demonstrating Jev processing 1,500 personal emails. The post praised the model’s classification speed, followed by a longer YouTube video. Observers discussed whether issuing requests in parallel was more efficient than combining queries into batched calls.
Email categorization is a standard scenario in TypeSafe’s documentation, covering folder routing, urgency scoring, and spam detection. Vogel tested the system on a substantial personal archive rather than synthetic samples. The post omitted formal benchmark statistics, such as accuracy metrics, error distributions, or handling of low-confidence predictions.
In this pattern, the application manages mailbox storage and mail actions, while Jev evaluates the declared classification questions.
On September 30, 2026 this desk wrote and labeled 30 fictional messages, six each of urgent, billing, newsletter, personal, and phishing, then sent them one at a time. jev-1.13.0 put every message on the diagonal of a five-by-five confusion matrix. Precision and recall were 1.0 in each class. No urgent message was filed elsewhere, and no other class was marked urgent. p50 was 919.1 ms, p95 was 2,017.1 ms, and the list cost at $0.042 per million input tokens was $0.000536 for 12,772 input tokens. The lines were written to be easy to tell apart. They are not an authorized, de-identified export of anyone’s mailbox, and they do not stand in for the 1,500-message clip.
This site's reading
Editorial notes evaluating claims against primary sources, contextualizing findings alongside related implementations, and defining technical terms.
Verify
The post reports 1,500 personal emails and notes the author's positive reaction. It does not publish precision, recall, confusion matrices, or the full label set.
Responding to questions about parallel requests versus batched evaluations, TypeSafe documentation indicates multiple questions about a single state can share a request, while 1,500 distinct emails require individual requests unless formatted into a single batch. We did not independently run the evaluation.
The author's enthusiasm represents a personal demonstration rather than a published benchmark.
On September 30, 2026 this desk wrote 30 fictional messages and labeled them before the calls, six each of urgent, billing, newsletter, personal, and phishing. They are not Ryan Vogel's mail, and they are not a customer export. Nobody authorized a live mailbox for this check. The request used jev-latest. All 30 responses named jev-1.13.0 and returned a label inside the set. p50 was 919.1 ms and p95 was 2,017.1 ms, from 777.1 ms to 2,556.0 ms, one call at a time from this host. Input tokens were 12,772 and output tokens were 1,632. At $0.042 per million input tokens, output at $0, the list cost is $0.000536.
The confusion matrix is diagonal. Each class has true positives 6, false positives 0, and false negatives 0, so precision and recall are 1.0 for urgent, billing, newsletter, personal, and phishing. The missed-urgent list is empty. The false-urgent list is empty. The messages were written so the classes would separate. A perfect matrix on this set does not estimate error rates on a personal archive of 1,500 messages. Vogel's clip still has no published per-class precision, recall, or confusion matrix, and this desk does not have that mailbox.
Compare
This setup illustrates a basic classification task: assigning text to a defined category or workflow. Mudholkar's later 1,565-email run is the named-company table with a Gemini baseline, a confidence band, and a cost per thousand.
LangChain's subsequent tutorial follows a similar pattern by scoring ticket urgency and routing. Netlify's changelog provides a contact form triage example routing between sales, support, and spam. Vogel's demonstration predates these and uses a larger personal dataset.
It does not demonstrate whether Jev outperforms a fine-tuned classifier on his data. Reported confidence on Choice reflects the probability distribution rather than measured retrieval precision.
Terms
- State
- The input data provided to Jev, such as a single email or batch. The calling application retrieves and supplies the data.
- Choice
- Selecting a single label from a user-defined set (such as invoice, personal, or newsletter), returning a probability distribution across the options.
- Parallel questions
- Submitting multiple questions regarding the same state within one API request, distinct from classifying multiple different emails simultaneously.