Published
MotherDuck's prompt_jev() labels 100,000 AG News rows in 40 seconds for $0.50
MotherDuck shipped prompt_jev(), a SQL function that calls TypeSafe's Jev. On 100,000 articles drawn from the AG News training split, the published table is 2,484 rows per second, 89% accuracy, $0.50, and 40 seconds. GPT-5.6 Terra on the same sample is 52 rows per second, 88%, $37.58, and 31 minutes 59 seconds.
MotherDuck posted on September 21 that text classification in its warehouse had a new SQL function, prompt_jev(), backed by TypeSafe’s Jev. The post says 100,000 rows took 40 seconds and $0.50, against an LLM at 32 minutes and $37. TypeSafe quoted that post on September 22 and wrote “50x faster. 100x cheaper. Reliable as duck.”
The write-up is Introducing prompt_jev(). The page we read is dated 2026/09/21 and does not name an author. The function is on paid MotherDuck plans. A call takes a text column, an instruction, and a choice list. The list can be plain labels or {label, description} objects. The scoring query in the post reads result.choice and result.confidence. The opening paragraphs also say the function can return a score or a yes/no. The statements they print are choice.
The table is 100,000 articles from the training split of AG News, the four-class news set from Zhang, Zhao, and LeCun (2015), scored against the dataset labels. The reproduction script loads hf://datasets/fancyzhx/ag_news/data/train-00000-of-00001.parquet and draws SAMPLE 100000 ROWS (reservoir, 43). Null choices are counted apart and are not scored as wrong. The published table does not print how many nulls there were.
Jev is 2,484 rows per second, 89% accuracy, $0.50, and 40 seconds. gpt-4o-mini is 84 rows per second, 80%, $1.93, and 19 minutes 45 seconds. gpt-5-nano is 94, 83%, $1.58, and 17 minutes 49 seconds. gpt-5.6-luna is 61, 84%, $3.53, and 27 minutes 25 seconds. gpt-5.6-terra is 52, 88%, $37.58, and 31 minutes 59 seconds.
The tweet’s 32 minutes and $37 line up with the terra row. Jev’s printed accuracy on that row is one point higher, 89% versus 88%. Throughput against terra is 2,484 / 52, about 48 times. $0.50 is about 1.3% of $37.58, which is about 75 times cheaper, not the 100 times in TypeSafe’s quote. Against gpt-4o-mini the same dollars are about 3.9 times, and the rows-per-second ratio is about 30. The blog says Jev exceeded the other models by more than 25 times across cost, accuracy, and speed. The accuracy figures are 89% versus a range of 80% to 88%. Cost versus mini, nano, and luna stays under 8 times.
A later sentence says runs at 1 million and 10 million rows were similar, and in some cases faster than the 100,000-row baseline. The page does not include those tables.
This sample comes from the training split. It is not AG News’s held-out test. jeff’s published AG News number, 75.5% for the local model and 90.5% for Jev, is 1,600 public items inside a different eight-dataset script. The 89% here and the 90.5% there are not two readings of one run.
We did not execute the SQL.
This site's reading
Editorial notes evaluating claims against primary sources, contextualizing findings alongside related implementations, and defining technical terms.
Verify
Primary post is @motherduck on September 21, linking motherduck.com/blog/motherduck-supports-jev, dated 2026/09/21. The page we read has no byline. prompt_jev() is a scalar SQL function on paid MotherDuck plans. The published sample classifies a text column with choice := a list of labels, or a list of {label, description} objects, and the scoring query reads result.choice and result.confidence. The intro also says the function can return a score or a yes/no. The printed statements are choice. The benchmark draws 100,000 rows with SAMPLE 100000 ROWS (reservoir, 43) from the AG News training parquet on Hugging Face (hf://datasets/fancyzhx/ag_news/data/train-00000-of-00001.parquet), Zhang, Zhao and LeCun 2015, four topics. NULLs are counted separately and not scored as wrong. The table does not print that null count. Rows per second, accuracy, retail cost per 100k, wall time: Jev 2,484 / 89% / $0.50 / 40s; gpt-4o-mini 84 / 80% / $1.93 / 19m 45s; gpt-5-nano 94 / 83% / $1.58 / 17m 49s; gpt-5.6-luna 61 / 84% / $3.53 / 27m 25s; gpt-5.6-terra 52 / 88% / $37.58 / 31m 59s. The tweet's "32 min and $37" matches the terra row. Against that row, 2,484/52 is about 48 times the throughput and $0.50 is about 1.3% of $37.58. TypeSafe's September 22 quote says "50x faster. 100x cheaper." $37.58 / $0.50 is about 75 times, not 100. The blog line that Jev exceeded the other models by more than 25 times "across cost, accuracy, and speed" does not match the accuracy column (89% versus 80% to 88%). Cost versus mini, nano, and luna is under 8 times. A sentence says tests at 1 million and 10 million rows were similar and sometimes faster. No table for those sizes is on the page. This sample is the training split, not a held-out test. We did not run the SQL. TypeSafe's Master Customer Agreement section 2.3(f) forbids publishing benchmarks of the Services; the figures are reported as published.
Compare
duckdb-jev and Jevflake also ask TypeSafe from SQL. Those READMEs publish throughput or a ten-ticket sample, not topic accuracy. jeff's AG News figure is 75.5% versus Jev 90.5% on 1,600 public items across eight datasets, a different cut from this 100,000-row training sample. Hosted Jev on that smaller table is 90.5%, and here it is 89%. The two scripts are not the same measurement.
Terms
- prompt_jev
- A scalar SQL function on paid MotherDuck plans. It sends a text value and a choice, score, or yes/no question to TypeSafe's Jev and returns a structured result the rest of the statement can filter.
- AG News training sample
- MotherDuck's published prompt_jev table uses 100,000 rows drawn with a reservoir sample, seed 43, from the AG News training parquet. It is not the dataset's held-out test split.