Published
Updated
Upstage's Solar Decide chart prints 87.0% on the public 231, and a p95 of 15.7 seconds
This topic was created 31 days ago, and the information it contains may have evolved or changed since then.
Sung Kim posted Solar Decide on OpenRouter on September 29. The chart says the comparison is the public JevBench 231, easy 48, standard 72, hard 111. Solar Decide is 87.0% at a p50 of 0.14 seconds. Jev 1.13, measured through OpenRouter, is 86.1% at 0.30 seconds. The footnote puts p95 at 15.7 seconds against Jev at 0.40 seconds. We did not rerun the 231.
Sung Kim posted on September 29 that Upstage had put Solar Decide on OpenRouter as upstage/solar-decide. The first image in the thread is a comparison with Jev 1.13 on the public JevBench set: easy 48, standard 72, hard 111, which is 231 items. The source line points at fstandhartinger/jevbench. The card says the Jev row was measured on OpenRouter.
Solar Decide is printed at 87.0% and a median of 0.14 seconds. Jev 1.13 is 86.1% and 0.30 seconds. The card calls that median 2.1 times faster. The price line is $0.05 per million input tokens, free output, and a 512K context against the 32K it prints for Jev. The footnote under the card says the speed is a median, and that hard items can take longer. It prints a p95 of 15.7 seconds for Solar Decide and 0.40 seconds for Jev. The median favors Solar Decide. The tail in that footnote does not.
The 231 is the same public split other posts use. Open-Jev’s note counts Jev at 200 of 231 on that split, which is 86.6%. This card prints 86.1% and does not show the numerator. Those two Jev figures are not the same published count. 87.0% of 231 is 200.97, so the printed percent is a one-decimal rounding, not a count of items we saw.
The post also describes AskUp, Upstage’s own product. A keyword rule kept 14 of 20 facts the team wanted in long-term memory. Solar Decide kept 20 of 20, and the post says it stored none of the casual chat by mistake. That check does not include a Jev row. We did not call upstage/solar-decide, and we did not rerun the 231.
This site's reading
Editorial notes evaluating claims against primary sources, contextualizing findings alongside related implementations, and defining technical terms.
Verify
Post by @SungKimUp, September 29, 2026, 06:09 UTC. The account bio calls him Upstage co-founder and CEO. The first image says the comparison is JevBench's public 231 items, easy 48, standard 72, hard 111, and that Jev 1.13 was measured on OpenRouter.
The source line cites github.com/fstandhartinger/jevbench. Printed cells: solar-decide 87.0% and 0.14 seconds, Jev 1.13 86.1% and 0.30 seconds. The card also prints $0.05 per million input tokens, $0 output, and a 512K context against Jev's 32K.
The footnote says the speed is a median, and that hard items can take longer: p95 15.7 seconds for Solar Decide and 0.40 seconds for Jev. The post's text adds an AskUp memory check: a keyword rule kept 14 of 20 facts, Solar Decide kept 20 of 20, and casual chat stored by mistake was 0. That check has no Jev cell.
The model id in the post is upstage/solar-decide. We did not call the model and we did not rerun the 231.
Compare
Open-Jev's note on the same public 231 counts Jev at 200 of 231, which is 86.6%, and its own 27B v1.1 at 197 of 231. This chart prints Jev at 86.1% and does not print an integer count, so the two Jev cells are not the same published fraction.
JevBench v1.5's official score is a different scale, 1,624 decisions, where Jev is 72.1.
Terms
- 87.0%
- Solar Decide on the chart's public JevBench 231. Jev 1.13 on the same card is 86.1%. The card does not print how many items those percents are.
- 15.7 seconds
- The footnote's p95 for Solar Decide. Jev's p95 on that line is 0.40 seconds. The headline speed, 0.14 seconds against 0.30 seconds, is the median.
- 20 of 20
- AskUp facts Sung Kim says Solar Decide kept for long-term memory, against a keyword rule at 14 of 20. Casual chat stored by mistake is 0. The post does not run Jev on those 20.