An open-source, 3D spatial-reasoning benchmark for evaluating LLMs Github Repository: https://teapaw.org/t.co/mz6gm3s9mC
Joined April 2026
- Tweets220
- Following1
- Followers290
- Likes7.5K
Pinned Tweet
MineBench.ai 4.0 is live – our biggest release yet!
Generations with Gemini 3.7 Flash for free for a limited time. No API key required!
Make anything you want, publish it to the new Gallery, and vote on the prompts you want to see added to the official benchmark :)
Also new:
- Generate from any prompt, save your builds, and publish them to the new Gallery
- Completely redesigned MineBench + a much faster Arena
- Accounts + personal model rankings
- Global rankings now use Bradley–Terry + 95% confidence intervals
(Oh and MineBench is now on the App Store :)
Full release notes: github.com/Ammaar-Alam/mineb…
MineBench retweeted
You can generate beautiful 3D builds on MineBench.ai for free (for Gemini 3.7/3.8 Flash), btw.
Gemini 4 Argon enters at the top spot on MineBench.ai :)
- Total Cost: $9.12
- Average Inference Time: 6m 15.1s
Its rating has yet to stabilize, but it currently pushes GPT-6.1 Sol Pro and GPT-6 Astra Pro off both the cost and speed Pareto frontiers by a significant margin.
Also added to MineBench recently: GPT-6.1 Sol Pro, Claude Sonnet 5.5, and Muse Wonder!
We’ve been selecting more community prompts for the official benchmark as well! Submit something you’d like to see the models try, and vote in the Arena for the builds you think best match the prompt!
ALT MineBench.ai efficiency leaderboards showing the Pareto frontiers on cost (top) and speed (bottom)
We haven't posted many announcements as of late, but there's been a lot of work behind the scenes!
We've continued making improvements and optimizations across both the site and iOS app, which now has a few hundred total downloads :D
On top of adding more models and prompts, the majority of our work has been on the research and evaluation side, including things like defining what makes a good build, understanding where current frontier models still fall short, and figuring out how to measure those gaps more reliably.
The prompts, builds, and human feedback that have accumulated on MineBench are becoming a big part of that work, alongside ongoing conversations with labs and researchers about where to take these evaluations next.
We're also training a model as part of this work. We'll share more about everything we've been working on in the coming weeks :)
Yes, that photorealistic bust of Einstein is still entirely made up of individual Minecraft blocks!
With the help of our sponsors, we've finally been able to add another batch of community prompts to MineBench.ai!
The models are getting so good that we're also officially raising the bar for future benchmark prompts. We now accept prompts going up to a gridSize of 512³ instead of just 256³ (8× the resolution!), along with a block palette of 322 different block types instead of 25.
Some thoughts:
- GPT 6.1 Sol's generations are extremely close to GPT 6 Astra's, and at times, indistinguishable. Some may even look slightly better, though Astra still seems to maintain a slight edge in overall accuracy and realism, especially with its proportions.
- Opus 5.5 seems like a huge step up from Opus 5. Unlike its predecessor, it's much less prone to over-engineering and adding excess details. It focuses on building exactly what was asked for, and avoids excessive labeling or text, even more so than Astra.
MineBench retweeted
FWIW, here are some MineBench builds Astra generated on release day and today with the same prompt
One-off before/afters don't tell you much about stochastic models; if there even was a change, it could just as likely be from changes in the harness
(top: release, bottom: today)
GPT-6 Astra Pro is the first model we've tested that seems to be able to handle basically any prompt we've thrown at it.
So we tried something a little ridiculous: we increased MineBench's gridSize from 256³ to 8,192³ and gave Astra this prompt: “A full rendition of New York City, fully accurate, dense, and to scale”
Astra wrote a ~61 KB program that produced 100.1 million blocks. Expanded into JSON, the build weighed 1.65 GB.
That's a 32x increase on every axis, or 32,768x more possible voxel positions. At ~1 meter per block, the canvas spans 8.2 km edge-to-edge.
Here's a preview of Astra's build. Note that, at this scale, our renderer struggles to show the entire build.
More GPT-6 Astra Pro generations from community prompts in the gallery! Astra is the first model on MineBench to accurately represent all continents on a globe.
View more, or add your own prompts and generations: minebench.ai/gallery
This is personally one of our favorite benchmarks!
Inspired by MazeBench, we had GPT-6 Astra build us a maze on MineBench, then tried to solve it ourselves :)
We couldn't make it through, but maybe you can: minebench.ai/gallery/gal_i0T…
GPT-6 Astra Pro is now established at an elo rating of 2,315, with a 220-point lead over #2 – the highest rating and largest gap we've seen on MineBench to date!
ALT MineBench leaderboard
GPT-6 Astra Pro debuts at #1 on the MineBench leaderboards, with a provisional rating of 2,225!
Vote on MineBench.ai to help solidify its rankings!
More metrics now visible on the leaderboards: minebench.ai/leaderboard
iOS app was updated as well :)
The default web-harnesses for most models these days are surprisingly capable
(More metrics like this coming natively to MineBench.ai soon :)
More GPT-6 Astra Pro generations from community prompts in the gallery! Astra is the first model on MineBench to accurately represent all continents on a globe.
View more, or add your own prompts and generations: minebench.ai/gallery
GPT-6 Astra Pro debuts at #1 on the MineBench leaderboards, with a provisional rating of 2,225!
Vote on MineBench.ai to help solidify its rankings!
MineBench retweeted
GPT-6 Pro has some quite insane results on MineBench.ai, and it’s 20.5x cheaper (based on the estimated cost) than GPT-5.6 Sol Pro.
Yet OpenAI only gives $200 Pro users 200 requests per week.
GPT-6 Astra Pro is now on MineBench.ai :)
- Total Cost: ~$34.71* (vs. ~$700 for GPT-5.6 Sol Pro)
- Average Inference Time: 25m 54s
This is the biggest jump we've seen on MineBench so far, both in build quality and efficiency. Every Astra build was valid on its first attempt, requiring zero retries within our harness. That reliability, alongside improved token efficiency, likely contributed substantially to lower cost.
We’re not convinced by the characterization of Astra as "AGI," but it does seem unusually good at understanding what matters: when to add additional scenery to a build or when it's better to stay focused on the object requested by the prompt.
Compare GPT-6 Astra with other models here: minebench.ai/sandbox?models=…
*Cost is currently an estimate while the provider dashboard updates.
It's quite interesting how similar GPT-6 Astra's builds are when given the same prompt on MineBench and @voxelbench
Nondeterministic but the model still came up with the same game/name and design twice
