co-founder @refresh_dev (YC P25), building RL training gyms | Prev AI @Uber , CS @UofIllinois | believer in communities
San Francisco
Joined February 2022
- Tweets1K
- Following2.3K
- Followers2.2K
- Likes19K
The main bottleneck to rl environments is the amount of disk space you have available on your MacBook.
Christopher Settles retweeted
We are releasing Ultramarine 1 Flash, a frontier-class model built for knowledge work. It is open-source and achieves similar performance to Fable 5, while being 60x cheaper.
It was post-trained by a team of 5 for ~$300k all-in. Available on API now, with weights releasing in 2 weeks.
Christopher Settles retweeted
Huge day for us today: @resend is now billing through Autumn
This is a full circle moment. @johnyeo_ using the Resend API is actually what inspired us to build a devtool in the first place. They set the standard for product quality and user experience.
They moved to @autumnpricing because they needed a single, reliable engine to monetize all the new products they ship, and their growing enterprise motion meant a lot more custom contracts to manage.
Resend is the most downloaded email API in the world and growing like crazy. We’re super proud they chose us to meter, enforce and bill their highest-volume workloads.
Christopher Settles retweeted
Today we're launching WebCloneBench: a benchmark which measures a model's ability to explore, copy, build and test various web apps.
For each task, we give the model a reference site from one of Gmail, Google Sheets, Jira, DocuSign, Shopify Admin, Craigslist, Figma, Slack and Zendesk. It is instructed to build a fully feature-complete clone of every part of the application with an 8 hour time limit.
Another animation benchmark the same day! How awesome that we're all tying together.
@himanshustwts we should collab!
Today, I am excited to announce Animation Bench. Our first benchmark to evaluate frontier models on web animation reconstruction.
Frontend generation is a common commercial use case for coding agents. In addition to static layout it contains transitions and interactions triggered by dynamic behaviours. We evaluated four frontier models on 48 tasks from real websites across three axes:
1. Visual Similarity
2. Motion Consistency
3. Layout Correctness.
We find that motion consistency is often incomplete or missing.
Christopher Settles retweeted
Too many people claim that coding LLMs can handle 3D, but how exactly? We need more benchmarks on this.
Christopher Settles retweeted
New continual benchmark just dropped! Excited to chug away at this, the models are good in some aspects but we are still far away from models making high fidelity 3d movies.
Today at Refresh we're proud to release our first public benchmark, BBv1.
We talked to animation experts who did the 3D modeling for harry potter, and realized that so many of the capabilities of the LLMs were still lacking.
Verifiers include VLM as a judge, along with geometry!
If interested in 3D animation evals, DMs open!
It is said often
To all the data company founders, if you can’t do 18 hours of work 6-7 days a week in bursts when needed, you’re ngmi as a category leader.
This is what Mercorians are doing regularly, and what we did at Scale AI back when Scale was still relevant. This is also why folks either survive and thrive after 6 months or they leave after 1-3 when the shiny logo aura wears off and it dawns on them that the work is done with blood, sweat, and tears on top of being adaptable and needing to learn at warpspeed.
This is also why Scale AI is cooked is because I hear folks are doing 50 hr weeks now with good work life balance.
It’s the harsh and brutal honest truth is that for operationally driven products and businesses, the champs will simply outwork you.
But if you can survive and thrive, you get granted with the power of Beast Mode. Afterwards, everything else seems like a walk in the park.
Christopher Settles retweeted
We’re sharing a solution to the Navier-Stokes Millennium Prize Problem, one of the deepest problems at the frontier of mathematics.
The proof was produced by a group of agents, using an OpenAI next-generation model significantly more capable than GPT-6 Astra.
The problem concerns whether the description of smooth three-dimensional fluid motion modeled by the Navier-Stokes equations can break down. It has remained unresolved for roughly 90 years.
Christopher Settles retweeted
Introducing Omnara: the open-source alternative to Claude Managed Agents.
Omnara is an API for creating and interacting with production-grade agents. You build the agent. Omnara handles hosting, durable state, sandboxes, permissions, secrets, and the rest of the production infrastructure.
Available now @omnaraai
Christopher Settles retweeted
Meta MSL superstar
“The word general in artificial general intelligence is loosely defined. Much of the flexibility we observe in current LLMs may be interpolation across the extraordinary breadth of internet data rather than pure extrapolation beyond it.”
This quoted post is unavailable.
Christopher Settles retweeted
Hello again, world!
Hacker News has been pretty quiet on X. Time to fix that.
Starting today, we'll be sharing interesting things people are building, discovering, and discussing on HN.
And let us know what's caught your eye lately. There's always room for another great link!
