Pinned Tweet
23.6 → 184 tok/s.
Qwen3.8-Flash-Next. One RTX 6000 Pro. ~7.8x faster decode, ~99.8% of BF16 intelligence kept.
We post-train, quantize and write kernels until open models go brrr locally.
Weights on HF. Try it. DM for early access.
matteo horvath retweeted
Own Your Intelligence
@xuster
Most companies building with AI still rent their intelligence from frontier labs.
As open models improve, more companies will want models they can run, train, and control themselves. The opportunity is to turn a company’s workflows, expertise, and failures into specialized models that improve with the business.
this is goethe, our open source inference engine
we kept building new kernel + model combos for clients and kept hitting the same wall: getting them to actually run fast on the client's hardware
so we built our own engine. install it, run goethe start, and your gpu box becomes an openai + anthropic compatible api
blackwell down to ampere. apache 2.0
we think everyone should own their inference, so we open sourced it
11.8 → 143 tok/s.
DeepSeek-V4.1-Flash-Ternary.
510 GB base → 142 GB pack.
~12.1x faster decode at concurrency 1.
~92% of FP4 intelligence kept.
ternary is not free and we lose ~8% vs FP4
for coding and research agents, a model you can run 12x more often beats a slightly smarter one you can't.
model card + all numbers: meshapplied.com/models
serve it with goethe (apache 2.0): github.com/meshapplied/goeth…
matteo horvath retweeted
Introducing Fern, Mirendil’s first model: pretrained from scratch, with up to 10M tokens of context!
That’s ~0.5M lines of code (roughly Megatron-LM scale) or days of an agent’s work, all in context at once.
Its first version, Fern 0.1, had a short round of post-training 🚢
matteo horvath retweeted
The importance of open source and local AI as a counter balance is dawning on people – at San Francisco, CA
matteo horvath retweeted
Zigong, traveling north of Market and turning back toward The Arena, passed by an enterprise and saw an old engineer preparing his PR. He had typed the code by hand in vim and traced the execution in a debugger -- straining, sweat-dark, spending enormous effort for tiny results.
Zigong said: "There is an agent for this. In one day it produces one hundred PRs, with next to no effort and great results. Wouldn’t you like it?"
The gardener’s face darkened -- and he laughed. "I have heard from my teacher: where there is artificial intelligence, there are artificial dealings; where there are artificial dealings, there is artificial progress. When artificial progress lodges in the goals, the Way will not carry the team. It is not that I do not know the thing. I would be ashamed to use it."
Zigong hung his head and could not answer.
insane alpha from @eastdakota and @dillonrolnick
i really like that they are not p(doom) maximizing in their conversation, and i highly respect the mission of both companies.
Yesterday AI was used to uncover major vulnerabilities in the infrastructure of the internet.
It's yet another example of how profoundly AI is reshaping the web.
But what emerges on the other side is very much up for grabs.
We were lucky enough to recently introduce two heroes of the Rebel Alliance
@eastdakota, CEO & cofounder of @Cloudflare
@dillonrolnick, CEO & cofounder of @NousResearch, the creators of Hermes, the largest open source agent harness in the world.
Together they discussed who gets to build, control, and benefit from an internet run by AI
It's worth a listen if you care about the future of the web.
0:00 Preview
1:41 From a Discord Group to an AI Lab
8:45 Can Anyone Actually Pause AI?
12:59 Would You Trust an Agent With Your Inbox?
14:58 Why Matthew Wouldn’t Trade Places With Sam or Dario
20:28 The Dead Internet Theory Was Wrong
22:28 Agents Don’t Click Ads. Who Pays?
30:38 Where Should Your Agent Actually Run?
32:32 Why American Builders Are Using Chinese Models
38:37 What Happens When Our Agents Work Together?
matteo horvath retweeted
14 yo asked how to write good essays. I asked if he meant the kind kids write in school, or actual good essays as written by adults. How strange that I have to ask that. If he asked how to solve a math problem, I wouldn't have to ask if he meant the school way or the real way.
matteo horvath retweeted
We're introducing TasteBench, a new standard to measure Research Taste on real world ML tasks.
Existing benchmarks have limited datasets and eval data ultimately becomes part of training data. In TasteBench, we continuously generate new ML tasks on OOS data no model has trained on.