Accelerating LLMs @Basetenco - long-context and embedding inference (https://xitter.cf/t.co/IdBf5U7mS3) - opinions are my own.
San Francisco, CA
Joined February 2024
- Tweets146
- Following157
- Followers528
- Likes879
d) baselabs/elephant now running at ~50ms latency on any datacenter near you.
our decision models are:
a) frontier
b) free for the week on @baseten Model APIs
c) completely open-source
Michael Feil retweeted
Nemotron-3.5-Embed is now available in turbopuffer native embedding
on our enterprise PDF search benchmark, Nemotron-3.5-Embed 8B improves recall@20 by ~5% over the next best model
turbopuffer.com/docs/embeddi…
We’re releasing more open models to help teams across Europe build AI with their own data, on infrastructure they control.
Announced at GTC Berlin, the models help developers, researchers and enterprises build specialized AI agents, train robots, design proteins and develop quantum computers.
See what’s new: nvda.ws/4iwBMvF
Michael Feil retweeted
Today we’re releasing two decision models, elephant-1 (9B) and mammoth-1 (27B), trained by the @baselabs team, available for free for one week on our Model APIs.
mammoth-1 is #1 on the Jev Decision Index and elephant-1 is tied for #1 on the Artificial Analysis Decision Index.
It’s a new model, but more importantly, this is an open-source model you can access, deploy, and post-train on Baseten (or wherever you do your best work).
The free week ends 10/27/26 at 11:59 PM PT. Try it and let us know what you think.
Try mammoth-1 here: baseten.co/library/mammoth-1…
Try elephant-1 here:
baseten.co/library/elephant-…
Michael Feil retweeted
We trained the #1 decision models in the world, mammoth-1 and elephant-1!
This is our first Base Labs model release and we're excited to open-source all weights and training details. We really focused on scaling high-quality data in these training runs – not only synthetically generating new decision data but stacking how we validated our data. This follows something called our 'swiss cheese' approach to looping over and improving training data.
There are also a bunch more low hanging fruit when it comes to training these models, and we expect that we'll improve quickly over the coming weeks.
Today we release mammoth-1 and elephant-1!
We post-trained mammoth-1, the #1 ranked best decision model in the world, and elephant-1, which is the best model under 15B (and beats Jev).
We're open-sourcing all model weights and provide a technical report detailing all our training and data methods.
While working on GLM5.2 chat-templating for Basetenkenizer, found a small bug, thats now fixed with latest minijinja 2.22 (xitter.cf/feilsystem/status/2081…) and will render the chat-template correctly. Will be included in future releases.
github.com/mitsuhiko/minijin…
We shipped the worlds fastest tokenizer for KimiK3. Basetenkenizer now available on day 0 on Baseten and on pypi.
Michael Feil retweeted
@baseten model performance team is absolutely cracked. @Zai_org GLM 5.2 is now 4x faster running at full 1M context!
Already available to use in your favorite coding harnesses, here it is COOKING in @FactoryAI Droid and @opencode
Docs for how to get it in comments
Michael Feil retweeted
Biotech R&D is generating more scientific AI models than ever, from protein structure prediction to molecular docking to sequence analysis. But the infrastructure to run them hasn't kept up.
Today we're announcing Benchling Inference, powered by Baseten. Together with @benchling, we're delivering on-demand GPU capacity built for the bursty, high-stakes demands of scientific workloads. With Benchling Inference, scientists can:
→ Deploy models in seconds, not weeks
→ Keep proprietary models inside their VPC if needed
→ Benefit from economics that work even at small and mid-size biotech scale
Benchling and Baseten decided to team up because we believe that research teams shouldn't have to manage HPC queues, negotiate cloud contracts, or become GPU experts to run frontier models on their own data.
Six years of inference expertise are now available where science happens.
Read more here: benchling.com/blog/announcin…
Michael Feil retweeted
OpenAI is shutting down text-embedding-3-small?!?
I strongly believe that if you shut down a closed-source embedding model that you should open-source. Imaging the trillions of tokens that will no longer be queryable.
cc @romainhuet
lets go baseten -- rooting for you.
We are excited to have @baseten as a day 0 launch partner for Kimi K2.6!
Their inference stack brings KV-aware routing, NVFP4 on Blackwell, multi-modal hierarchical caching, and prefill-decode disaggregation, so K2.6 runs the way it's meant to in production.
Try it out at: baseten.co/library/kimi-k26
Michael Feil retweeted
why join a dense team when you can join an moe team @baseten
we're a dense team of 10 jane street engineers, ioi medalists, and sunshine enjoyers down in palo alto.
user numbers up >1,000% since partnering with @fraser
still early, dm me something exceptional you've done if u wanna join!