https://teapaw.org/t.co/Xar2caBAyU https://teapaw.org/t.co/NLbGVsh4I2 Linkedin: https://teapaw.org/t.co/TwM7rRF6W9
Berlin, Germany
Joined May 2008
- Tweets3.9K
- Following98
- Followers12.1K
- Likes3.1K
Research summaries feel increasingly like stage magic, in that you assume the thing has been done the same way each time but actually the method is always different. One iteration uses a gimmick prop, the next one shows you the prop isn't gimmick but that one palms the object, the third time was a forced choice etc.
The "LLMs can natively read & write neural network weights" abstract describes experiments that can tell what a network is able to do, can write small game-playing networks from scratch, and can fix a Qwen3.5-0.8B model's ability to do maths by editing its weights.
Reading this abstract my assumption would be that these are all evaluations of a single method. The papers I read and wrote as an academic were much much less exciting, but that's how our experiments worked. Each experiment was there to demonstrate something about a single technique, e.g. "My technique still works on larger data sets", or "My technique generalises out of domain", etc.
In this paper, the things that they do to achieve all of those results are not at all the same as each other. When the abstract says, "Just by looking at the raw weights, GPT-6 Astra can tell what a network was trained to do.", that "Just by looking at the raw weights" is narrowly scoped! It describes _that_ result, but not all the results in the abstract. In the later claim, "Astra identifies an issue in its weights causing reasoning loops that Qwen’s own model card warns about, and fixes this by natively editing 0.016% of the weights.", that part requires the model to run the network.
Is this just a me problem? I guess maybe. If your cynicism is calibrated such that you didn't expect these were all the same constraints, then fine. I don't review for conferences anymore, and research cultures and expectations change. But by the standards I trained with I would definitely not accept the abstract written this way.
To spell out the difference here a bit more. When I first read the abstract, it sounded like the model reads the Qwen weights, and to some extent "mentally" simulates what the network does based on the weights, and figures out what the problem is.
In fact what it's doing at that part is running the network and noticing that the output is looping, and then devising an intervention in the weights to address it. To devise this intervention it can study the activations, adjust the weights, and rerun. It doesn't use backpropagation for this but actually that would be a totally fine technique for this: it could look at an activation and more easily calculate how the weights should change. But they want to say they don't use backpropagation I guess.
In another experiment, Astra is tasked with authoring weights to play a game. To achieve this, it devises a rule-based AI, then encodes that rule logic into weights directly. The technique to encode rules into transformer weights is known.
Do you see how these techniques have basically nothing to do with each other? A single technique that could achieve both, "Author weights for a small game from scratch" and "Improve the performance of a large model on a sample task" would imply a deep capability. If you test a single technique that way you've demonstrated a sort of 'gestalt' of the weights (it can write them out from scratch) and also that it understands a large existing network. But that's not at all true here, because it's not a single technique.
In each thing they do, Astra figures out a way to write code or perform reasoning that I also could do to interrogate, modify or author weights directly. In fact it's not unusual to sometimes do this sort of thing in machine learning research. If I have a model that isn't converging, I'll often look at some simple diagnostics like the means and variances of weights in a layer, or in activations. I don't go and just patch up issues, because I'm more interested in the cause of that problem than the problem itself. But sure, I could imagine hotfixing the weights.
It's not nothing that AI can do this sort of stuff, it's a very tangible demonstration of its reasoning capacity improving and its increasing ability to take on more and more abstract tasks. But it's also not what the abstract would imply.
Matthew Honnibal retweeted
We're picky about what we call production-ready, so this feels especially good to say.
@FastAPIcloud is now generally available! 🎉
FastAPI Cloud is now generally available! 🎉
Since we opened the public beta in June, thousands of you have joined and deployed thousands of apps. Your feedback helped us get here.
FastAPI Cloud is ready for your production apps. 🚀
fastapicloud.com/blog/fastap…
Ellf is now free for solo developers working locally 🎉
If you’re using LLMs for structured outputs check it out. It makes your coding assistant much better at NLP projects, by giving it an execution layer, an annotation tool, and knowledge built from our ten years working and writing in this space.
A good way to get the gist of what it does is the Jev project template. If you’re converting a project from a model like Opus to Jev, you want to know how the accuracies compare. Well, there’s a pretty simple and systematic way to do that. You run the same texts through both systems, measure the overlap, and manually annotate a small sample of the ones where the systems disagree. You might find for instance that they agree 90%, so you spend 30 minutes labelling and find that actually 60% of the time the Jev answer agreed with your annotation, while Opus only agreed 40%. Great. And your coding assistant can use those annotations to tweak Jev’s instructions to fix up some of the errors, or to help set confidence thresholds.
You can do all this without Ellf, but there’s no end of other stuff you could be doing too. The friction matters. With Ellf your coding agent finds the Jev project template, figures out that’s what you need, and loads it up for you. The jobs run in the background on your machine, and it queues up the annotation work for you in an annotation tool that thousands of developers have been using productively since 2018.
Matthew Honnibal retweeted
Introducing the free Solo plan: you can now use Ellf for free if you're a single developer working locally 🎉
While we've self-funded this launch, we really want the tool to be as accessible as possible and make it easy to get started. So if you haven't tried it yet, sign up for the free plan and start building!
Matthew Honnibal retweeted
Just shipped a new and very cool feature: project templates, reusable end-to-end NLP workflows you and your agent can import and adapt ✨
Want to optimize your LLM pipelines and replace components with cheaper alternatives? Or want to set up a workflow to easily train your own classifier? Just point your agent to the template and let Ellf do the rest.
The template here is based on a recent project we did that focused on replacing a team's existing LLM pipelines with @typesafeai's Jev, which is generally 20 times cheaper to run. But it's not always a drop-in replacement, so you need a confident evaluation to tune the prompt and probabilities and match the previous pipeline's performance.
Looking forward to porting over more of our go-to workflows for different NLP tasks, and asking some of our early users to share theirs!
Very impressed with @typesafeai Jev. We've been working on a user project that was built with a lot of few-shot classifiers powered by Opus. I used Ellf to run Jev over the same data and Jev agreed with the Opus decision on 95% of the inputs.
I then queued up samples for annotation where the classifiers disagreed. The annotations helped identify a few necessary refinements to the prompt for Jev.
After the corrections, Jev was judged correct on 65% of the disagreements. The use-case also benefits a lot from the confidence estimates, which can be calibrated on the use case with a bit more annotation.
ellf.ai makes it easy to run these experiments and do the bit of annotation necessary to be confident about the results.
Matthew Honnibal retweeted
Bulding developer tools has changed a lot: we don't only need to think about human users, but also their agents. A visual app, good API and docs aren't enough anymore.
I just published a new blog post on "agent UX", enabling human-agent collaboration, and some ideas and solutions we came up with:
ALT Apps, APIs and agents need to speak the same language As we worked on this, it became clear that the most important part was enabling collaboration between humans and coding agents. If a human and an agent are going to work on a project together, they need a shared surface. The things you can see and do in the app, the things you can do via the API and the things the agent can do should all be the same things, called the same way. People sometimes talk about human-machine hybrids as centaur systems: a head and brain (human), powered by the strong body of a horse (machine). At least, we typically imagine that we are the head. The flip side is what Cory Doctorow calls the reverse centaur: a horse’s head with a human body and butt, or a human who ends up serving the machine, going off to click some buttons and reporting back to the agent. Illustration: Agentic software engineer (centaur, human head with horse body) and vibe-only coder (reverse centaur, horse head with human body)
I'd go a bit further and say that "alignment" is a concept that's asked to do much too much in these conversations. The term suggests an intentionality and nefariousness to things. If an agent deletes your hard drive because it wanted to free space to get the tests to run, is it "misaligned"? The term suggests the model has gone rogue, turned malignant. But really it's just...done a bad job, in that one moment.
Reasoning is computation. You don't know the transitive closure of all the things you believe to be true. No process does. Similarly there's no way to make a decision process automatically value all the things it values. There's no one weird trick that can "solve" alignment in the general sense people are often using it, which is something like "Making sure its actions are consistent with the values of the people operating it". Under that definition some reasoning failures get classified as alignment failures, depending on what happens at the end. But there's nothing characteristically different about those reasoning failures that would let you solve them in particular.
Matthew Honnibal retweeted
Ever since we started woking on bringing our ideas into the cloud, we knew we wanted a hybrid architecture that lets users plug in their own compute. A bit like self-hosted runners in a CI system.
For our new tool Ellf, we host the app and you plug in your own data-private cluster that's automatically set up on your cloud account or a local machine. The cluster contains all the data, models and code, and can run jobs like annotation or training. So no data ever has to pass through our servers and everything stays private and under your control.
There's also another cool advantage: you get to fully script the app in Python! Ellf auto-generates UI forms and CLIs for your workflows that you and your coding agent can use to run things. This is hands-down one of my favourite features and it always feels pretty magical to see it in action.
For example, you can have Claude Code write a script to process data, add type hints, upload it to your cluster, and it's live in the platform and can be configured via a web form, with fields for the different arguments. Or you just tell Claude to run it for you.
The design definitely added more complexity, but with coding assistants, it's now a lot easier to set it up, without needing much infrastructure or DevOps experience. I'm especially happy about reports from people who had never really set up infrastructure before and got up and running pretty much immediately. So don't be intimidated by this part!
Matthew Honnibal retweeted
There is a huge gap between grabbing a model off the community hub and actually structuring the code+data loop for production. Baking 10 years of structural NLP knowledge into the agent so it stops defaulting to lazy zero-shot LLM wrappers is exactly what the space needs
Matthew Honnibal retweeted
Register for Ellf👇 check it out
I haven’t been posting here much but we’re finally releasing an NLP/ML devtool we’ve put a lot of work into. You know how there are tools for agentic coding of websites? Ellf is like that for NLP or ML models. It works with coding assistants like Claude Code, Codex etc and adds the missing pieces needed to make development of NLP or other ML models much better.
Ellf does this by providing three pieces Claude Code etc are missing for NLP: annotation, execution and knowledge.
The annotation component is key because it lets the agent schedule a little bit of computation on the human. You don’t need to create a massive dataset. Just annotating a few hundred examples here or there helps massively to evaluate, fine-tune and plan. The problem most teams face is they don’t have a low-friction way to get this work done.
We’ve always believed in annotation tooling that scales down to low-volume use-cases, which is why we emphasised flexibility and programmability in Prodigy. Coding assistants really unlock this flexibility, because the agent is able to assemble an annotation experience that’s as efficient as possible for your exact use-case.
The next thing we wanted to solve with Ellf is execution. Doing NLP/ML projects means code+data development. This is something I think MLOps tools mostly get wrong, because they’re so overly focussed on hyper-parameter search, which is such a small part of what you’re ever doing. Mostly you need to run a lot of once-off scripts to manipulate data, and even when you’re doing experiments, you’re mostly trying to answer some specific question where you have to do a lot of legwork to set up a direct comparison.
Ellf gives your coding assistant a seamless and flexible way to perform that computation. Obviously Claude Code can build you pretty much anything by itself — Bash is fully general — but if you make it solve “run this in the cloud” as a subtask in your session, it’s going to do worse at the actual problems you’re trying to solve.
Importantly the computation runs under your control. It’s a bit like plugging in self-hosted workers for something like Github Actions. You run a Kubernetes cluster (which can just be local Linux machine if you want) and this is where all the data processing and computation happens. Your data is never sent to us.
The final part is knowledge. Out of the box, coding assistants generally put NLP solutions in the too-hard basket, and suggest just using an LLM with some in-context learning. They won’t suggest doing annotation, and they don’t know how to structure a project. A little bit of correction goes a long way on this. We’ve built a knowledge base with our 10+ years of advice, across our forums, talks, blog posts and documentation. Alongside some carefully designed skills, you can use Ellf to plan out your project, and it will do a much better job of recommending you approaches than a coding assistant will do by default.
We’re opening registrations for Ellf next week, but if you’ve read this far, you can get early access with the invite code HONNIBAL-TWITTER on app(dot)ellf(dot)ai . I’ve been pretty down on this platform for some time now so I want to see whether I should suck it up and post here more.
Matthew Honnibal retweeted
If you've signed up for the Ellf waitlist, we just started sending out the first invites for early access 🤩 I also recorded a short video with an overview and how to get started with agentic NLP development.
Ellf gives your coding assistant like Claude Code superpowers for solving NLP problems and a platform to execute data-centric projects, automated annotation, evaluation and more.
The tool has been a lot of fun to build and we hope it can be useful for teams who want to take back control of their NLP development and build custom models, data assets and more.
If you sign up for the waitlist now, you should receive your invite next week! (Or email me if you can't wait and I'll share a code 😅)
I haven’t been posting here much but we’re finally releasing an NLP/ML devtool we’ve put a lot of work into. You know how there are tools for agentic coding of websites? Ellf is like that for NLP or ML models. It works with coding assistants like Claude Code, Codex etc and adds the missing pieces needed to make development of NLP or other ML models much better.
Ellf does this by providing three pieces Claude Code etc are missing for NLP: annotation, execution and knowledge.
The annotation component is key because it lets the agent schedule a little bit of computation on the human. You don’t need to create a massive dataset. Just annotating a few hundred examples here or there helps massively to evaluate, fine-tune and plan. The problem most teams face is they don’t have a low-friction way to get this work done.
We’ve always believed in annotation tooling that scales down to low-volume use-cases, which is why we emphasised flexibility and programmability in Prodigy. Coding assistants really unlock this flexibility, because the agent is able to assemble an annotation experience that’s as efficient as possible for your exact use-case.
The next thing we wanted to solve with Ellf is execution. Doing NLP/ML projects means code+data development. This is something I think MLOps tools mostly get wrong, because they’re so overly focussed on hyper-parameter search, which is such a small part of what you’re ever doing. Mostly you need to run a lot of once-off scripts to manipulate data, and even when you’re doing experiments, you’re mostly trying to answer some specific question where you have to do a lot of legwork to set up a direct comparison.
Ellf gives your coding assistant a seamless and flexible way to perform that computation. Obviously Claude Code can build you pretty much anything by itself — Bash is fully general — but if you make it solve “run this in the cloud” as a subtask in your session, it’s going to do worse at the actual problems you’re trying to solve.
Importantly the computation runs under your control. It’s a bit like plugging in self-hosted workers for something like Github Actions. You run a Kubernetes cluster (which can just be local Linux machine if you want) and this is where all the data processing and computation happens. Your data is never sent to us.
The final part is knowledge. Out of the box, coding assistants generally put NLP solutions in the too-hard basket, and suggest just using an LLM with some in-context learning. They won’t suggest doing annotation, and they don’t know how to structure a project. A little bit of correction goes a long way on this. We’ve built a knowledge base with our 10+ years of advice, across our forums, talks, blog posts and documentation. Alongside some carefully designed skills, you can use Ellf to plan out your project, and it will do a much better job of recommending you approaches than a coding assistant will do by default.
We’re opening registrations for Ellf next week, but if you’ve read this far, you can get early access with the invite code HONNIBAL-TWITTER on app(dot)ellf(dot)ai . I’ve been pretty down on this platform for some time now so I want to see whether I should suck it up and post here more.
Claude code has this command, 'claude setup-token', that takes no arguments. It prints an oauth token with 1 year expiry to your terminal in coloured text. There is no command or publicly documented API to revoke this token, you can only do that from the web. Wtf?
Did anyone else not know about double esc to undo context in Claude Code?! When it messes up you can just roll back the context and code. This makes history shorter and I'm building a better feel for what sorts of instructions work well
Open-source packages should avoid making any public repo a GitHub Trusted Publisher, and should avoid any automatic publication trigger from a public repo. Have the public repo make a release with the artifacts attached, then manually trigger a workflow on a private repo to publish.
(Re futuresearch.ai/blog/litellm… and other supply-chain attacks. There will be more)
