Ella (@hc_ella) header
Ella (@hc_ella) profile picture

ai research @withbeaker 👩‍🔬 prev: codegen @Bloomberg / 🤖🚗 @StanfordAILab @SISLaboratory Lyft L5 (@Woven_Toyota) / ML @GoogleAI @Meta

NYC
Joined September 2021
Photo by Ella (@hc_ella)Photo by Ella (@hc_ella)Photo by Ella (@hc_ella)Photo by Ella (@hc_ella)Photo by Ella (@hc_ella)
Nobody enjoys staring at data. Yet, it's still one of the biggest levers in machine learning. Introducing data review in Beaker. While it improves your agent, Beaker (@withbeaker) analyzes your dataset and flags samples that might be broken: correct answers marked wrong, tasks that contradict their checks, insufficiently specified rubrics, etc. Try it on your agents and datasets: withbeaker.ai/blog/before-yo…
1
2
1
17
1,142
What if the failure you're trying to fix isn't a failure at all? Faulty evals waste engineering time and can teach agents the wrong behavior. @withbeaker checks the eval while improving your agent. On AutomationBench: ~14 percentage points of training-score improvement per run, plus 8 evaluation concerns flagged across two runs. On LiveNewsBench: 20 questions flagged, including 6 confirmed grader errors. Before you fix the agent, check the eval 👇 withbeaker.ai/blog/before-yo…
3
2
3
8
1,264
Make your agents faster (and cheaper) ⚡️
We made an AppWorld agent 36.6% faster and cut its model cost by 34.6%, while keeping a similar goal completion score. The model stayed the same. @withbeaker autonomously tested changes to the agent’s prompts and tools: fewer repeated lookups, shorter instructions, less unnecessary context. It then checked the optimized agent on tasks it hadn’t used during optimization. 2h24 from baseline evaluation to final results. I wrote up what changed, with the results and the full run 👇 withbeaker.ai/blog/speeding-…
2
8
377
Last week, @rikiyatakehi fixed a looping model by editing its weights. We pointed an AI agent at the same problem. It never ran the model and autonomously found a different fix. No training run, nor gradients. That's weight-space programming. We gave the same task to Beaker, @withbeaker, our autonomous agent-improver. It analyzes how an agent behaves and fails, formulates hypotheses on what and how to fix, validates or invalidates them, in a loop, to hill-climb a pre-defined score. The target: Qwen3.5 0.8B, which fails 93% of MATH-500, often by getting stuck in loops until it runs out of thinking tokens. The model has a tendency to generate "Wait", and Beaker turned it down. From the weights and the scores of its own edits, it figured out what was going wrong and autonomously fixed it by editing 1,024 numbers, a millionth of the model's parameters. MATH-500: 7.4% -> 37.4%. That's a different fix from the original blogpost. They found a way out of generation loops, and Beaker found a way to not fall into them. Theirs is stronger alone (51.1% in our harness), but the two stacked together compound: 58.7%. When properly harnessed, less capable models are good weight space programmers. Beaker powered by GPT6.1 Sol found the same neurons as Beaker with GPT6 Astra, at $4-16 per run vs $28-79. We also pointed Beaker at new goals. We told it to tune the weights such that the agent comes up with shorter answers, and it did, with answers a third to half shorter. Beaker already tunes prompts, skills, memory, tool implementation, orchestration, etc. to improve agent quality, latency cost and determinism. Weights are the next knob it has access to, coming soon. Sign up at withbeaker.ai Blog: withbeaker.ai/blog/weight-sp… Tuned-weights on @huggingface: huggingface.co/collections/r… Public runs: app.withbeaker.ai/public/run…
17
23
5
140
12,974
Production agents are leaving performance gains on the table. Our autonomous AI engineer, Beaker, took GPT-5.2 from 30% → 48% on AutomationBench without changing the model, beating OpenAI's newest model (GPT-6 Luna, untuned) at 39%. Don't wait for the next model 👇
withbeaker.ai/blog/dont-wait…
Stop waiting for the next model I often hear ai engineers at vertical ai app cos say they need the next model releases to unlock the next step-change in their product capability. What if your current model was already good enough? We tuned an agent's harness around a model 9 months older than the latest one. It beats the latest model by almost 10 points.
1
2
5
460
twitter is the bloomberg of ai research
134
292
34
4,737
104,311
Researchers in our #AI Engineering Group & CTO Office are publishing 8 papers at @aclmeeting this week about agentic systems, #responsibleAI & financial NLP; learn why their results are notable + how their work will advance the state-of-the-art bloom.bg/4eTnAdn #ACL2026NLP
8
11
930
Larry Ellison is right. Models are trained on the same data. The differentiator now is expert human feedback applied to real workflows. Doctors, lawyers, engineers. People who actually do the work. Observing how models fail in production, providing the cognition to fix it, and feeding that signal back into training. That loop is the new moat: observe failure → extract expert judgment → build verifiers → post-train → deploy → observe new failures. Each cycle makes the model better at harder, longer-horizon tasks that general training data never covered. The companies that own that loop will power the next generation of AI.
Larry Ellison $ORCL highlighted something critical: models like ChatGPT, Gemini, Grok, and Llama are all trained on largely the same public internet data. When everyone trains on the same information, models inevitably converge. That’s why AI is moving toward commoditization. The real moat isn’t the model itself. It’s the proprietary data behind it. Companies that can train on exclusive datasets gain an advantage competitors can’t replicate. Having data that no one else has will allow you to dominate your market.
52
97
13
889
196,094
This is awesome! Go team Flint!! 🚀
When the cost of code goes to zero, marketing is your only advantage. Introducing Flint. It builds you a unique page for every ad, keyword, and customer. We’re already doubling conversions for @Cognition and @Graphite. Sign up at @tryflint.
358
Text-to-SQL remains one of the cornerstones of robust agentic systems, especially in domains with low tolerance for error. One insight: treat SQL generation more like software testing. Our agentic framework PExA explores this idea and pushed SOTA on the Spider 2.0 leaderboard
242
I'm excited to share that our paper "EntSUMv2: Dataset, Models and Evaluation for More Abstractive Entity-Centric Summarization" was accepted to #EMNLP2023 ! Please stop by our poster to learn more about our research on Dec 9th at 11am (GMT+8) in the East Foyer #NLProc
Researchers from @Bloomberg's #AI Engineering Group co-authored 4 papers at @emnlpmeeting in Singapore this week; learn more about their research, why the results are notable, and how their work will advance the state-of-the-art in #nlproc bloom.bg/3GTbK2p #EMNLP2023
1
2
681
You can check out our paper here: aclanthology.org/2023.emnlp-…
1
218
Learn more about our paper, "Extractive Entity Centric Summarization as Sentence Selection using Bi-Encoders," and stop by our poster at @aaclmeeting !
Researchers/engineers from our #AI Engineering Group are co-authors on 4 papers at @aaclmeeting this week; learn more about their #NLProc research, why the results are notable, and how their work will advance the state-of-the-art in #NLProc bloom.bg/3XjNsFr #AACL22
6
🎉 #aacl22
Replying to @TechAtBloomberg
(2/4) Congrats to Ella Hofmann-Coyle, Mayank Kulkarni, Jane Xie, @gtcomputing’s Mounica Maddela (@mmaddela1005), & @Daniel_Preotiuc for having their paper “Extractive Entity-Centric Summarization as Sentence Selection using Bi-Encoders” accepted for #aacl22
6
All those songs singing about love...they've should have been singing about Paris ✨😍
8
Stop by our #SIGIR2022 booth to talk with our #AI researchers & engineers about how they tailored #opensource #informationretrieval technologies & built #questionanswering systems from scratch to help @Bloomberg's clients find the information they need, when they need it #NLProc
7
23
Fill in the blanks 🟩🟨⬛️
1
3
My superior sense of navigation was humbled today. Apparently I'm lost without the nyc grid and low latency maps. At least I had time to take this photo while pondering how lost I was
2
10
Riding the tube is a pretty surreal experience after being accustomed to the subway. Wait...no one is worried about being pushed into the tracks? How on earth is it so clean? It's not normal to see someone eating a pigeon? I guess this is why New Yorkers are built different 😅
2
9