
@WisconsinCS ‘26 / Prev AWS / UW Research Train in systems @getcrackedio
Joined December 2025
- Tweets402
- Following189
- Followers91
- Likes1.3K
Samad Syed retweeted
Will stronger LLMs make harnesses obsolete?
New MIT CSAIL paper argues the opposite: Harnesses compose model capabilities to generalize across longer, deeper, & more strategic tasks, while enabling more data-efficient training: tinyurl.com/2fwk4uap
Samad Syed retweeted
Rashid Bashir, dean of the Grainger College of Engineering and vice chancellor for Chicago strategic partnerships at the University of Illinois-Urbana Champaign, has been named as the next chancellor of the University of Wisconsin–Madison.
news.wisc.edu/rashid-bashir-…
Samad Syed retweeted
We’ve heard again and again that enterprises don’t want to pour all their valuable IP, data and tacit knowledge into someone else’s model. They want control of their own weights.
Samad Syed retweeted
Introducing Qwen3.8-27B-pi-adapted 🤗
We adapted Qwen's model specifically for the pi coding agent using reinforcement learning and a LoRA adapter.
The results, measured as pass@1:
🚀 SWE-bench Pro (Hard 51): 29.4% → 62.7%
💻 Terminal-Bench 2.1: 39.5% → 55.8%
⚡ 55–62% fewer tool calls per task
This is the power of open models: you can adapt them to YOUR tools and YOUR use cases. Let’s build! 💛
Open weights: huggingface.co/harness-hub/Q…
Methodology: huggingface.co/blog/harness-…
Samad Syed retweeted
I remember reading this back when I was in undergrad.
A really good recipe.
hot take: nothing wrong with papers per se, but… here’s the recipe
github.com/okhat/blog/blob/m…
Samad Syed retweeted
oasys lab is shipping research at higher velocity than 99.9% of startups ship product
nice work @lateinteraction - super inspiring. also my vote for best name on the list
Samad Syed retweeted
terminal bench and TB science voted #1 and #2 for "benchmarks trusted most to indicate if an LLM is frontier"
I asked 56 people in AI which benchmarks they'd trust most to indicate if an LLM is frontier:
25 votes: Terminal-Bench 4 (@alexgshaw, @andykonwinski, @Mike_A_Merrill)
20 votes: Terminal-Bench Science (@StevenDillmann, @lschmidt3, @sanmikoyejo)
13 votes: FrontierCode (@penlume, @ybenpan, @denizbirlikci, @silasalberti)
10 votes: FrontierSWE (@evan_j_chu, @_rajanagarwal, @MatternJustus, @ardnoKhtnayhsiR, @sanket__mhatre, @akshitkr)
8 votes: Humanity's Last Exam (@longphann_, @hendrycks, Alice Gatti, @ziwen_h, @natliml)
7 votes: Vending-Bench (@axelbacklund, @lukaspet)
5 votes each:
- AutomationBench (@danielwshepard, @SalimansRobin, @TweetAnnaMarie)
- DeepSWE (@winkey_h, @charleyslee, @LeonardMainnet, @serenaa_ge)
- PostTrainBench (@full__rank, @hrdkbhatnagar, @AmyPrb, @shiraeis, @karinanguyen, @MatthiasBethge, @maksym_andr)
The most common comment was that people don't really trust benchmarks at all.
56 people voted. People couldn't pick a bench their company worked on, an index, or a human-voted board. Disclosure: on this list, I'm a small investor in Abundant, Cognition, Emulated, Fleet, Hillclimb, Latch, and Snorkel. I didn't vote.
Counts are net: a few benchmarks received negative votes.
Continued:
Samad Syed retweeted
Fun Fact: Basically none of the initial railroad companies made money. They overbuilt, took on way too much debt, and ended up in brutal pricing competition with each other, and nearly all went bankrupt.
Samad Syed retweeted
remzi understands that as the AI paradigm advances, the decisions about what we build, and who we’re building for, will impact billions of lives.
he was my undergrad advisor at UW-Madison. i met him and andrea when I took their operating systems class. i remember during section how he was a total master of the terminal (tcsh) and emacs and i promised myself i’d match his skillset some day. i started turning my mouse upside down in the computer labs to learn keyboard commands faster.
after all these years, it’s an honor that i still get to work with @RArpaciDusseau, and also to have him advise @LaudeInstitute, where our open research residents get to benefit from his perspective just like i have.
“The root of academic freedom is to be able to look into the world with clear eyes and to speak the truth. If universities can’t do it, then who will?”
Laude Research Advisor Remzi Arpaci-Dusseau, a longtime @UWMadison professor, is the founding Dean of @uwmadcai. At a key moment for computer science, AI, and the world, he sees universities as well-positioned to challenge ideas and ensure technology serves the public good.
Worth a read: onwisconsin.uwalumni.com/the…
Nice section in this AWS post on ShipStation Global’s AI IT help desk.
A dspy.RLM module holds their entire Slack history as a variable, and mines it into a knowledge base of resolved issues.
aws.amazon.com/blogs/machine…
Samad Syed retweeted
Never felt closer to a fundamental change in how we build retrievers, and a different kind of memory for agents.
ARI is basically test-time training for retrievers, where gradients derived from a reasoning model's preferences teach the retriever where to look *at search time*.
Lots more to do here to get oblique search queries as manageable as non-oblique ones, and that will include first figuring out sub-linear (late-interaction-esque) search.
But the path ahead has never been clearer. Read @dianetc_'s blog!
Is there a way to scale a reasoning model’s judgment for searching a massive corpus?
We introduce the Amortized Reasoning Index (ARI), a new kind of retriever that updates its weights during search, using gradients derived at inference time from the ranking preferences of a reasoning model.
Every session, current agents spend lots of reasoning deciding which search results matter, and then throw all of that judgment away upon compaction. ARI is our attempt to “amortize” that judgment, so that reasoning spent on past queries keeps improving search on future ones.
With no relevance labels, ARI takes a 400M cross-encoder that starts out weak and grows its Recall@100 on held-out queries by up to 7×, ending above Qwen3-Embedding-8B on various oblique/reasoning-intensive search tasks.
Read our blogpost: dianetc.com/musings/ari/
Preliminary results and discussions are below!
Is there a way to scale a reasoning model’s judgment for searching a massive corpus?
We introduce the Amortized Reasoning Index (ARI), a new kind of retriever that updates its weights during search, using gradients derived at inference time from the ranking preferences of a reasoning model.
Every session, current agents spend lots of reasoning deciding which search results matter, and then throw all of that judgment away upon compaction. ARI is our attempt to “amortize” that judgment, so that reasoning spent on past queries keeps improving search on future ones.
With no relevance labels, ARI takes a 400M cross-encoder that starts out weak and grows its Recall@100 on held-out queries by up to 7×, ending above Qwen3-Embedding-8B on various oblique/reasoning-intensive search tasks.
Read our blogpost: dianetc.com/musings/ari/
Preliminary results and discussions are below!
I'm delighted to partner with @LaudeInstitute (along with my @Sanjay_Ghemawat, John Hennessy, Dave Patterson, Ben Gomes and @andykonwinski) to support funding of open science moonshots!
anyone who is close to research knows how broken the incentives are. doing open science at the frontier is harder than ever, compute costs alone price most teams out. we kept wondering what might happen if we give the right resources to the right researchers at the right time?
laude.org/updates/moonshots-…
Samad Syed retweeted
Data labs ranked by number of papers accepted into Neurips 2026
I asked people which data companies influence research direction at the frontier:
16 votes: Surge (@echen)
13 votes: Proximal (@calvinchen, @MatternJustus)
8 votes: Preference Model (@jennifgcrl, @Ning_Catsnail)
7 votes: Datacurve (@serenaa_ge, @charleyslee)
6 votes each:
- Mechanize (pre-acquisition) (@tamaybes, @EgeErdil2, @MatthewJBar)
- Mercor (@BrendanFoody, @adarsh_exe, @suryamidha)
5 votes: Fleet (@nicoup, @andrewthezhou)
4 votes each:
- AfterQuery (@CarlosGeorgescu, @spencermateega)
- Prime Intellect (@vincentweisser, @johannes_hage)
3 votes: Latch (@AlfredoAndere, @kenbwork, @kylegiff)
2 votes each:
- Abundant (@huyouare, @kehuangsf, @meji_abidoye)
- Artificial Analysis (@_micah_h, @grmcameron)
- Bespoke Labs (@madiator, @AlexGDimakis)
- micro1 (@aliansarinik)
- Terminal-Bench (@Mike_A_Merrill, @alexgshaw)
50 people voted. People couldn't pick a company they work at or founded. Disclosure: on this list, I'm a small investor in Abundant, AfterQuery, Arcanum, DLR, Fleet, Latch, Paper Instruments, Snorkel, and Vmax. I didn't vote.
Continued:
Samad Syed retweeted
LLMs hacking are gonna be so much fun man
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math.
rikiyat.github.io/blog/weight-sp…
github.com/RikiyaT/lm_und…
huggingface.co/Qwen/Qwen3.5-0…
huggingface.co/RikiyaT/Qwen3.…
Samad Syed retweeted
Machine Studying on the Weaviate Podcast 🎙️ thanks @CShorten30!
(yall can tell I had a great time there from the thumbnail!)
What should 24/7, "always on" agents actually be doing when nobody is asking them for anything? 👾♻️
I am SUPER EXCITED to publish the 148th episode of the Weaviate Podcast featuring Jacob X. Li (@jacobli99), a Ph.D. student @MIT_CSAIL! 🔥
Jacob's paper frames the answer as Machine Studying. Similarly to how a student prepares for an exam they haven't seen, an agent studies a corpus (code base, drug discovery papers, system logs, ....) without knowing the downstream task! 📚📝
Instead of leaning entirely on inference-time compute, can an agent build arbitrarily deep understanding ahead of time?
I loved the paper's definition of expertise. An expert efficiently uses inference-time compute, measured as the area under the performance-vs-compute curve of a ReAct agent loop. Studying intelligence! 📐
Jacob lays out three bets every studying method makes: synthetic data, weight updates (LoRA, KV-cache cartridges, a trained retriever), and amortized context like notes and cheat sheets. 🛠️
We also discussed why evaluation as a science matters once recursive self-improvement happens in domains with no benchmark at all. 🧪
This was a super fun conversation, I learned so much from Jacob, and I really hope you find it interesting!
YouTube: youtu.be/9iZIl0vZvvk
Spotify: spotifycreators-web.app.link…
Samad Syed retweeted
Senior-SWE-Bench is joining @FireworksAI_HQ’s Specialized Intelligence Index.
Developed in collaboration with researchers from @Princeton and @UWMadison, Senior SWE-Bench evaluates coding agents on realistic, open-ended software engineering tasks, assessing correctness and code quality across 100 tasks from 12 open-source repositories. The tasks cover complex features, migrations, bugs, and performance issues.
Explore the results: fireworks.ai/specialized-int…
This quoted post is unavailable.
Databricks CEO Ali Ghodsi says we already have all the intelligence we need, and the opportunity now is applying it productively:
"I think we already have AGI. I actually think we have achieved AGI. We just need to make it useful."
"Apply it to everyday things that we wanna do, and make it valuable at simple things. That's all we need, and that's what's gonna prove out to be very, very valuable for everything, businesses, companies, organizations."
"We don't need more intelligence, honestly."
@alighodsi (UC Berkeley, 2025)
Read the latest Profiles of the Future by @bhorowitz and @mikemcg0 on Databricks & the Future of the Firm: a16z.news/p/databricks-and-t…
Samad Syed retweeted
My first open model lecture, covering all the many pieces that build the ecosystem, and the open questions/debates determining where it goes next.
How does distillation fit in?
How do we manage risks?
How do we see access to models changing in the future?
I'm excited to revisit these questions about every 6 months, to give a state of the union on the field.
Chapters
00:00 Introduction
01:36 What is an open model?
08:47 Why should we care about open models?
18:52 How competitive are the best open models today?
25:28 How are open models built? The role of distillation
34:52 Are people actually using open models?
41:47 What comes next with open models?
47:32 Conclusions
What questions did I not answer that you have?
Samad Syed retweeted
AWS is sold out. Meanwhile, how much capacity is sitting in idle, over-provisioned RDS instances?
Lakebase architecture is the future: compute scales based on load and it's never wasted on idle databases
Five years into his run as CEO, Andy Jassy has nearly tripled the size of AWS, grown Amazon's revenue by about 80%, and expects to spend $220 billion this year to win AI. Amazon's AI business hit a $25 billion run rate in three years, growing several hundred times faster than early AWS.
Jassy joins a16z's Ben Horowitz and Erik Torenberg to discuss the decisions behind that run, from rebuilding Amazon's fulfillment network after the pandemic to the AWS origin story.
They also cover why data centers became a political fight, Amazon's custom chips, its stakes in Anthropic and OpenAI, why inference needs to get 10x cheaper, AI's most underrated risk, and why every company will have to reinvent its products from scratch, plus Jassy's advice for anyone who's 18 today.
0:47 How Amazon has changed since 2021
1:51 How the pandemic reshaped retail
3:30 Why AI supercharged AWS's growth
5:20 When leadership principles clash
9:36 $60M vs. $25B: early AWS vs. early AI
13:20 How the AI CapEx cash cycle works
15:44 Why the US must build data centers
20:55 The labs-to-enterprise AI barbell
23:12 Demand vs. longtime AWS customers
28:11 Annapurna, Graviton, and Trainium
33:55 Backing Anthropic and OpenAI
37:15 Why AI means smaller teams
42:13 AI's underrated risk: security
46:29 Why Amazon rewired Alexa's brain
53:07 What Jassy tells his kids about AI
56:36 Betting on AI, robotics, and space
YouTube: youtu.be/-YMMRQ1qscI
@ajassy @Amazon @bhorowitz @eriktorenberg






































