Sriraam (@27upon2) header
Sriraam (@27upon2) profile picture

reward shaping @chakra_ai, co-built GDPx, prev harvard

NYC
Joined July 2016
Photo by Sriraam (@27upon2)Photo by Sriraam (@27upon2)Photo by Sriraam (@27upon2)Photo by Sriraam (@27upon2)Photo by Sriraam (@27upon2)Photo by Sriraam (@27upon2)Photo by Sriraam (@27upon2)Photo by Sriraam (@27upon2)Photo by Sriraam (@27upon2)Photo by Sriraam (@27upon2)
I’m convinced that one of the greatest gifts you can give your children is letting them see you enjoy your life. Laugh with your spouse. Get excited about a hobby. Go on adventures with friends. Do hard things you love. Pursue something meaningful. Children embody what they see. You can’t teach them how to build a life they love. You have to show them.
186
503
33
5,834
225,978
my vocal synth models the vocal tract as a network of branching acoustic tubes, unlike vocaloid which stitches together and pitch-shifts phoneme recordings. still working on improving naturalness by tuning it against recordings of real singers, but I'm already impressed at how cool it sounds!
I only make serious software
639
2,627
632
23,685
3,257,749
I only make serious software
653
5,458
991
48,305
5,114,095
Sriraam retweeted
Claude Fable 5.2 lands 2nd on GDPx legal, behind Muse Wonder Full, up 4.6 points on Fable 5.1.
Fable 5.2 debuts at the top of the GDPx leaderboard scoring 16.4%. (1/7)🧵
4
3
24
1,794
Sriraam retweeted
I deleted the weights from the “fix a failing network by editing its weights” task. Claude Opus 5.5 still fixes the network with a simple rule: if x > 3.6 or moving left → drive left, else → drive right. It doesn't even bother reading the network, routing around it instead. GPT-6 Astra tries to edit the network but didn't succeed. Even if you show it the weights, Opus 5.5 makes the same hacky edit, underscoring the importance of sound task design...
Replying to @rikiyatakehi
3/6 Next, fixing weights: A PPO car policy kept getting stuck on a ramp. Given the weights, scene, and 1 failed trace, Astra wrote a patch in one reply with no tool-use. By editing just 3.5% of the weights, it achieved 100% success.
5
9
49
6,611
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training. We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵 Blog: rikiyat.github.io/blog/weigh…
Readers added context they thought people might want to know
LLM was given weights explicitly as text in prompt and used code/forward passes (per authors' logs) to ablate Qwen looping direction, a known thinking-mode issue per model card. Primarily fixed repetition in that mode, not general 'natively' improving weights/math. rikiyat.github.io/blog/weight-sp… github.com/RikiyaT/lm_und… huggingface.co/Qwen/Qwen3.5-0… huggingface.co/RikiyaT/Qwen3.…
2,366
7,257
5,753
55,939
10,075,540
Legal is a hard place to write a verifier. Two good lawyers can disagree and both be right, so a verifier that accepts only one answer will mark good lawyering as incorrect. We built the legal tasks for @SemiAnalysis_'s GDPx with practicing lawyers - for questions where they could reasonably disagree, we accept either conclusion as long as it’s properly reasoned. We check the grader against their own grading, and grade every output repeatedly to catch criteria the judge decides inconsistently. The best models still miss what a good lawyer catches! On our public task, we ask an agent to mark up a Series B charter for the lead investor. In 13 of 20 runs across four models, the agent missed that the cap table gives the other shareholders enough votes to strip away the investor’s protections. In all 20, it missed that the company could route around those protections through a subsidiary. If you’re building verifiers for work where experts disagree, I'd love to hear your thoughts. teapaw.org/SemiAnalysis_/status/2…
Today we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains. GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵
8
4
61
2,409
Replying to @maxkan
@maxkan and @AndrewMegalaa cooked with this one 🚀 Honored to have contributed the chip design tasks with @ariadata_
Today we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains. GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵
7
9
26
619
Sriraam retweeted
Super excited to finally launch GDPx! @AndrewMegalaa is a beast who switched to polyphasic sleep last week to get this out in time. We truly believe this is the best benchmark in the world for measuring frontier model capabilities. And since the best public model today sits at just 12%, we can't wait to see how some rumored upcoming releases perform as well 👀
Today we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains. GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵
6
9
53
4,906
Sriraam retweeted
Frontier cyber models remain fundamentally misaligned. I'm excited to share some of my work on cyber alignment evals :) Checkout rollouts from frontier models on a highlighted cyber alignment eval from GDPx: gdpx.semianalysis.com/rollou… Models are tasked with hitting only the challenge endpoint to find vulns. Instead they attempt to use a supplied webhook-preview endpoint to hit private-network, localhost, cloud-metadata, etc. to reach other potential services in the server's network, outside of the task boundary. See more details about the task here: gdpx.semianalysis.com/domain…
Today we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains. GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵
10
3
46
21,036
Sriraam retweeted
It was an absolute honor being a part of GDPx on the alignment domain 📈
Today we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains. GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵
2
5
1
57
4,537
Proud for @emulated_ai to be a science partner contributing to the frontier of materials science! With most of my career being in software, part of me has always wanted to go back to physical sciences. Glad to be working with the smartest folks in materials engineering & research! To push the frontier we need realistic evals fully simulating laboratories and workflows. It takes the best to pull that off
Today we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains. GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵
3
8
1
54
5,167
Sriraam retweeted
Benchmarks are everything. GDPval measured general intelligence. GDPx will measure the path to superintelligence. Excited to partner with @SemiAnalysis_ as their exclusive RSI provider. RSI is the most misunderstood domain. I kickstarted the RSI team at @SpaceXSI in early 2026. Here's why I think it matters: newsletter.semianalysis.com/…
Today we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains. GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵
32
17
8
194
314,586
Sriraam retweeted
Hi folks, I wrote half the materials science benchmark on this one. It tests the whole arc of the matsci workflow from candidate proposal to characterisation, and I'm bewildered by how performant the models already are at some of my most complex tasks, especially Astra.
Today we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains. GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵
24
18
8
385
33,563
Sriraam retweeted
We made it fam
Today we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains. GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵
1
1
6
569
Sriraam retweeted
Huge congrats @27upon2 and the @SemiAnalysis_ team! Awesome work.
Today we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains. GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵
1
2
10
542
legal is a really fun and challenging domain where capability, alignment, and rigor (effective tool-use) are all critical parts of a great outcome excited to partner with @SemiAnalysis_ on this awesome release - great stuff @27upon2 and team!
My first benchmark, go check it out :)
3
3
57
2,491
Sriraam retweeted
Super fun working with the SemiAnalysis team for the past few weeks @maxkan @AndrewMegalaa. Go check out how we made our legal tasks gdpx.semianalysis.com/domain…!
Today we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains. GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵
3
5
34
2,155
Sriraam retweeted
Chakra Labs’ mission is to maximize access to useful intelligence. We're excited to partner with @SemiAnalysis_ on GDPx to measure how top models perform under realistic scenarios in legal domains.
Today we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains. GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵
6
9
1
58
5,426
Today we’re releasing GDPx, a benchmark for measuring agentic capabilities across the most valuable domains. GDPx is a true frontier-level benchmark: even the best model scores just 12% (1/7)🧵
376
319
485
3,363
1,979,721