Abe (@abraham_jmb) header
Abe (@abraham_jmb) profile picture

2L / SWE / amLaw 100 SA

New York / Dallas
Joined April 2022
Photo by Abe (@abraham_jmb)Photo by Abe (@abraham_jmb)Photo by Abe (@abraham_jmb)Photo by Abe (@abraham_jmb)Photo by Abe (@abraham_jmb)Photo by Abe (@abraham_jmb)Photo by Abe (@abraham_jmb)Photo by Abe (@abraham_jmb)
thinking about starting a legal data labeling lab. my thesis, as pinned, is that in transactional work every specific type of task has to be hill climbed. i’ve been creating custom evals with lawyer clients and have a beat on the dynamic. i know model labs are scaling up legal RL, and i can see from my own usage that law is just as token demanding as engineering. however, unlike engineering, and even finance, i doubt the legal labeling can be off-shored. i’m thinking i would want to start off sub contracting for the major data labels - any thoughts on where to start?
21
6
1
128
21,980
8
11
2
257
32,268
BREAKING: The top 1% of US businesses by AI spending allocated a record $10,199 per employee to AI in October, according to Ramp data. Since January 2026, this figure has more than tripled. The top 10% of firms spent a record $801 per employee last month, up +180% since January. The median firm's average AI spend per employee rose to a record $16, up +164% since January. In other words, the top 1% is now spending 625 times more per employee as the typical company. Meanwhile, AI adoption among US businesses on Ramp rose +1.0 percentage point in October, to a record 56.8%, up from 45.5% in January. Corporate AI spending is accelerating across the board.
140
136
55
1,541
315,103
👀👀
Can AI agents correctly identify periapical lesions on a panoramic x-ray of the jaw? Introducing Periapical Lesions Bench (dylantirandaz.com/projects.h…) 🧵
4
443
“superhuman AI could be just 1.3 years away.”
Now we are talking: An Anthropic researcher says broadly superhuman AI could be just 1.3 years away.
1
3
1
4
756
i know it’s rich coming from a law student but i don’t think biglaw rates are high enough. esp not in adversarial legal. and esp not for s tier attorneys who are top 1% at using ai. value capture is down stream of value add.
I think we will see the $100,000 billable hour before we see the end of the billable hour
10
2
3
87
57,749
fyi i do actually believe this but i have become privy to fact that easiest way to gain lawyers’ respect is to complain about $2k an hour rates.
12
4,418
RL 👐 Legal. all star lineup tonight. come bring your thoughts. topics: 1. the role of RLVR in legal: what's actually verifiable in legal work, and is judgment one of those things or not? 2. harness and context engineering vs post-training for legal tasks: what gets you further today? 3. sovereign intelligence vs out-of-the-box lab tooling: firm-owned or local models vs whatever the labs ship. 4. can complex legal work be saturated without proprietary data, or is the moat the firms' own work product? 5. implications for GCs and law firms: where to focus today and where the puck is going?
Today at 5 p.m. Pacific, join @aaronklaw, @abraham_jmb, @BenSovocool, @zebanderson, and me for a conversation on Legal AI: Quality, Judgment, and Training. For example: - How do we measure the quality of legal AI? - How do we build that quality into AI systems? - How do we train models to produce quality legal analysis and exercise legal judgment? teapaw.org/i/spaces/1VNEJmgKBoMoJ…

5
8
48
3,620
Mercor CEO Brendan Foody (@BrendanFoody) reveals the only 2 durable moats left: 1. Brand 2. Proprietary data “Those are going to be the 2 primary moats to look for in the application layer companies that are able to build durable & potentially legendary companies.”
1
4
1
38
6,300
really good eval for lawyers to pay attention to. midpage (beast product) + opus 5.5 wayyy outperformed opus 5.5 w/just web search. why? becuz opus + web only actually reviews "33% of the cases it relied on." big picture, our tooling and prompting needs to onboard agents into a process that elicits rigor by default. maybe, just like the best way to get a child to whisper is to whisper to them, the best way to get an agent to adopt a rigorous process is to demonstrate it in your own work. midpage.ai/blog/opus-5-5-wit…
9
1
1
64
5,913
Replying to @theo
3
20
2
1,054
35,396
absent emergent properties, hill climbing frontier legal needs seeded tasks (for frontier labs) or a metagrader (for sovereign intelligence). if emergence shows up at all, it prob is models learning to context engineer disorganized enterprise data without being too lazy or stupid. thanks to new wave of general enterprise licensing. but current hill climbing for complex legal work is failing. for 1) you have arguably second rate contract attorneys getting 2-10 hours to complete frontier tasks won't get there when teams of s tier attorneys spend 5x as many hours on the same work and 2) thousands of synthetic derivatives of an already largely synthetic, rushed, environment ain't gonna cut it. the juice is in the full trajectory of a real lawyer's matter work. unless some firms start landing avant-garde client engagements that allow these real life trajectories (ideally ai-forward trajectories) to be properly repackaged into lab tasks i predict a degree of stalling. that makes me pretty bullish on a fragmented industry where metagraders leverage continuous sovereign intelligence and custom models win.
Do people still buy the emergent properties discourse? There exist data vendors that either find public benchmarks on which models do poorly or make benchmarks themselves. These data vendors then get humans to solve these benchmarks (or create simulations that capture the key aspects of these benchmarks) and sell the data to the big labs. This process guarantees that if a benchmark is public, then eventually the frontier models will get better at it. A good rule of thumb is to assume nothing is emergent unless there is significant evidence to the contrary—evidence that can be reproduced by third parties.
3
1
15
1,413
Nice DeepSWE score! Does it survive a lowercase filename? Introducing MaxxBench: 89 DeepSWE tasks with small changes to the instructions or starting code. Same engineering problem, slightly different requirement. Models clustered around 74% on DeepSWE end up 23 pp apart. Good at coding, or good at DeepSWE? 🧵
239
410
306
5,483
950,106
Really looking forward to this discussion and would encourage everyone to save some time next Thursday at 5pm PT for this discussion. If (like me) you have strong opinions on the nature of “legal judgment” this should be right up your alley.
Next Thursday (Nov 12), @zebanderson, @abraham_jmb, @aaronklaw, @BenSovocool, and I will host a live X Space on legal AI: How do we measure the quality of AI-generated legal analysis and judgment? How do we build that quality into AI systems? And how do we train for it? Join us for the conversation!
1
1
6
741
Next Thursday (Nov 12), @zebanderson, @abraham_jmb, @aaronklaw, @BenSovocool, and I will host a live X Space on legal AI: How do we measure the quality of AI-generated legal analysis and judgment? How do we build that quality into AI systems? And how do we train for it? Join us for the conversation!
1
3
2
11
1,741
SFT 🤝 GRPO
Replying to @trycrosby
We paired AI-proposed redlines with attorney edits, added synthetic examples, and had lawyers relabel our validation set. Then: supervised fine-tuning + reinforcement learning, rewarding correct verdicts and, only when correct, good reasoning.
1
8
978
"the most successful people are the ones who get what they want." so heavy token usage can be a decent proxy for an inability to get what you want. if humans stay in the loop on enterprise work, evals shouldn't just test ai on a task; firms should test humans + ai, and see who's on the pareto frontier.
3
1
1
7
1,020
Our research team developed RIFT (RubrIc Failure mode Taxonomy), a framework that evaluates the rubric behind a benchmark score. It computes quality signals from the rubric's content and ties them to a taxonomy of nine failure modes. In controlled tests, RIFT identified injected failure modes with 75.0% accuracy, against 56.7% for an LLM judge. RIFT was applied to 147 tasks from five public rubric-graded benchmarks across finance, medicine, software engineering, professional advice, and reasoning. Missing criteria appeared in 50% of tasks, misaligned criteria in 48%, ungrounded verification in 37%, and redundant criteria in 35%. Bundled criteria (7%) and vague wording (10%) were less common. Read the results from @ankitaich30, @jonathanschloss, Charles Dickens, and @pham_derek: snorkel.ai/blog/checking-the…
5
17
110
6,319
Hot take: demand for AI compute is literally infinite, and it's going to reshape venture capital. Given enough time and compute, agents will run scientific calculations and experiments that produce novel patents, theorems and other IP someone can capitalize on. So there will always be a buyer for leftover compute, purely from an investment standpoint. Saying compute demand is infinite is really just saying that the human appetite for scientific discovery & innovation is infinite as well.
6
7
4
68
4,815