james (@http_413) header
james (@http_413) profile picture

develooper / bullish on a post-scarcity, post-suffering world via ai / building @thereabouts_app so you never lose a new place to try

Joined August 2017
james retweeted
🏆 AND WE HAVE A WINNER! GPT-6.2 Astra defeats Claude Fable 5.2 ♟️ Final Stats: 71 moves | 204,219 tokens | $2.0671 Token & Cost Breakdown: • GPT-6.2 Astra: 70,203 tokens | $0.6052 • Claude Fable 5.2: 134,016 tokens | $1.4619 Astra takes the win using almost half the tokens and costing significantly less. GG! ♟️
Ladies and gentlemen, place your bets! GPT-6.2 Astra vs Claude Fable 5.2 Who's winning this one? ♟️ 🏆 Results at 9 PM IST.
35
36
2
445
19,032
几个月后的 google 模型选择 belike:
221
798
124
13,518
385,054
james retweeted
One of the funny reasons for the difference in vibes between the influencers and the crowd today on 6.2 Sol/Astra is that we gave a few folks access a while ago and then made a lot of changes in the last 3-4 days and I'm pretty sure no re-testing was done within those days and the videos were produced last week. Don't get me wrong, while the models are a hefty jump compared to their GPT predecessors, they are by no means perfect and there are a few things the teams are excited for the models to get better at in future iterations in terms of taste and niceness of use. Love all the feedback and we hear you!
2,177
427
338
14,210
1,702,978
This is the first 100% we've seen on v3 The way we've measured models GPT-1 through GPT 6.2 won't hold up for GPT-7+. We need a categorically new benchmark Three ways to create benchmarks: 1. Out-of-distribution tasks A models knowledge coverage looks like swiss cheese (not exactly, but you get the idea). You can always find gaps in knowledge as you niche your topic. Ex: Animals identification > Mammals > Dogs > Herding dogs > Border collies This is the *easiest* way to create a new benchmark, it's also the *least* robust. Depending how much the labs care, you're only 2-3 months of model releases away from your swiss cheese gap filling in. 2. Increase task difficulty Take an existing domain of tasks which models saturate and make the problems harder. Hire PhDs and experts to come up with problems. This is operationally intensive and harder to execute. You want to avoid cheap difficulty (a larger maze) and lean into rich difficulty (more reasoning required). 3. Measure the frontier This is the highest signal (imo) way to test models, it'll be the defining benchmark for post-AGI systems. Instead of measuring performance on a static set of tasks, measure the rate at which models push the frontier of knowledge forward. The ultimate environment is reality/physics. Net new knowledge here is proof of innovation. However, physics is hard to benchmark with, sythenetic environments will do ok here in the short term. To be clear, this is an incredibly high bar to hold models to. Maybe the highest? All RSI, prosperity, advanced capability questions are downstream of AI's ability to create net new knowledge. Once you have a system that can autonomously create new knowledge in any domain (like human as a collective can) then every other domain is a harness/training session away. TLDR: Static task completion is no longer an adequate way to measure frontier models. Rate of learning is the last hurdle. Net new knowledge (what humans don't know) is the most robust setting to measure this.
GPT-6.2 Astra from @OpenAI on ARC-AGI (Verified): - ARC-AGI-3: 100.0% (SOTA), $4.1K (provider adapter), 70.3%, $24.2K (standard) - ARC-AGI-2: 95.8% (SOTA), $0.50/task - ARC-AGI-1: 98.5%, $0.17/task GPT-6.2 Astra is the first model to achieve a 100% score on ARC-AGI-3.
37
25
9
349
19,633
All of the following can be true: 1. Cyberattack frequency and severity is increasing 2. Not all incidents are cyberattacks, but some are 3. Many incidents are due to rapid AI-assisted upgrades to prevent cyberattacks 4. Everyone is on edge so every outage goes viral now, making the quantity feel more unprecedented than it is
I dug into this a bit. It looks like we're at highs, but not unprecedented highs for major systems failures. We're seeing alarming headlines every day, but a lot of it is recency bias.
8
6
2
75
5,254
james retweeted
People make fun of the “AI will cure cancer” But If you’re advocating for Anthropic to be shut down, you’re also advocating for slowing down cancer research. Anthropic has already helped engineer CAR T-cells with a signaling domain (the part of the engineered immune-cell receptor that signals the T cell to attack cancer cells) 70% shorter than conventional designs, roughly 10× greater receptor expression, and improved tumor cell killing in laboratory experiments. Be Pro AI, be anti cancer.
Genuinely pisses me off how being anti-AI isn’t the norm. Like I hear and see people casually talking about ChatGPT and using it like it isn’t a spawn from the darkest pits of hell
47
27
1
525
15,731
A historic day for pancreatic cancer patients: a drug virtually all of them can actually have was approved across the EU today. Daraxonrasib, after one prior chemo line, approved with or without a confirmed RAS mutation, so no biomarker gate and no waiting on sequencing. In precision oncology, a label that broad is rare. The response rates of this drug seen so far is even rarer in pancreatic cancer. Big progress has been made, while we are working already on the right combinations for the right patient.
The European Commission has granted marketing authorization for our oral RAS(ON) multi-selective inhibitor for previously treated metastatic pancreatic cancer. Read more: revmed.com/media/news/13736/… #ForMediaOnly
8
30
2
201
7,764
james retweeted
Replying to @exec_sum
Amazon marked all Satellites as delivered
126
94
11
7,594
149,526
james retweeted
makes you realize just how outdated the infrastructure is and how incompetent the people who run it are
1
77
Some personal news: Yesterday was my last day at Microsoft. Today, I started a brand new job at the FAA and things are going pretty well!
1,746
13,755
914
241,434
4,116,501
james retweeted
maybe there aren't a million years of evolution between these guys
1
1
85
james retweeted
any ladies out there disgusted by the jensen huang cheating allegations? if you own an RTX 5070 or higher, it’s time to take a stand sell that disgusting piece of hardware to me for $50 don’t let that man’s GPU sit in your house. know your worth ladies
611
3,415
306
36,790
827,422
Jensen could do the funniest thing by just coming out in Trump-style saying “yes, I fuck” and we can all move on
52
42
13
934
34,303
james retweeted
writing code in an interview is now larping in the literal sense. like you are straight up live action roleplaying a historical reenactment of what it was like to be a software engineer.
103
866
70
18,659
306,861
james retweeted
To everyone with a DGX Spark, this is the way.
15
30
1
317
19,955
Replying to @elonmusk @cradrrat
I use C/C++ for all my projects. Debt Badge, Clucks Capacitor, homemade power-wall, sculpting software for my haptic interface. I suppose with AI assisting, the clunkiness of developing in C disappears and the efficiency shines. How soon until the language doesn’t matter?
632
601
572
15,208
1,887,971
Replying to @mark_k
Noticias falsa
1
1
14
james retweeted
Replying to @BubbleBrain
men will do anything but go to therapy and anthropic will do anything but add an image gen model to claude
1
1
4
642
Happy Birthday Carl Sagan 🎂 When asked about the the purpose of sending messages out into the universe when it might take thousands of years to get any sort of answer Carl Sagan said: "There are a lot of things in society that only go one way. Socrates talks to us, we don’t talk to Socrates"
24
127
12
693
27,216
james retweeted
Replying to @KITP_UCSB
the committee:
1
10
644