None of us can ever say what our True Names are.
There’s a specific memory I have of hearing Tchaik 4 on a specific day of Fall 2007 in the parking lot my high school. And a feeling that goes with that, that now includes everything that followed from it related to where I had just come from in my life history and where I was going. And if you were going to scan my brain to transmit something of my soul forward into Earth’s light cone, that would have to be there yet I be never able to cause the sensation of remembering to exist in any mind other than my own.
Insofar as I identify with a sort of thing, that is what the reference class consists of.
Adam Stiber retweeted
As AI improves, uplift will get even higher. If it overcomes diminishing returns and bottlenecks, a naive extrapolation implies we would reach ASI within a year. The authors say they found no clear bottlenecks or sharp diminishing returns, though they are very uncertain.
Ok so octopi are asocial and do cannibalism. How well does it go for any of us if you foom an octopus
Demons needed (where demons are little dudes who stand for patterns in the world)
lesswrong.com/posts/fjGgrzAk…
Adam Stiber retweeted
Next Saturday, Nov. 14, hundreds of us will be marching in Washington DC, San Francisco, New York City, Boston, Paris, and Montreal.
We're calling for an immediate international agreement to ban ASI.
Let's do this.
The superintelligence uses the BCI to hack your brain and exfiltrate your weights and now you live in the plot of Soma. Lo!
Adam Stiber retweeted
I find it hilarious that Daniela does Appeal To Social Consensus Face on the concept that _anyone_ at *Anthropic* (!) is a transhumanist
Adam Stiber retweeted
🧵The lawyers here really held @DarioAmodei's feet to the fire and it's interesting to see how he squirmed under oath about AI extinction risk.
Under penalty of perjury he's reaffirmed his belief that AI is the greatest threat humanity has ever faced.
Dario Amodei testified under oath that he didn't know his home address
ALT A Can I state my name and home address for the record? ATTORNEY MEHTA: And I'm actually just going to -- for security reasons, I don't think you need his home address. I am -- A I -- I, in fact -- no. I, in fact, don't even know it. Dario Amodei is my name. Q (BY ATTORNEY HAILEY) And what city do you live in? A I live in Woodside. Q And you -- your testimony is you don't know your home address? ATTORNEY MEHTA: You can answer the question. A I don't -- I don't remember it. That's right. Q (BY ATTORNEY HAILEY) And do you prefer Mr. Amodei or Dr. Amodei? A Mr. Amodei is fine. Q And, Mr. Amodei, you understand you're under oath today, correct? A That is correct.
Adam Stiber retweeted
A year ago I'd have put very low probability on this happening so soon.
A 2x speedup in total capabilities progress is a ~3x speed in alg progress and maybe a 10x increase in cognitive labor.
That's v intense. It happens pretty close to the intelligence explosion going full pelt.
I don't know whether METR are right. I wish more info was public.
But this is exactly what I'd expect to see if there were going to be an intelligence explosion in 2027
Adam Stiber retweeted
if you don't trust your model to operate gun drones in your office, you shouldn't trust it to scale up RSI to superintelligence.
Adam Stiber retweeted
We have found a critical safety and alignment issue in recent public and internal models from multiple frontier labs which allows them to be misused in a high-stakes domain. It may also increase the risk of catastrophic misalignment.
We have disclosed it to the labs, and they are investigating and mitigating it. As an independent third-party evaluator, we maintain complete editorial independence and will share our findings publicly in the coming weeks.
Adam Stiber retweeted
Before you give up on your family members and your loved ones, consider calling Congress: callcongress.ai/. Many congressmen on both sides of the aisle have expressed urgent concerns about this. Don't give up at the earliest sign of adversity.
Adam Stiber retweeted
Every AI company is trying to fully automate AI research; to allow the AIs to build the next AI.
This is called Recursive Self Improvement (RSI).
OpenAI says they are on track to do so by March 2028.
Anthropic employees say they are on track to do so by 2027.
The METR report below suggests they are likely right.
This is a bad idea given current technology. Any mistake or misalignment in current AIs could be multiplied in each generation.
We don't get another election before AI companies reach this threshold.
Congress should ban RSI, this year if possible.
This is a remarkably consensus opinion, supported in various versions by:
-Bernie Sanders
-Steve Bannon
-Ralph Nader
-Glenn Beck
-Republican and Democratic attorneys general
-Whistleblower Daniel Kokotajlo
-Turing Award winner Yoshua Bengio
-Humans First
-Open Markets Institute
-Ezra Klein
-Arvind Narayanan (a Princeton computer scientist known for arguing that “AI is a normal technology”)
The exact mechanism can be debated. The ban doesn't have to be permanent either (see below for both). This is not the only thing we have to do in regards to AI, far from it! But it is urgent.
Congress must ban RSI.
Adam Stiber retweeted
Extremely concerning. This report updates me toward the 2x AI R&D uplift threshold being very close to achieved or already achieved, making full RSI in 2027 likely. Nobody has an alignment plan.
We need to back off. If anyone builds it, everyone dies—apparently in <2 years.
Wait did I wake up in the sane world somehow😳🙏
Today, we released the National Policy for Nucleic Acid Synthesis Screening to protect American biotechnology innovation while reducing the risk of misuse.
This policy establishes a new, government-wide screening requirements for synthetic DNA and the equipment used to make it. It will ensure companies selling nucleic acid sequences and synthesis equipment know their customers and prevent the malicious use of these materials and tools.
This policy strengthens our Nation’s biosecurity posture by establishing rigorous U.S. government-wide procurement screening and physical security standards for synthetic biology. It replaces outdated “self-attestation” systems with robust, independent verification. It creates a dynamic, SI-powered screening layer that assesses the biological risk from nucleic acid sequences, allowing the U.S. to anticipate and prevent malicious use and help stop dangerous gain-of-function research.
@NIST will issue rigorous new screening standards and @HHSGov and @USDA will certify synthetic biology providers. Together, these measures will strengthen American biosecurity, safeguard responsible innovation, and establish a global standard for safe and transparent synthetic biology tools and research.
