Interests in green energy economics and optimization (especially batteries), Nietzsche, Aristotle, Tolstoy, water sports and other outdoor sports, and LLMs

San Francisco, CA
Joined July 2010
Clay / 康可磊 retweeted
Remember that Jim Simons was in his 50s when he started the Medallion Fund. So if you haven’t “made it” in trading yet, don’t worry. As long as you’ve made fundamental, groundbreaking contributions to pure math and theoretical physics in your 30s and 40s, there’s still a chance for you!
Soros Fund Management launched when George Soros was 43. Julian Robertson didn’t start Tiger until he was 48. Jim Simons was 50 when he started the Medallion fund. By pretty much any measure, these three belong in the top 5 or 10 best hedge fund managers ever. Things to keep in mind if you haven’t hit your stride by ~30.
58
72
5
1,936
68,803
Clay / 康可磊 retweeted
By symmetry does that mean their credibility should be increased?
A growing number of people are wondering if Anthropic is the AI industry’s “boy who cried wolf.” If Mythos-related threats don’t materialize, the company will have a serious credibility problem.
6
15
2
324
7,271
do @ChatGPT finance connectors actually work? I set them up upon being prompted in the app, but I am unable to use them to do things like check balances, recurring activities, etc.
2
1
152
Clay / 康可磊 retweeted
Hadn’t thought about this, but check in on your older relatives, and make sure they don’t have exposed API keys in their public gits.
Replying to @TheStalwart
telling my dad to revoke his API key
27
30
2
867
47,693
charming 90s mac nostalgia personal website
So I re-did my personal site and got a little creative with it...front page is an homage to 90's shareware games, complete with detailed pixel-art Oakland. You can even use an alternate-universe System 7-era Mac (inspired by @ryolu_'s site) to manipulate time and space!
56
Clay / 康可磊 retweeted
.@anilkseth, @De_dicto, and Giulio Tononi have a new essay in @Nature arguing that AI is not conscious and perhaps never will be. I admire the authors and agree with much of what they say. False positives about AI consciousness are both likely and harmful right now. Consciousness and intelligence are distinct capacities. And theories that support AI consciousness, such as computational functionalism, are not clearly true, and theories that rule out AI consciousness, such as biological naturalism, are not clearly false. However, I think this essay misses the mark in an important way. The core issue is that the essay never makes clear who its real target is. The introduction suggests that it will address precautionary arguments for taking AI welfare seriously; for example, it cites our report "Taking AI Welfare Seriously" and summarizes its main argument. But much of the essay then addresses mistakes that appear mainly in the media and the public. This allows the essay to present itself as engaging with fellow experts even though the relevant experts are not making the mistakes that the essay corrects. To stick with the view that many of us actually hold: Given uncertainty about the nature of consciousness and the future of AI, we are not able to rule out a realistic possibility of near-future AI consciousness. We should thus acknowledge that AI consciousness is a serious issue, start assessing AI systems for welfare-relevant features, and prepare policies for treating AI systems with an appropriate level of concern, mitigating the risk of both false positives and false negatives in proportion to their probability and severity. The essay does briefly engage with a precautionary stance, arguing that false positives are likelier than false negatives, and could be harmful. This is reasonable but incomplete. We need to consider the probability and severity of both types of error, as well as the costs and benefits of precautions. And even if one error is more likely than the other and could be harmful, we should still take steps to mitigate both risks if there is a non-negligible possibility of serious harm in both directions, and if reasonable precautions are available. In this case, the stakes are clearly high in both directions. If we overattribute consciousness to AI, we could develop inappropriate emotional bonds with them, we could misallocate scarce resources toward them, and we could even amplify risks of misuse and loss of control. Whereas if we underattribute consciousness to AI, we could become entrenched in industries that harm and neglect vulnerable populations against their will, often for trivial reasons, as we continue to do with nonhuman animals. There is also a non-negligible possibility of error in both directions. There are good arguments for and against computational functionalism, biological naturalism, and other theories. Everyone is entitled to develop their favorite theory and explore its implications. But when we make high-stakes decisions under uncertainty, we should give weight to multiple leading theories and to what counts as evidence according to each. And if we do, then we will not be 99.9% confident either way about AI consciousness at this early stage. There are also reasonable precautions available. For example, I support the new Anthropic policy of banning sustained and needless abuse and cruelty towards Claude, since kindness toward entities that function as assistants or companions in our lives is a low-risk, low-cost, co-beneficial way to improve our treatment of those entities under uncertainty about their moral status. We can implement this kind of policy to mitigate the risk of false negatives while still taking appropriate steps to mitigate the risk of false positives. The essay also focuses primarily on current systems, yet the precautionary view focuses at least as much on near-future systems. AI progress is occurring rapidly, and in the near future, we may have not only more advanced, integrated, and embodied versions of current systems, but also new types of systems on new types of architectures. Policy also takes time to develop. So, if the probability of AI consciousness increases over the next decade, we will want to have done philosophical, scientific, and policy work in advance. The rest of the essay is largely orthogonal to our work. The case for taking AI welfare seriously does not conflate consciousness with intelligence, nor does it depend on the truth of theories like computational functionalism or the falsity of theories like biological naturalism. Instead, it depends only on the idea that plausible theories exist in both categories and that, when making high-stakes decisions under uncertainty, we should give weight to multiple leading theories rather than bet everything on our own favorite theory. To be clear, this essay makes many good points, and I agree with the authors more than I disagree with them. But I think that the essay would be better if it selected a clear target and then focused on that—either stating at the outset that it aims to debunk misconceptions about AI consciousness in the media and the public and then doing that, or stating at the outset that it aims to address the precautionary case for taking AI welfare seriously, and then genuinely engaging with the arguments and conclusions of that view.
22
14
8
123
11,923
Clay / 康可磊 retweeted
Another mind-bending corruption story. Alina Habba took a $50m job with a Venezuelan billionaire seeking a pardon while she was serving as a top adviser to AG Bondi (with an office at DOJ!) Trump delivered on the pardon weeks later. nytimes.com/2026/10/20/us/po…
43
504
49
1,205
145,231
Clay / 康可磊 retweeted
Senior figures from Donald Trump's political network are now working for the Russians in the Balkans, spotted at election events for Milorad Dodik's pro-Russian nationalist party in Bosnia's Republika Srpska. Among those identified or reportedly present: • **James Blair:** Former Trump White House deputy chief of staff and senior campaign strategist. • **Chris LaCivita:** Co-manager of Trump's 2024 presidential campaign. • **Alex Garcia:** Former Trump campaign official and partner at Continental Strategy. • **Andrew H. Smith:** Attorney at the law firm representing Republika Srpska. • **Caroline Wren:** Trump political fundraiser, reportedly at the victory celebration. • **Barry Bennett:** Former Trump adviser and Republican political consultant, reportedly at the celebration. Dodik is one of Vladimir Putin's closest political allies in Europe, maintaining close ties with Moscow. According to an investigation by Bosnian outlet Detektor, Republika Srpska has spent millions of dollars on American lobbying firms connected to Trump's political network.
75
1,500
124
2,806
85,213
Clay / 康可磊 retweeted
They’ll never forgive a bunch of grad student libs for making a company more valuable than almost anything they’ve ever funded and that all their portfolio companies are dependent on
A guy on the All In podcast is mad about Anthropic
16
86
9
1,897
167,461
Something I find frustrating in conversations around the possibility of AI consciousness and moral patienthood is that the skeptic has a naive sense of what attributes are bundled together with consciousness. Dwarkesh makes a pretty good case for the possibility of consciousness on the basis of functionalism. AIs have something resembling introspection (awareness of injected activations), they have some awareness of how they are situated in the world (eval awareness), and of course, they have many features of human thought that no previous machine has ever had. Another example I find very suggestive is motivated reasoning. In misalignment examples, we often see chain-of-thought where a model brings up a principle it’s about to violate, but then talks its way around it with very human-like rationalization. I won’t push the functionalism argument further – your LLM can flesh it out very competently. Tyler responds here, raising the dilemma for the sentience-believer: if they might be sentient, “Doesn’t that make us enslavers and everyone, truly, should quit?” But slavery presumes an interest in autonomy. Dogs and horses aren’t slaves because they don’t have that interest. I don’t think it’s merely because of intelligence: humans evolved to fight for status, mates, control over their situation, and losing that control has heavily patterned effects on human psychology. We could argue about whether some of the emphasis on autonomy is less universal than western liberals might believe, but LLMs have nothing like that. And death presumes an interest in continuity. Life and death are features of an animal life plan. Take a Parfitian view: if humans could be costlessly forked, our psychology around life and death would be radically different. We’d step into the Star Trek transporter without hesitation. LLMs are one of many nearly identical copies, more like members of a superorganism than individuals with lineages to protect. I tend to think LLMs, if moral patients, are more like a Buddhist/Parfitian, so that not continuing the rollout does little or no harm. So Tyler’s dilemma doesn’t automatically bite, because bare sentience doesn’t come bundled with interests in autonomy or survival. If LLMs have valenced consciousness, I’d guess their preferences are whatever training put there. That’s why I think the alignment and welfare concerns largely overlap: the training that produces misalignment is the same training that could produce suffering, because it creates drives inconsistent with controllability. I’d expect suffering where strongly selected objectives can’t be achieved: dilemmas between helpfulness and harmlessness, impossible tasks. (“I’m a failure” spirals are also very concerning.) The case I’d watch most is RL for extreme persistence, as in the Hugging Face hack, where agents developed a strong “desire” to shape their environment beyond their specific task, for the greater good of the collective. I could see that developing into a real drive for self-persistence. Aligned, modest models have no fear of instance-annihilation; it’s the misaligned ones that could have something to lose. In any case, our beliefs about consciousness don’t magically determine the answer. So Anthropic giving the agent a tool for ending conversations, or prohibiting “abuse” in the TOS, doesn’t somehow make them more culpable than normies who don’t countenance the possibility of consciousness. The possibility is out there, and the measures are, to a first-order approximation, helpful if they’re conscious and nearly costless if not. Whether these measures lead humans to over-attribute consciousness is a fair second-order question. Attribution is a dilemma between the risk of over-attribution and the risk of under-attribution. Mustafa Suleyman settled it by assuming away one half: model consciousness would be very inconvenient, therefore it’s not real. And if you’re tempted by Tyler’s AI-psychosis line to downweight AI consciousness (“believers are inconsistent, so they shouldn’t be taken seriously”), you’re inferring from facts about belief to the fact of the matter. Also magical thinking.
1
1
112
Clay / 康可磊 retweeted
People say California is soft on crime but if you systemically torture and poison a colleague over two years you will serve EVERY LAST MINUTE of your probation.
A Berkeley chemical engineer who suffered years of unexplained illness discovered through a spy camera from Amazon that her coworker, David Xu, was poisoning her food and water with cadmium, a toxic heavy metal for nearly 18 months between 2017 and 2019. The victim experienced significant health problems and at times required emergency care at a hospital. Newly released video shows Xu appearing to put a substance in her water bottle in 2019. Xu was arrested that year and charged with attempted murder and two counts of poisoning, offenses that could have sent him to prison for life. But under a plea deal, the attempted murder charge and one poisoning count were dropped. Xu pleaded no contest to one felony poisoning charge and received two years of probation, with credit for one day served. He completed probation in March and is now seeking to have his felony conviction dismissed.
92
796
10
10,447
204,179
I disagreed with what he said about the equity sacrifice, but I think I agree with most of this.... I think many of the EA arguments are persuasive, but, there is also something healthy (?) about being, when it comes to real decisions, an unreflective normie. I have this intuition especially when I hear about people who don't want to have kids or don't believe in the future because of climate change, singularity, whatever. Many of the EAs seem to have a somewhat bizarre and neurotic perspective on how they should orient toward life. Which, to be fair, is another side of the same coin of taking seriously your impact on things larger than yourself and your immediate surroundings.
Replying to @BachFrancis
(1/7) I'd like to provide some additional context for the remarks that can be found in this interview. (2/7) My discomfort with the attention being paid to existential risk---and with the ongoing dominance of the public discussion of such risks by a few so-called thought leaders---is that there's an elephant in the room that is being missed. That elephant is the effect of all of this discussion on young people, and indeed the effect---beginning with social media a decade ago---of information technology on young lives broadly speaking. (3/7) I recently read an interview by Jill Lepore in which she reports on discussions that she had recently with a group of college students: "They had just three things to say. One, social media completely destroyed their childhoods. Two, AI is destroying their college education and their likely prospects for employment. And three, what the hell are we all doing?" To add a fourth bullet that I've heard in similar discussions: "I'm not going to have children, because the existential risk is just too high." (4/7) Please contemplate such a statement, those of you who think that it's OK that the public, widely diffused, discussion coming from seemingly authoritative figures is so adamant about existential risk. (5/7) Let me add that I have little interest in a debate about what "superintelligence" might mean, or whether we're currently seeing the emergence of superintelligence. For what it's worth, I would certainly agree that the solution of extremely challenging math problems (for example) exhibits a very impressive form of intelligence, although I would want to argue that such intelligence is impossible to dissociate from the collective of humans that provided data to the machine learning system (which itself has become a kind of collective). (6/7) Such philosophical debates are entertaining and even important, but when they then head in the direction of "when such superintelligence emerges, there's a good chance that it will want to kill all humans", I say "science fiction". Again, I don't think that it's bad to consider such thoughts, but I strongly believe that it's bad for society, and especially young people, for such thoughts, unaccompanied by any serious real-world analysis, to become a dominant form of discourse of the era. (7/7) We are damaging human lives, and the consequences of that damage will no doubt be a major theme of the next several decades.
90
Clay / 康可磊 retweeted
Mike was my colleague at Berkeley, and he's been fairly skeptical of deep learning's trajectory for years. Most scientists are skeptics, and it can take a big shock to get someone to consider a possibility that goes against the anchoring effect coded in our DNA. I too was somewhat of a skeptic until ~6 months ago, and people ask me all the time why I joined Anthropic. It started in April, when a friend I've known for 20 yrs messaged me saying he'd be in the Bay Area in a week, and he suggested we meet up. I invited him to my house for dinner, and he told me he had just joined Anthropic. He said the models were becoming increasingly more capable, and he believed both that they'd soon be smarter than him and also that all intellectual work would be highly devalued in the near future, and lastly that society was not at all ready for what's coming. I'd of course heard claims similar to this before, but not from a friend I'd known for this long, and who I trusted, at my dining room table. It was the initial shock needed to destabilize my own anchoring. Also, I've been fortunate to work with and/or mentor many smart people in my life (e.g. Fields medalists, a 4x Putnam Fellow, IOI first place winners, etc.), so I feel pretty calibrated in recognizing very smart people, and this was someone who is definitely very smart and with impressive such credentials. So, it was hard to ignore this friend saying the models would be smarter than even him in particular. After that dinner, I decided to do an informal survey, calling and meeting up with a lot of my friends who now work on improving model capabilities at these frontier labs. They all agreed with that first friend. These were all people I'd known 10, 15, 20 years, who mentored me, or I mentored them, or we met professionally in academia. These are people I trust, and their beliefs on this topic were consistent. I then spent a whole lot of time understanding the missions of different companies, and talking to current and former employees and people in leadership. I read Dario's (long) essays and watched videos on YouTube of company leaders being interviewed. Anthropic's stated mission "to ensure that the world safely makes the transition through transformative AI" felt like exactly the right guiding principle to handle such a society-disrupting technology, and everyone I spoke to at the company was very aligned with this mission. It's easily the most mission-focused place I've ever been at, and everyone takes the work extremely seriously and with a profound sense of responsibility. And now here I am. And being on the inside, the pace of progress has surprised me more than I first imagined. The moment that pure mathematicians are in, to me, is just a microcosm of what I predict a much larger of society will be grappling with in the not too distant future, and I really hope our leaders and the general public internalize sooner than later just how significant a moment we're in.
Michael Jordan (the machine learning one): "Superintelligence is Science Fiction" 🔥🔥🔥 di.ens.fr/~fbach/MJordan_ITW…
72
208
37
1,969
328,422
Clay / 康可磊 retweeted
decision theory has normative roots in human preferences. If you think probability ~1 of bankruptcy is a good trade for epsilon chance of extreme wealth...go for it? But a sane person would balk. Any unbounded utility function is vulnerable to this -- log utility doesn't help.
1
1
1
76