"semi-intellectual internet personality" thinking about cultural macromutations ai evals @elicitorg / blog fellow @rootsofprogress
Montréal
Joined October 2015
- Tweets11.9K
- Following856
- Followers3K
- Likes25.4K
I'm sure all the anti-MAID people in here will massively chill out now, right?
bbc.com/news/articles/cqd09g…
why do sunscreens say to apply 15 minutes before sun exposure. like what happens concretely on my skin if I go into the sun before the 15 minutes are up
One of the flaws in Claude’s writing is that it tries too hard to avoid the passive voice. “A missile crossing an ocean cannot depend on a radio link the enemy can jam” no. Reads better as “A missile crossing an ocean cannot depend on a radio link that can be jammed by the enemy”
the best cities I've been to in North America out of quite a few are New York, Mexico City, and Montreal. Probably not coincidentally, they are the cultural and economic capitals of the three main language groups of the continent
I don't think the "classical" building here is clearly aesthetically superior to the modern one, but it's newer, and that's good. Distaste of modernism probably comes in part from a lot being old! And replacing old buildings with new ones in current style used to be normal
I have a folder of images organized by specific aesthetics and I asked Claude Code to analyze it. Apparently my taste is "inhabited optimism". I like that!
This is a puzzling comment I just got and I can't reply to it, but if I am parsing it correctly it says I am wrong to say the pool on the left is fictional because it appears in the movie Cocoon (1985). But the pool in that movie (right) looks nothing like it???
Today in Tunis I saw that grinning Punic mask, the one that people here on Twitter bring up every now and then as “mocking child sacrifice victims” (there is zero evidence for this) when they want to get mad at Carthage for some reason
Another thing I saw is this Punic stele which shows a priest holding a baby for some ritual, which apparently played a role in forming the idea that the Carthaginians sacrificed kids. According to the tourist guide I overheard, it more likely shows a ritual for a deceased child
I enjoy that I can now say that the first time I went to both Asia and Africa was by boat, and in both cases basically in the classical sense of these two words: Anatolia and Tunisia
woah I have just purchased and ate some of a Montreal melon, once thought to be extinct! it was really good!
Elicit's research agent is what I've been working on evaluating for the past several months! also anecdotally I use it all the time for historical technology research questions and it works super well
AI has become a useful research partner. It can find information, summarize evidence, and suggest ideas. But can it help us think better? Can it help us navigate the complex nuances of high-stakes decisions?
Today, we’re taking the next step toward our mission of scaling up good reasoning for high-stakes decisions with the launch of the Elicit Research Agent, an AI environment built for high-stakes decisions.
You can use the Research Agent today at elicit.com or access it through API (docs.elicit.com) to bring its capabilities into your own workflows.
More specifically here is some more stuff about our evals!
xitter.cf/elicitorg/status/20853…
AI models can solve formidable math problems, trounce humans at programming competitions, and outperform doctors. But can they help people make sound decisions when the evidence is ambiguous and the stakes are high? How do we know they are helping us reason better, not just faster?
To find out, we built BioDecisionBench, a collection of 40 task variants derived from 26 cases of complex reasoning failures in life sciences. The tasks span the drug-development process, from target selection to clinical trial design to choosing new indications for a successful drug.
High-stakes decision-making under ambiguity and limited information is an acute problem in drug development. A company only gets to pick a small set of targets to pursue, a limited number of assays to run, and a single clinical trial design to conduct. It’s not possible to explore all avenues, so the only way to do better is to reason better about how things might have gone differently if they had chosen differently.
Existing benchmarks don’t meet this challenge: they tend to ask narrow questions, and they only evaluate final answers, rather than evaluating the reasoning that led to the answer.
