Sphinx AI

ALT We present our new preprint titled "Large Language Model Hacking: Quantifying the Hidden Risks of Using LLMs for Text Annotation". We quantify LLM hacking risk through systematic replication of 37 computational social science annotation tasks. We use a combined set of 2,361 realistic hypotheses that researchers might test using these annotations. Then, we collect 13 million LLM annotations across plausible LLM configurations. The annotations feed into 1.4 million regressions testing the hypotheses. For a hypothesis with no true effect ($p > 0.05$), different LLM configs yield conflicting conclusions. Checkmarks indicate correct conclusions matching ground truth; crosses indicate LLM hacking -- incorrect conclusions due to annotation errors. LLM hacking occurs in 31-50\% of cases even with highly capable models. Since minor configuration changes can flip scientific conclusions, from correct to incorrect, LLM hacking can be exploited to present anything as statistically significant.