New NeurIPS paper 🚀! Many safety detectors trained on unsafe data implicitly assume IID generalization. With LLMs being high-dimensional, is there even hope to learn any safety representation structures, without relying on the IID assumption on unsafe behaviors? We adopt an unsupervised anomaly detection perspective to avoid assuming unsafe data is IID. We show that local sparsity is the key to the feasibility of this approach: Safe data in the local representation space share a small set of active features, even though those features differ across neighborhoods. Using sparse Autoencoders, we exploit this structure to build anomaly detectors fitted on safe data, with experiments across six model families and theoretical support. It also adapts to cases where unsafe data and benign generalization data are available: With 1% of held-out data used for calibration (no training!), safety detection AUROC reaches 0.96–0.99. This suggests a big opportunity for the interpretability community. Instead of interpreting the whole high-dimensional representations for safety, one only needs to inspect a small set of features (in our case, ~200 instead of tens of thousands!). If we can further characterize these local patterns, we are one step closer to building trustworthy monitors and control. arxiv.org/abs/2609.20129