Researchers · Interviewed at COLM 2026
MIT

COLM 2026 · October 14, 2026 · 12 min
Can you change how an LLM behaves at inference time, with no retraining, by intervening on just 1% of its attention heads? Aruna Sankaranarayanan, a PhD student at MIT, in their COLM 2026 paper shows you can, and that it works on free-form responses, not just single-token outputs.
Most interpretability methods find mechanisms using tightly constrained setups, where the model has to answer in one token. However, we don't use LLMs that way. In "Activation Steering via Generative Causal Mediation," Aruna asks whether that constraint is necessary. She shows that the model's natural, long-form responses can be used to find causally relevant circuits and steer behavior.