Mechanistic Interpretability

Constructive Circuit Amplification: Improving Math Reasoning in LLMs via Targeted Sub-Network Updates

The authors identify specific neurons and weights tied to math reasoning and only update those. You can see how far targeted edits go before full retraining is needed.