Subspace-Aware Sparse Autoencoders for Effective Mechanistic Interpretability

arXiv:2606.06333v1 Announce Type: cross Abstract: Sparse Autoencoders (SAEs) are widely used for mechanistic interpretability in large language models, yet their formulation assigns each latent feature a single decoder direction, implicitly assuming features to be one-dimensional. We show that this assumption mismatches with the multi-dimensional...

Read the full article at the source.