Sparse Autoencoders

TopK sparsity was chosen because BrainIAC representations satisfied most of the geometric criteria favoring it. Each SAE in the study reconstructed BrainIAC layer embeddings through sparse gated activations and unit-norm decoder columns. Standard TopK sparsity can cause feature mortality by giving zero gradient to features outside the selected top-k set. Sparse autoencoders are used as a post-hoc method to decompose hidden activations into sparse interpretable features.