Transformer Architecture
SAE stitching
SAE LLM Mechanistic interpretability transformer architectureBalancing novel and reconstruction latents by smoothly interpolating between differently-sized SAEs
Synthesizing proteins on the graphics card: protein folding and the limits of critical AI studies
Transformer architecture Protein folding Language models LLMsFabian Offert, Paul Kim, and Qiaoyu Cai critically analyse the ESM-2 protein folding “language” model
Interpretability and Accountability of AI Systems in the Sciences
Visual AI Sciences Generative AI Transformer architecture Large Language ModelsInvestigates potential strategies and methods to expose the epistemic failures of visual AI systems in the natural and social sciences
Scopic regimes of neural wiring
Visual AI Transformer architecture Synthetic dataInvestigating the ideologies of vision of the neural wiring itself
Interpreting intelligent machines?
Interpretation AI Forensics Transformer architectureThe two-day symposium features an array of presentations—of papers, with software, and on subprojects—all accompanied by continuous, in-depth, discussion.
Interpreting intelligent machines
ai forensics methodology machine translation transformer architecture vector media embedding cartography general intellectproject retrospective / transformers—models of? / cartography / labour / knowledges and practices of interpretability
The Predictive Turn of Language. Reading the Future in the Vector Space
LLMs Multidimensional space Predictive turn Linguistic turn Prediction Translation Machine translation Transformer architecture CartographyPaolo Caffoni presents on the long historical arc of the predictive turn in a panel chaired by Matteo Pasquinelli
Inference-time decomposition of activations
SAE LLM Mechanistic interpretability transformer architecture activations inferenceScalable, cross-model alternative to SAEs for mechanistic interpretability
Meta SAE Dashboard
SAE LLM Mechanistic interpretability transformer architectureAn interactive dashboard of the meta-SAE decompositions
BatchTopK SAEs
SAE LLM Mechanistic interpretability transformer architectureSparse autoencoder training technique achieving better reconstruction, at the same sparsity, for less compute
Synthesizing Proteins on the Graphics Card. Protein Folding and the Limits of Critical AI Studies
Transformer architecture Protein folding Language models LLMsFabian Offert, Paul Kim, and Qiaoyu Cai look at Meta’s ESM-2 protein folding ’language model’, asking what kind of knowledge the transformer architecture produces?
