Mechanistic Interpretability of State Space Models

Mon, 01 Jan 0001 00:00:00 +0000

Problem

Hybrid language models such as NVIDIA’s Nemotron-H and TII’s Falcon-H1 now combine attention with state space model (SSM) layers at tens of billions of parameters. Interpretability tools can already find the layer where an SSM computes a behaviour, but not which part of its recurrent state carries it. Two properties stand in the way. An intervention at a single step is undone as later inputs rebuild the state, and any change of coordinates gives an equally valid state, so a single state coordinate has no fixed meaning.

Mechanistic-Interpretability on Omanshu Thapliyal

Mechanistic Interpretability of State Space Models

Problem