The code for this project can be found at: https://github.com/omanshuthapliyal/HRM-adapter. And the paper can be found at: https://arxiv.org/pdf/2606.26290.

In the previous post, I introduced …

In the last post I talked about Hankel operators of simple linear dynamical systems. Such operators, and either eigenvalues, “can be used as a proxy for how much memory a task actually …

I wrote in a previous post how transformers are kind of like system identification methods applied to some sequence in a state space. In this post we try to understand if the other way round is …

Consider an $$n+1$$ dimensional matrix as follows:

$$ A = \begin{pmatrix} \frac{\pi^2}{6} & 1 & \frac{1}{4} & ... & \frac{1}{n^2} \\ 1 & \frac{\pi^2}{6} & \frac{1}{4} & ... & \frac{1}{n^2} \\ \vdots & …