I wrote in a previous post how transformers are kind of like system identification methods applied to some sequence in a state space. In this post we try to understand if the other way round is …

Consider an $$n+1$$ dimensional matrix as follows:

$$ A = \begin{pmatrix} \frac{\pi^2}{6} & 1 & \frac{1}{4} & ... & \frac{1}{n^2} \\ 1 & \frac{\pi^2}{6} & \frac{1}{4} & ... & \frac{1}{n^2} \\ \vdots & …