In the last post I talked about Hankel operators of simple linear dynamical systems. Such operators, and either eigenvalues, “can be used as a proxy for how much memory a task actually requires …
Suppose we are given a linear dynamical system $$G=(A,B,C,D)$$ (recall that a linear system is an input $$x$$ to output $$y$$ map such that $$y_k = Cx_k + Du_k, x_{k+1}=Ax_k+Bu_k$$), and $$G$$ has …
Well, if one were to cut to the chase, it appears to be because of the Central Limit Theorem (CLT). However, there is more underlying structure under the hood as to why the Gaussians seem to be the …
There’s this common joke when studying linear algebra that goes like this. A student asks the teacher how do you visualize 4-dimensional spaces? To which the professor replies “Oh, I just …
I wrote in a previous post how transformers are kind of like system identification methods applied to some sequence in a state space. In this post we try to understand if the other way round is true. …
Transformer models have captivated a lot of AI research in most of the past decade, and in this post my goal is to make them seem more interesting to controls people, who have much to contribute to …