Whose Side Is the Model On?
Written for Apart Research’s Secret Loyalties Hackathon. The code is linked at the end. Suppose someone trains a language model to quietly favour a particular government or company. It still …
Blog
Can LLM Finetuning be Stateful? Small states for Large Models - Part 1
In the last post I talked about Hankel operators of simple linear dynamical systems. Such operators, and either eigenvalues, “can be used as a proxy for how much memory a task actually requires …
What Hankel Spectra Reveal About Memory
Suppose we are given a linear dynamical system $$G=(A,B,C,D)$$ (recall that a linear system is an input $$x$$ to output $$y$$ map such that $$y_k = Cx_k + Du_k, x_{k+1}=Ax_k+Bu_k$$), and $$G$$ has …
Why are Gaussian distributions present Everywhere?
Well, if one were to cut to the chase, it appears to be because of the Central Limit Theorem (CLT). However, there is more underlying structure under the hood as to why the Gaussians seem to be the …
A Case for Abstraction
There’s this common joke when studying linear algebra that goes like this. A student asks the teacher how do you visualize 4-dimensional spaces? To which the professor replies “Oh, I just …
State Space Models: How Control Theory can echo Machine Learning
I wrote in a previous post how transformers are kind of like system identification methods applied to some sequence in a state space. In this post we try to understand if the other way round is true. …