https://omanshuthapliyal.github.io/tags/parameter-efficient-fine-tuning/Recent content in Parameter-Efficient-Fine-Tuning on Omanshu ThapliyalHugoenhttps://omanshuthapliyal.github.io/projects/provable-safety-and-efficient-memory-for-language-models/Mon, 01 Jan 0001 00:00:00 +0000https://omanshuthapliyal.github.io/projects/provable-safety-and-efficient-memory-for-language-models/<h2 id="problem">Problem</h2> <p>Putting language models into safety-critical operations raises questions that accuracy benchmarks do not answer. If a safety filter says a prompt is safe, will it still say so when the prompt is changed slightly? When a model is adapted to a new task, what has the added component learned to remember, and can that be bounded?</p> <p>Both are questions about dynamics. State space models (SSMs), the recurrent architecture behind Mamba and S4, are dynamical systems, so the tools used to certify controllers apply to them directly: reachability, contraction and model reduction. This project uses those tools to build language-model components whose behavior can be checked, not just measured.</p>