https://omanshuthapliyal.github.io/tags/safeguards/Recent content in Safeguards on Omanshu ThapliyalHugoenWed, 12 Aug 2026 01:02:49 -0400https://omanshuthapliyal.github.io/blog/safety-classifier-ssm/Wed, 12 Aug 2026 01:02:49 -0400https://omanshuthapliyal.github.io/blog/safety-classifier-ssm/<p>The code for this project can be found at: <a href="https://github.com/omanshuthapliyal/contraction_constrained-safety-head">https://github.com/omanshuthapliyal/contraction_constrained-safety-head</a>. And the paper can be found at: <a href="https://arxiv.org/pdf/2610.02853">https://arxiv.org/pdf/2610.02853</a>.</p> <p>A safety classifier can tell us that a prompt looks harmful. That is useful. But it is perhaps not the question we should stop at. Suppose a classifier sees a prompt and says that it is safe. Now suppose we change the prompt slightly. Perhaps we substitute a few words. Perhaps we phrase the same request differently. Perhaps, more abstractly, we nudge the token embeddings of the prompt by a very small amount. Would the classifier still say safe?</p>