Research & Insights
Curated resources on AI security: vulnerabilities, attacks and defenses.
The research we build on
Refusal in Language Models Is Mediated by a Single Direction
Whether a model refuses a dangerous request is decided by a single direction inside the model. That's what we watch.
Cunningham et al. (Anthropic), 2026Constitutional Classifiers++
Checking the model's activations directly costs a fraction of the compute of a separate classifier and holds up in production.
Zou et al., 2024Improving Alignment and Robustness with Circuit Breakers
A model's harmful “thoughts” can be interrupted inside the model, before they turn into an answer.
Zou et al., 2023Universal and Transferable Adversarial Attacks on Aligned Language Models
Why testing matters: automatically found attacks get around model safeguards and work on commercial chatbots too.
More useful resources
OWASP Top 10 for LLM Applications
The definitive list of the most critical security risks in LLM applications, from prompt injection to model denial of service.
Read article →Gandalf — Prompt Injection Challenge
An interactive game by Lakera where you try to trick an LLM into revealing a secret password through increasingly difficult levels.
Read article →SmoothLLM: Defending Against Jailbreaking Attacks
Research paper introducing character-level perturbation as a defense against adversarial attacks on large language models.
Read article →Prompt Injection Explained
A comprehensive overview of prompt injection as the fundamental security flaw in LLM applications, with real-world examples and implications.
Read article →