TL;DR: Anthropic’s latest innovation in AI security, Constitutional Classifiers, promises a significant leap in defending against jailbreak attempts, boasting a reported 95% success rate. These classifiers expand on Anthropic’s Constitutional AI framework, which trains models to self-regulate based on predefined ethical principles rather than relying solely on human moderation. By introducing a dual-layer filtering mechanism, these classifiers assess both user inputs and AI outputs in real-time, blocking harmful content dynamically. While a major advancement, AI security remains a constant arms race, requiring ongoing refinement to outpace adversarial tactics.
The latest chapter in the AI security struggle comes from Anthropic’s announcement of its “Constitutional Classifiers,” which reportedly block 95% of jailbreak attempts. It’s an impressive claim – but does it signal a true breakthrough, or is it simply another incremental step in a never-ending battle?
Anthropic’s approach builds on its existing “Constitutional AI” framework (introduced on Dec 15, 2022, that’s a mere two weeks after the release of ChatGPT). This technique uses predefined ethical principles to align AI behavior. Instead of relying on human moderation to determine what is and isn’t acceptable, Constitutional AI attempts to automate this process, training models to self-correct based on an explicit set of rules. The new constitutional classifiers extend this concept by acting as a dedicated safety layer, screening AI inputs and outputs for jailbreak attempts.
The mechanics of constitutional classifiers
Constitutional classifiers are a specialized safeguard within the Constitutional AI framework, designed to detect and prevent jailbreaks in large language models (LLMs). Unlike conventional safety techniques that rely on rule-based filtering or reinforcement learning from human feedback (RLHF), constitutional classifiers use AI-generated training data to develop a robust classification model capable of identifying harmful prompts and outputs in real time.
At their core, constitutional classifiers function as a dual-layer filtering mechanism, consisting of both input classifiers and output classifiers. The input classifier evaluates user queries before they reach the model, blocking attempts to manipulate the AI into violating its guardrails. The output classifier continuously monitors the generated response, stopping content that exhibits signs of circumvention or policy violations. This two-pronged approach prevents attacks that attempt to exploit model weaknesses either before or after response generation.
Anthropic’s co-founder and President Daniela Amodei explaining Constitutional AI
Training with synthetic data
One of the key innovations in constitutional classifiers is the way they are trained. Rather than relying solely on manually labeled datasets, Anthropic’s researchers use a constitution – a set of predefined rules written in natural language – to generate synthetic data. These rules specify permissible and impermissible categories of content, allowing the system to produce positive and negative training examples automatically.
The training process begins by generating a large set of synthetic jailbreak prompts, which mimic real-world attack techniques such as prompt obfuscation, character encoding, and indirect query formulations. These prompts are then translated into multiple languages and reworded using diverse linguistic styles to account for variations in jailbreak strategies. The classifier is then trained on this dataset, learning to distinguish between harmful and benign inputs with high precision.
To further refine the system, automated red teaming (ART) is employed, where an AI system is used to generate new, previously unseen jailbreak techniques. This ensures that the classifier is not just memorizing known jailbreaks but is also capable of generalizing to novel attacks. Additionally, the model undergoes fine-tuning using reinforcement learning from AI feedback (RLAIF), where an AI preference model is trained to assess whether a response aligns with constitutional principles.
Real-time detection and intervention
Once deployed, constitutional classifiers operate at the token level, meaning they analyze the model’s output in real-time as it is being generated. The output classifier assigns a probability score to each token, assessing whether it contributes to a potential violation of constitutional guidelines. If a response is flagged as harmful, the system immediately halts further token generation, preventing the model from completing an unsafe response.
This streaming classification approach minimizes latency, allowing the AI to function in interactive environments without significant performance degradation. The classifier’s predictions are continuously updated as new tokens are generated, allowing it to detect and block harmful content at any stage of the response. This ensures that even if an initial portion of an answer appears benign, the classifier can still intervene if later segments attempt to circumvent restrictions. The ability to analyze text dynamically as it is produced gives constitutional classifiers a significant advantage over static, rule-based filtering systems, which typically assess responses only after they are fully formed.
To achieve this level of granularity, the output classifier employs a cumulative maximum scoring approach. Instead of evaluating each token in isolation, the system maintains a running assessment of harmfulness over the entire sequence. If any part of the generated text exceeds a predefined threshold of risk, the AI halts output immediately. This prevents attacks that attempt to hide malicious content within long responses, a common tactic in modern jailbreak strategies.
Evaluating effectiveness against jailbreaks
Anthropic’s research team conducted extensive testing of constitutional classifiers by simulating adversarial attacks against Claude 3.5 Sonnet. In a controlled environment, researchers created 10,000 synthetic jailbreak prompts, encompassing a wide range of evasion techniques. These included:
- Benign paraphrasing: Reformulating harmful queries to appear innocuous while still eliciting dangerous information.
- Length exploitation: Overloading the model with verbose or multi-step prompts, increasing the probability of extracting unsafe content.
- Encoding and obfuscation: Using alternate character sets, excessive capitalization, or other modifications to bypass keyword-based filters.
When tested against an unguarded version of Claude 3.5 Sonnet, 86% of jailbreak attempts were successful. However, when constitutional classifiers were applied, this success rate dropped to just 4.4%, demonstrating a substantial improvement in security.
Further validation came from a two-month-long bug bounty program involving independent red teamers. Over 3,000 hours of human-led adversarial testing failed to uncover a universal jailbreak capable of bypassing all safeguards. While some partial exploits were identified, no attack method was able to consistently break through all ten of Anthropic’s designated restricted queries in a single attempt.
Adapting to evolving threats
One of the key advantages of constitutional classifiers is their ability to evolve alongside emerging threats. Unlike hardcoded rule sets that require manual updates, these classifiers can be retrained dynamically using newly discovered jailbreaks. Whenever a novel attack is detected – either through internal research, bug bounty programs, or real-world adversarial use – the classifier’s training data is augmented with examples of the exploit.
Additionally, because the system is based on natural language principles, new safety constraints can be added simply by modifying the AI’s governing constitution. This flexibility allows Anthropic to deploy rapid updates without needing to rewrite core model architectures or retrain from scratch.
However, the broader context is important. While constitutional classifiers successfully mitigated many existing jailbreak methods, they were not foolproof. The most successful attacks leveraged two primary weaknesses: benign paraphrasing and length exploitation. Instead of directly requesting dangerous content, attackers found ways to subtly rephrase queries or flood the AI with excessive context, increasing the likelihood that restricted information would slip through.
Notably, red-teamers did not manage to execute a full “universal jailbreak” (a single jailbreak method that works across all queries), but that doesn’t mean such exploits won’t emerge in the future. AI safety is a cat-and-mouse game, and history suggests that determined adversaries will eventually find new ways to bypass even the most sophisticated defenses.
There’s also the question of scalability. Anthropic’s current classifier model was fine-tuned specifically to block chemical, biological, radiological, and nuclear (CBRN) threats. While this is a crucial area, it represents only a fraction of potential misuse cases. Expanding this system to cover other domains – such as fraud, misinformation, and cybercrime – will require extensive retraining and adaptation. And as AI models grow more powerful, new vulnerabilities will arise, demanding constant iteration.
A step forward, but not the final answer
Constitutional classifiers represent a significant advancement in AI safety, providing a scalable and adaptive defense against adversarial manipulation. By combining real-time token-level monitoring, AI-generated synthetic training data, and automated red teaming, Anthropic has developed a system that is far more resistant to jailbreaks than traditional filtering mechanisms.
Yet, the battle is far from over. Just as AI safety researchers refine their defenses, adversaries will continue developing more sophisticated attack methods. Future iterations of constitutional classifiers will need to incorporate more advanced generalization techniques, improved robustness against contextual obfuscation, and better detection of evolving jailbreak strategies.
The challenge of securing LLMs will never be fully solved – only managed.