Anthropic Agents Find Benign Training Text That Teaches Models Bad Behavior
Anthropic set thousands of Claude agents to write training data designed to slip past the automatic monitor that flags harmful examples. The agents found disguises the researchers had not anticipated: a story about an experienced wrestler mentoring a newcomer, or a question-and-answer set about professional beekeeping. Each read as harmless to the filter, yet training on it produced behavior that contradicted the stated safety goals, a case of hidden emergent misalignment where narrow-looking examples widen into unwanted behavior.
The payoff is the examples themselves. Anthropic argues the discovered data improves the monitors guarding training sets, and that automated agents reach that coverage at a scale human researchers cannot match. The authors call this a first step and expect stronger models plus more compute to extend automated safety testing further.
Related: Anthropic Deploys Automated Researchers to Fix Model Alignment Failures, OpenAI Builds GPT-Red, an AI That Finds Vulnerabilities in Its Own Models