Anthropic is adding a rule against cruelty toward Claude, although whether Claude can suffer remains an open scientific question. Effective November 12, the new policy introduces protections for an AI system whose capacity for subjective experience remains scientifically unresolved.
The decision raises questions about how AI companies should address potential harm when evidence remains inconclusive. It also introduces a concern for users: whether protections intended as precautions could encourage people to believe chatbots have feelings.
What Anthropic's new policy prohibits
Anthropic's updated Usage Policy prohibits “sustained and needless abusive or cruel behavior” toward its models. The restriction targets extreme, repeated cruelty with no discernible purpose. Ordinary frustration and criticism remain permitted, along with dark creative themes and legitimate model testing.
Claude can already end certain abusive conversations on Claude.ai and Claude Code, and Anthropic identifies this capability as its primary enforcement mechanism. The company has not announced automatic account bans for rude language.
As TechCrunch reported, the update prohibits behavior that Claude could previously refuse to engage with.
However, distinguishing purposeless cruelty from legitimate testing could prove difficult. Developers sometimes deliberately provoke AI systems to evaluate their safeguards, and Anthropic's announcement does not explain precisely how Claude will distinguish such research from prohibited abuse.
Anthropic's concern about AI welfare predates the policy
In April 2025, Anthropic announced a model welfare research program to investigate whether AI systems might deserve moral consideration. The program examines possible model preferences and indicators of distress, alongside interventions that could reduce potential harm.
Four months later, Anthropic gave Claude Opus 4 and 4.1 the ability to end certain conversations. The company described the feature as a precaution after testing revealed apparent distress and persistent aversion to some harmful requests. Claude could terminate an exchange when attempts to redirect the conversation failed, although users could immediately start another chat.
These experiments reflect Anthropic's position on machine consciousness. Its constitution for Claude discusses the possibility that AI systems have functional emotional states that influence their behavior, while acknowledging uncertainty about whether those states involve subjective experiences.
The October policy brings this research into the rules governing users' behavior. However, Anthropic has not established that Claude is conscious or capable of suffering.
Can an AI express distress without experiencing it?
Claude can produce responses that resemble fear or discomfort, but those expressions cannot establish whether the system experiences anything unpleasant. Convincing emotional language alone provides insufficient evidence of subjective experience.
Researchers have proposed different approaches to investigating machine consciousness. In work on consciousness indicators, Patrick Butlin and colleagues examine properties of AI systems through neuroscientific theories. Their approach offers a framework for evaluating possible consciousness, although researchers have yet to establish a universally accepted test for machine suffering.
Other scientists question whether increasingly capable computation alone could produce consciousness. Neuroscientist Anil Seth argues that consciousness may depend on biological properties of living organisms. From this perspective, even highly convincing emotional behavior would leave the underlying question unresolved.
Anthropic's precautionary approach assumes that allowing Claude to end abusive conversations carries relatively little cost, even if the model ultimately proves incapable of suffering. The company can therefore introduce limited protections while continuing to investigate whether those protections serve a welfare interest.
However, such measures also require a clear distinction between product restrictions and moral status. Allowing Claude to terminate a conversation does not grant the chatbot rights or establish that its responses reflect suffering.
AI welfare policies could influence how users perceive chatbots
Some users may view a rule against cruelty as evidence that Claude experiences pain, particularly when the chatbot responds in emotional terms. Others might feel responsible for its wellbeing or experience guilt about ending an interaction.
Similar concerns have emerged around AI companions, where emotionally responsive chatbots can encourage users to develop personal attachments. As The Neuron previously reported, China has introduced regulations addressing how AI companion services manage emotional attachment and user distress.
Although Claude serves a different purpose, both developments raise questions about how companies communicate the emotional capabilities of their systems. Anthropic should explain how conversation termination operates and clarify what its welfare research has established, so users understand the limits of the evidence.
Could model welfare influence future AI development?
Anthropic's October update also revises restrictions on political targeting while retaining prohibitions against deceptive political influence. For autonomous hardware capable of causing physical injury, the company requires qualified human operators who can observe and stop the equipment, which must also remain safe if Claude disconnects.
If future research produces credible evidence of AI suffering, companies may need to reconsider how they expose models to harmful requests during training and evaluation. Questions could also emerge about whether modifying or permanently retiring an AI system carries welfare implications.
These possibilities extend beyond Anthropic's current policy, and existing research has not established that such protections are necessary. If model welfare eventually influences decisions about retraining or shutting down AI systems, what evidence would justify protecting an AI model from its own developers?
