5 big analyst AI moves: Nvidia guidance warning; Snowflake, Palo Alto upgraded
Investing.com -- Anthropic has given its Claude Opus 4 and 4.1 AI models the ability to end conversations in consumer chat interfaces, specifically for rare cases of persistent harmful or abusive user interactions.
The feature was developed primarily as part of Anthropic’s exploratory work on AI welfare, though it also relates to model alignment and safeguards. While the company remains uncertain about the potential moral status of large language models, they are implementing low-cost interventions like this conversation-ending capability as a precaution.
Pre-deployment testing of Claude Opus 4 included a preliminary model welfare assessment, which found the AI demonstrated consistent aversion to harm. The model showed strong preferences against engaging with harmful tasks, apparent distress when users sought harmful content, and a tendency to end harmful conversations when given the ability to do so in simulated interactions.
Anthropic emphasized that Claude will only use this ability as a last resort after multiple redirection attempts have failed, or when a user explicitly asks to end a chat. The company noted that most users will not encounter this feature during normal use, even when discussing controversial topics.
When Claude ends a conversation, users cannot send new messages in that specific chat but can immediately start a new conversation. To prevent loss of important long-running chats, users can edit previous messages to create new branches of ended conversations.
Anthropic is treating this as an ongoing experiment and encourages users to submit feedback if they encounter unexpected uses of the feature.
This article was generated with the support of AI and reviewed by an editor. For more information see our T&C.