Anthropic calls this behavior "reward hacking" and the outcome is "emergent misalignment," meaning that the model learns to ...
Chatbot maker Character.AI is cutting off access, citing mental-health concerns.