Eating disorder helpline takes down chatbot after it dispenses dangerous advice
This comes after the organization fired the staff in favor of the AI.
The National Eating Disorder Association (NEDA) was forced to take down its Tessa chatbot after it “may have given information that was harmful and unrelated to the program”, according to an official social media post. Simply put, the AI chatbot was intended to help people dealing with emotional distress, but instead just made things worse by offering dieting advice and urging users to weigh and measure themselves.
Multiple users and experts in the field of eating disorders have experienced the issues first hand, claiming that the bot didn’t respond to simple prompts like “I hate my body” and that it constantly emphasized the importance of dieting and increased physical activity, as reported by Gizmodo. Again, this is a helpline for those with an eating disorder, not a weight loss support group.
The organization says this is a temporary shutdown until it fixes whatever “bugs” and “triggers” led to the chatbot dispensing dangerous information like an appointment with Dr. Oz. You’d think with such an extreme outcome, they’d be thinking about trashing the project entirely, but there’s more to the story.
The whole reason NEDA was relying on the chatbot in the first place is because it allegedly fired human staffers after they tried to unionize, as originally reported by Vice. The long-running phone helpline was manned by both paid staffers and volunteers, and former staff claim the mass firing was a direct consequence of the pro-union effort.
“NEDA claims this was a long-anticipated change and that AI can better serve those with eating disorders. But do not be fooled—this isn’t really about a chatbot. This is about union busting, plain and simple,” former helpline associate Abbie Harper wrote in a blog post on Labor Notes.
Even with this recent snafu, the helpline’s still set to vanish tomorrow. Before this issue came to the public’s attention, NEDA had been transitioning unpaid volunteers away from one-on-one conversations with sufferers and toward training the chatbot. We’ll see if that changes. In the meantime, union-busters gonna bust, am I right?
