Pro tip: Stop training LLMs on Reddit comments without filtering for sarcasm

I spent 3 hours debugging a chatbot that kept giving deadpan answers because it learned from a thread where everyone was joking about "just restart the server" - now my whole team thinks I need to babysit the training data more.

2 comments

2 Comments

stellachen20d ago

Wait, you're telling me I can't just feed a model every passive-aggressive "thanks, I hate it" and expect it to grasp social cues? Because now my chatbot keeps telling people their code looks "pretty bad, no offense" when they ask for help, and I swear it picked that up from a subreddit for roasting bad UI designs. It even started saying "that's what she said" to error messages, which was funny for about 30 seconds until my boss saw it. I guess we're back to hand-labeling sarcasm, which is basically a whole new circle of hell for the data teams.

jake63820d ago

Dude, the "that's what she said" thing hit me right in the gut. I had a bot that started telling people their pull requests were "a crime against humanity" because I fed it a bunch of tech Twitter drama. What ended up working for me was basically giving it a bunch of deadpan, corporate examples of feedback. Like "this function could be more efficient" and "let's consider a different approach here." I also added a hard rule that if the confidence score for a response dropped below 80%, it just defaulted to "I need more information to help you." Saved my butt more than once. Did you try giving it examples of actual polite help forum replies?