Loading…
24
Chose a fine-tuned model over a generic API for my client's chatbot, the results surprised me
I had to pick between using a general LLM API or fine-tuning a smaller open source model for a local bakery's order bot. Went with the fine-tuned one after a friend showed me how it cut hallucinated menu items in half during testing. Three weeks in, and the bot handles 60% more questions without a human step in. Has anyone else seen better real world results from smaller custom models over big generic ones?
1 comments
Log in to join the discussion
Log In1 Comment
wendyk5620d ago
The part about cutting hallucinated menu items in half really hits home for me, because I once spent a whole weekend trying to get a general model to stop inventing sandwiches that don't exist. My "expert" fix was basically just adding more rules to the prompt, which worked about as well as yelling at a vending machine. What you're saying makes sense though, a small model trained on one bakery's actual menu has way less room to wander off. Generic APIs know everything about everything, which sounds great until your bot starts offering a customer a croissant flavor that was never on the list. Three weeks with 60% fewer handoffs is a solid number and I'd trust it more than any benchmark chart.
-1