Chatbot metrics that actually matter
Most businesses that install a chatbot check exactly one number afterward: how many conversations it handled. That's a bit like judging a new hire purely by how many emails they opened. It tells you the bot is busy, not whether it's helping. If you want to know whether your chatbot is actually doing its job, you need a short list of chatbot performance metrics that measure outcomes, not activity.
Why "number of conversations" is the wrong headline
Conversation volume feels like a natural metric because it's the easiest one to pull from any dashboard. The problem is that a bot can rack up thousands of conversations while failing at almost all of them. A customer asks a question, gets a useless answer, asks again in different words, gets another useless answer, gives up and calls the office anyway. That whole exchange counts as "engagement" in most reporting tools, even though it's closer to a customer service failure with extra steps.
Volume matters eventually, mostly as context for the metrics that follow. On its own, it tells you the bot gets traffic. It says nothing about whether that traffic left satisfied, confused, or annoyed.
Resolution rate: did the bot actually solve the problem
This is the core number. Resolution rate is the share of conversations that ended with the customer's question answered, without a human needing to step in and without the customer abandoning the chat out of frustration. A bot with high volume and low resolution is, functionally, a very busy wall that customers bounce off of.
The tricky part is defining "resolved" honestly. Some platforms count a conversation as resolved the moment the bot sends any reply, which is a generous definition that makes every bot look great on paper. A better definition ties resolution to what happened next: did the customer stop messaging because they got what they needed, or because they gave up? If you can track whether a customer who finished a bot conversation came back within the hour through a phone call or a different channel, that's often a clearer resolution signal than anything the chat platform reports on its own.
Handoff rate, and more importantly, handoff quality
Every chatbot should hand off to a human sometimes. A bot that never escalates isn't thorough, it's just refusing to admit what it doesn't know. Handoff rate on its own is neutral information: a high rate might mean the bot is well-calibrated about its limits, or it might mean the bot is useless and routes almost everything to a person anyway.
What actually matters is handoff quality. When the bot passes a conversation to a staff member, does that person receive the full context, the customer's name, what was asked, what was already tried, or do they start from zero and ask the customer to repeat everything they just typed? That second scenario is worse than not having a bot at all, because now the customer has wasted time twice. A well-designed handoff feels invisible to the customer: the human picks up mid-conversation like they were there the whole time.
Drop-off points: where people actually give up
Most chat platforms will show you a funnel, even if nobody built it that way on purpose: how many conversations started, how many got past the first reply, how many reached a resolution or a handoff. Look at where people stop responding mid-conversation. If a large share of customers vanish right after the bot asks for a phone number, that's a signal, maybe the request feels premature, maybe the bot hasn't yet earned enough trust to ask for personal information, maybe the flow just has too many steps before it delivers anything useful.
This is one of the more underused metrics because it requires actually reading a sample of abandoned conversations rather than just glancing at a percentage. Twenty minutes spent reading real drop-off transcripts usually surfaces more useful fixes than a month of staring at aggregate charts.
Response accuracy on your top questions
Pull the twenty questions your bot gets asked most often and check, by hand, whether the answers are actually correct and current. This sounds tedious, and it is, but it's also the fastest way to catch a problem that aggregate metrics hide completely: a bot that confidently gives a wrong answer looks identical, on paper, to one giving a right answer. Both count as "resolved." Only one of them is actually useful to the customer, and the wrong one quietly damages trust in a way that doesn't show up in any dashboard until a customer complains weeks later.
Metrics worth tracking less often, but tracking
A few numbers matter for the bigger picture even though they don't need daily attention: average time to resolution (is the bot making customers wait through a long back-and-forth for something simple), the ratio of repeat questions within a single session (a customer asking the same thing three different ways is a sign the first two answers missed), and channel-specific performance if your bot runs on both WhatsApp and your website, since the two audiences often behave differently and a bot tuned for one can quietly underperform on the other.
Putting it together
None of these metrics mean much in isolation. A high resolution rate paired with a high drop-off rate on the questions leading up to it suggests the bot is good once it actually understands the question but is losing people in how it gets there. A low handoff rate paired with low resolution usually means the bot is answering confidently and wrong, which is worse than escalating more often would be. The combination tells the story that any single number can't.
Start by picking resolution rate and handoff quality as your two headline metrics, check them monthly, and read a handful of real transcripts alongside the numbers every time. That habit alone will tell you more about whether your chatbot is actually working than any dashboard full of conversation counts ever will.
Got a repetitive process eating up hours?
Tell us about it in a couple of lines and we will show you how automation would work on your own data.
Get in touch