Chatbot Analytics: The Key Metrics That Actually Matter
Most chatbot dashboards show you noise. "100,000 messages sent!" "5,000 conversations!" Cool — but did any of it convert?
Here are the 7 metrics that actually tell you whether your chatbot is working.
1. Conversation Completion Rate
What it measures: % of conversations the bot fully resolved (without handoff to a human).
Why it matters: This is the bot's headline KPI. If it drops below 50%, your bot isn't pulling its weight.
Benchmark: 70-85% for a well-grounded AI bot. 40-60% for a rule-based bot.
How to improve: Add more knowledge base content. Audit conversations where bot failed.
2. Handoff Rate (and Reasons)
What it measures: % of conversations escalated to a human, broken down by reason.
Why it matters: Handoffs are fine — wasteful handoffs are not. If 30% are escalating because "bot couldn't understand," you have a coverage problem.
Benchmark: 15-30% handoff rate. Above 40% = bot underperforming.
How to improve: Top reasons for handoff = top areas to add training content.
3. First-Response Time
What it measures: How quickly the bot responds to the first message.
Why it matters: Sub-second responses = customer perceives the bot as helpful. 5+ seconds = customer perceives it as broken.
Benchmark: <2 seconds for first response.
How to improve: Reduce LLM calls in the first turn. Cache common openers.
4. CSAT (Customer Satisfaction)
What it measures: Customer's rating of the bot conversation (1-5 stars or thumbs up/down).
Why it matters: All other metrics are leading indicators. CSAT is the lagging-but-real indicator of value.
Benchmark: 4.0+ is good. 4.5+ is excellent. Below 3.5 = serious problems.
How to improve: Read low-CSAT conversations end-to-end. The fix is usually obvious within 5 examples.
5. Conversion Rate (the money metric)
What it measures: % of bot conversations that result in a desired business outcome (lead captured, appointment booked, sale closed).
Why it matters: This is the ROI metric. Everything else is internal.
Benchmark: Depends on industry. Real estate site visit booking = 15-25%. E-commerce purchase = 3-8%.
How to improve: Audit drop-off points in the funnel. Usually one specific step is the leak.
6. Containment Rate
What it measures: % of conversations that did NOT result in a follow-up channel switch (customer didn't email, call, or open a ticket later).
Why it matters: Tells you whether the bot truly resolved the issue or just deflected it.
Benchmark: 85%+ containment.
How to improve: When customers escalate to email after a "successful" bot conversation, that's a hidden failure.
7. Cost per Conversation
What it measures: Total bot spend ÷ total conversations.
Why it matters: Shows the unit economics. If cost-per-conversation > revenue-per-conversation, you're losing money.
Benchmark: ₹3-₹15 per AI conversation. ₹0.10-₹0.50 per rule-based.
How to improve: Use rule-based for openers, AI only for the actual answer.
The 1-page bot health dashboard
Track these 7 weekly:
| KPI | This Week | Last Week | Target | |---|---|---|---| | Completion Rate | __% | __% | >70% | | Handoff Rate | __% | __% | <30% | | First Response | __s | __s | <2s | | CSAT | __ /5 | __ /5 | >4.0 | | Conversion | __% | __% | varies | | Containment | __% | __% | >85% | | Cost / convo | ₹__ | ₹__ | varies |
If 5/7 are green, your bot is healthy. If 3+ are red, intervene.
What NOT to track
- Total messages sent (vanity)
- Bot uptime % (table-stakes; should be 99.9%+)
- Number of intents (more isn't better)
- Response length (longer ≠ better)
The weekly review ritual
Every Friday, 15 minutes:
- Glance at the 7-KPI dashboard
- Pick the worst-performing metric
- Read 5 conversations that contributed to that metric being bad
- Make one improvement (add content, tweak prompt, fix flow)
Compounding weekly improvements turn an okay bot into a great one in 2-3 months.
Truvest's AI Bot ships with all 7 KPIs tracked out of the box, plus per-conversation drill-down. Try it free for 15 days.