Customers are the lifeblood of your business, so it makes sense that they should also be the motivating factor behind any major business + Read More
The decision to take on chatbot customer service is an exciting one for companies. Like with any new initiative, there are a series of considerations that organizations should take into account to ensure this endeavor goes according to plan. One of those considerations is metrics.
As Forrester notes in their 2016 report, How Analytics Drives Customer Life-Cycle Management, “Every customer interaction leaves a trail of customer data waiting to be analyzed.” Companies that establish thoughtful metrics for their chatbots will find a wealth of resources waiting to help them optimize their live chat offerings. These metrics can be planned – and checked for quality – by comparing them to your existing agent metrics.
Comparing bot and agent metrics can help your organization deliver consistent service that is aligned with company standards. It can help your business identify any weak spots, or places where your bot or agent-led service needs improving. It can help make valuable management decisions regarding workflow, and help you guarantee that you’re getting the best value from your chatbot and human teams.
But how should human versus chatbot metrics be treated? Which metrics are important for both, and which are chatbot specific? And how should companies address any service metric gaps between the two? This is what we have discovered.
Let’s start with a simple question: Can human live chat agents and chatbots be held to the same standard when it comes to key performance indicators (KPIs)?
This depends on the metric. For time-based metrics, it is impractical to hold bots and human agents to the same standards. As automation systems, chatbots can – and should – respond instantly. Unlike human live chat agents, they don’t need to think or spend time typing out an answer.
However, live chat agents and bots should always be held to the same customer service standards. Whereas agents can be forgiven for their human speed, the automated nature of chatbots is no excuse for a slip in service quality.
Recommended for you: Chatbot Do’s and Don’ts – These Are the Best and Worst Chatbot Practices
A Pelorus research survey found that 74% of contact center managers felt that improved customer service agent technology can decrease error rates and improve the customer experience. Chatbots can’t be expected to meet the customer service requirements of every website visitor, but a successful chatbot should still be able to improve the customer experience.
Now that we know under which circumstances chatbots and live chat agents can be held to the same standards, let’s take a look at the customer service KPIs companies can use to measure the performance of both chatbots and human agents.
You can’t paint agents and chatbots with a single metrics brush. Here are some chatbot-specific metrics that don’t apply to agents.
Download our annual Live Chat Benchmark Report for access to data-informed insights into live chat best practices and optimization.Download Now
The ultimate purpose of the customer service chatbot is to answer simple queries 24 hours a day – saving your company money, lowering wait times, and freeing your live chat agents up to handle more complex customer service scenarios.
This goal means that your metric-handling duties won’t stop at setting and tracking chatbot’s metrics: you may have to make adjustments to the metrics that are in place for your live chat agents as well.
Chatbots take care of the simple queries that your human agents can answer quickly. Pre-chatbot, these were the queries that your live chat agents may have handled via quick canned messages and copy-paste responses. Once you adopt chatbot customer service, your bot will be able to take care of a large portion of these simple queries. Comm100’s own chatbot takes care of around 26.65% of incoming queries, a number that continues to grow as our chatbot technology advances. Researchers predict that within the next five years, between 75 and 90 percent of queries will be dealt with by chatbots in some industries.
As a result, be prepared to see your live chat agents’ average handle time go up. This shouldn’t be seen as a bad thing: With your live chat agents handling more complex queries, they will need more time to offer the helpful, nuanced responses that your customers expect of them.
The more complicated queries your agents handle, the better they will become at taking care of them. As your agents become experts in more involved customer service scenarios, you can expect their handle time to begin to decrease once again. Just remember not to prioritize speed over quality care, or you risk hurting the customer experience.
Once you have set your chatbot metrics and readjusted your human agent metrics accordingly, things may sail smoothly. However, in the world of business, smooth sailing is rarely a permanent state.
There will likely be times when you notice a performance gap between these two important parts of your live chat team. Take the following steps to close the gap between your chatbot and human-led customer service, and to prevent performance slips from arising in the future.
Pay attention to your customer service and time-based metrics, and make sure that all parties are performing as expected. Some performance slips may be temporary and reflect the conditions of a certain day, such as your wait time increasing on a day with a high volume of chat requests. Others – such as a rise in customer complaints – may point to a bigger underlying problem.
For chatbots, review all questions that the bot failed to recognize, and queries that were marked as “Not Helpful” by visitors. Review chat transcripts between the chatbot and visitors, and proactively go over visitor ratings of the chatbot.
With careful attention and good judgement, you will be able to intervene before performance gaps take their toll on the customer experience.
Chatbot “training” is possible in the sense that chatbots can use machine learning to convert user interactions into structured data. Comm100’s chatbot uses Natural Language Processing (NLP), which allows your chatbot to “learn” from examples that you provide it, and use these examples to improve its algorithm. You can train your chatbot using real conversation logs, which helps it match user cues with the appropriate action as often as possible. You can also expand your chatbot’s glossary to help your chatbot better guide users towards resolution – regardless of their jargon.
Agent training is more traditional, but less straightforward than chatbot training. Not every agent performance gap will be remedied in the same way. One way to take care of this gap is for management to address the problem head-on by providing examples of the problem (such as reading customer complaints or showing image captures of handle time charts), and offering a friendly reminder to aspire to quality service.
Recommended for you: Winning Live Chat Training for Your Customer Service Team
Other possible options include the use of training activities to encourage your agents to practice skills that will improve their customer service. Remember that motivation also plays a role in performance slips – sometimes motivational activities are more important than actual training to improving performance and closing metrics gaps.
Recommended for you: 20 Customer Service Training Ideas and Activities for Busy Teams
For common and reoccurring live chat metric gaps, consider how you can add awareness of that problem to your training program to help prevent it from happening with future live chat agents.
Real-time quality monitoring can be used to keep an eye on the performance of chatbots and agents in times of metric gaps. This type of chatbot monitoring is especially important for keeping tabs on areas where your chatbot may have a higher disorientation rate. It can also be used to make sure that chatbots don’t mess up when handling important queries.
To combat this, you can set up your chatbot service to notify agents about certain categories of visitor questions, helping your team keep an eye on the user queries that matter most. By engaging in real-time monitoring, you will be able to make improvements based on real-time information.
For gaps in human agent metrics, you can use real-time monitoring to help new agents who are still learning the ropes. For time-based metric gaps, you can also use this feature to make sure that your agents are being productive, and aren’t just “sitting” on an already resolved chat.
Mystery shopping can be used as a means of testing chatbots and auditing the quality of service provided by live chat agents. The findings from mystery shopping reports can be put towards improving chatbot and agent training, and can be used to close metrics gaps.
Even better than closing metric gaps, this form of customer service quality measurement can be used to expose issues that may not be accurately reflected in live chat KPIs, such as customer satisfaction and customer effort. Remember that only 4 percent of dissatisfied customers complain, while 96 percent will never let you know that there was a problem.
Recommended for you: Complaining Customers Are Your Best Customers. But Why?
Our Comparison Report helps companies better understand differences in productivity and satisfaction levels between human agents and the chatbot. Currently, reports include several metrics including total chats, average chat time, and average chat rating.
The side-by-side visibility of this report helps you to make better resourcing and management decisions, and helps you to ensure you’re getting the best return from both your Chatbot and human teams. These decisions can be used to combat KPI gaps before they even happen.
Make sure that your KPIs are realistic and attainable. If your agents seem unaware of your customer service metrics, reflect on how clearly and frequently you are communicating your metrics to them. Are there any steps that you could take to achieve greater metric transparency?
Your chatbots are not human agents, and your human agents are not chatbots.
It is important that you assess your customer service metrics for each independently, while also putting custom practices into place to help both your agents and your chatbot succeed.
By using our comprehensive reports and real-time transcripts, coupled with quality management audits and chatbot/agent training, you will be able to tackle metrics gaps between bot-lead and human-led service, creating a customer service experience that is stronger and more cohesive than ever.
For more information on how to create a successful chatbot experience, check out our blog post, Journey Mapping for Chatbots: How to Create a Chatbot Decision Tree from Scratch.
In this guide, we break down some of the considerations and steps you’ll need to follow to ensure that your chatbot adds to customer experiences, without degrading them at any stage.Download Now