INTRODUCTION
AI is poised to change and disrupt many industries, and among the first will be the customer support industry. This is because the two major subsets of AI, machine learning and natural language processing, are particularly well suited to many of the routine tasks and requests involved in serving customers. For example, AI-enabled bots can be used to perform basic functions to solve simple customer issues (i.e. updating billing information or changing passwords); gathering and learning from available data; and recording and recognizing customer preferences and behavior patterns.
Leveraging AI driven customer service bots can improve the customer experience by empowering customers to solve their problems quickly on their terms, while at the same time lowering support costs for more mundane issues and freeing up support professionals to work on more complex customer problems. Additionally, AI can be used to mine customer data to create targeted products and services, with the ultimate goal of pleasing and retaining customers and creating a more customer-centric and profitable organization. Yet, as desirable as these objectives are, designing and implementing a sophisticated AI solution that is seamlessly integrated into the overall customer support experience is no easy task.
BACKGROUND
On the heels of their winning the Big Kahuna Award for Excellence in Artificial Intelligence –The Rosie Award — at the 14th Annual Customer Contact West: A Frost & Sullivan Executive MindXchange, we created this case study to dive deeper into Microsoft’s implementation of AI-enabled customer service via a virtual agent at their web and Windows client customer support entry points.
CHALLENGE
Microsoft’s key challenge was to empower customers to self-solve and adapt to meet their needs with a faster, more personalized resolution for their issues or questions. This was driven by the organization’s observation that some of the easier and more repetitive customer support requests were taking longer than needed, and involved connecting with a human — not always a necessity for simple service issues.
After extensive testing and experimentation, the company introduced a virtual agent (VA) — an AI-driven chatbot designed to perform actions on behalf of customers wherever possible. As Gabriele Masili, Chief Technology Officer, Customer Service and Support at Microsoft explained, “Each customer has different preferences when they need help – we found that the best digital support solution was to present them with a smart, AI-powered virtual agent that could solve the issue at their pace, and yet offered the opportunity to connect to a human expert whenever the issue was too complex, or simply when the customer preferred to do so.”
The results attained were rather remarkable. Here is a brief overview of Microsoft’s AI implementation success story, and the tools and thinking that guided them in digitally transforming and improving the customer support experience.
STRATEGY
Although able to perform basic customer service functions, the Microsoft VA was designed to do more than just present self-help options; its purpose was to solve the ultimate customer issue. A specific, stated goal was “to trigger the right content for the right customer.” For example, as soon as a customer signed on, the VA utilized what it knew about the customer (order history, operating system, title of page query was launched from) to tailor their experience. A second strategic focus was to have the VA perform actions on the customer’s behalf whenever possible.
CONTENT
The Microsoft VA presented three different types of content experiences depending on the question or customer intent, with AI and machine learning manifested in the following ways:
1. Dialogs – Conversational, expert-curated dialogs (i.e. sequence of possible questions and answers), ideal for personalized self-solve actions
2. Instant Answers – Expert-curated short response, ideal for simple questions
3. Web Results – Non-curated content from articles on support.microsoft.com and forum threads on answers.microsoft.com, ideal as solutions for “long tail” customer intents (to address low-volume contact drivers)
As referenced above, a key implementation challenge included triggering appropriate customer content. Because customer questions are not usually fact-based, (i.e. what is the capitol of Kansas?) a sophisticated approach and model are needed to provide effective, AI-enabled customer service. Understanding contextual data, such as identifying the user’s operating system, is paramount. The VA needs up-to-the-minute information to prevent redundancies like asking a question that the customer has already answered, which often leads to customer frustration. Microsoft addressed these issues by developing and testing a new, very intelligent machine learning model.
TESTING
Microsoft conducted dozens of tests and experiments with the VA tool before launching. Some of the tests were conducted to determine how to improve the use of contextual data. Other tests sought to make the dialog more precisely targeted to the customer’s real time issue and/or location. Additional experiments were conducted to take advantage of natural language processing and machine learning techniques. These included:
• Sentiment Analysis – The VA identified the sentiment of user messages and took special action to quickly help users who exhibited highly negative sentiment about their support experience
• Machine Reading Comprehension (MRC) – Using MRC, the VA automatically previewed and summarized support articles related to the user’s query
• Autosuggest – As a user typed their problem statement, the virtual agent suggested the most likely issues they might be facing. These suggestions were based on data from historical chats
All testing and experiments were done with the end goal of refining and improving the customer experience. Specific AI-enabled goals included improving the quality of customer service content, leveraging context more extensively, improving triggering functions and adding more options to fix customer issues.
RESULTS
Two core KPIs were used to evaluate the success of Microsoft’s VA implementation:
Improvement in Problem Resolution Rate (PRR) and Increase in Customer Satisfaction (CSAT).
PRR is defined by Microsoft as an indicator of the number of issues solved by the VA. Specifically, at the end of the conversation, the user is asked: “Did this solve the problem?” Then they are given the options of “yes” or “no.” There was a dramatic increase in the PRR as follows:
Increase in Problem Resolution Rate (PRR)
• PRR increased by 239% for top 20% of traffic in approximately one year
• PRR increased by 185% for all traffic in approximately one year
CSAT is defined by Microsoft as an indicator of overall customer experience with the VA. It is presented as a 1 to 5 star survey that asks customers to rate their experience and is calculated by averaging all CSAT responses received. The goal was to improve CSAT scores for the top 20% of traffic, and there was a big improvement in these scores as well:
Increase in Customer Satisfaction (CSAT) Scores
• Nine month data collection results: CSAT improved by 31%
In addition to significant increases in PRR and CSAT scores, additional testing showed that as a by-product, VA implementation deflected 12% of chat and call volume, leading to corresponding support cost reductions, and saved time for support professionals so they could focus on higher-value and complex customer interactions. Additionally, the Microsoft Customer Service and Support VA served as an ongoing research and model case for productizing the technology. Other successful early adopters include Macy’s and HP. Lastly; Microsoft just launched a private preview of what is now called Dynamics 365 AI for Customer Service, which will be widely available this year.
CONCLUSION
As evidenced by an improved overall customer experience that included higher PRR and CSAT customer experience scores and lower operational costs, the initial implementation of the Microsoft VA provided a digitally-driven, more personalized and efficient customer experience. The results showcase one important first step forward in a long and exciting digital journey to come for this technology leader.
