Introduction

AI Services from AWS, such as Amazon Lex, and Amazon Connect are used by organisations across the globe to build and deliver rich multi-modal self-service and conversational applications. Blog articles such as Creating a secure IVR solution with Amazon Connect and Enhance Your Contact Center Solution with Automated Voice Authentication and Visual IVR detail how organisations can build modern self-service applications securely, and deliver them over a variety of customer interfaces and channels. These applications can use conversational AI (and more traditional means if necessary) services to deliver rich customer experiences, and where needed, engage customer representatives to assist in meeting the customers needs. At Joulica, we provide analytics to answer questions such as: -

1. Where in the flow do customers most frequently opt to engage an agent?
2. What are the most frequent multi-intent journeys and are customers satisfied with these experiences?
3. How is each IVR or Conversational AI application performing over each channel?
4. How can Bots and Contact Flows be improved to decrease the frequency of customers abandoning automated services?
5. Which customer intents are responsible for high levels of repeat interactions and how does this change across channels?
6. customer satisfaction and sales?
7. How can external and custom data be used to segment the analysis?
8. How can results from A/B tests be analysed to continuously iterate and optimise flows?

In this article, we will use several examples to work through some use cases. All Joulica analytics are available for historical analysis using Quicksight and other BI tools – with the focus here on flow analysis via Sankey visualisations.

Core Analytics

The first example detailed here is based on the following use case, which is implemented with Amazon Connect.

1. Contact Flows have been created that handle customer interactions for each major line of business within a large retail bank. A single Contact Flow handles mortgage interactions, another handles credit card enquiries etc.
2. Integration with Enterprise applications allow these contact flows to be data driven, and hence driven by a combination of customer data and external configuration.
3. Amazon Lex is used for Conversational AI and both Voice and Chat channels are utilised.
4. Several Lex Bots are implemented, with each Bot servicing several Intents.
   Customers are routed to contact center agents when needed from within the Contact Flows, allowing important context to be preserved.

The following diagram shows an example of how customer interactions flow through the application.

The Contact Flows are shown on the left-hand side, with the outcomes shown on the right-hand side (in this example we are using the following outcomes: Automated, Abandoned and Queued (in which case the queue name is shown), but these are flexible and can be customised). Interactions are broken down by Channel, the Intent (e.g., Transfer), and in cases where the interaction was abandoned or queued, the last Lex slot that was not filled (e.g., Payee entry). Joulica allows users to drill down further, with the diagram below showing that

1. The Transfer Intent has a high level of automation (82 %).
2. The Pay Bill intent has a low level of automation (26%) and most customers (54%) fail at the point where the Bill Type is being specified.
3. The Statement intent performs much better on Chat (85 % automation) versus Voice (19% automation.

More Complex Applications

The next example builds on the above use case, but adds

1. Multi-intent conversations, e.g., customers who utilise the self-service application to carry out multiple transactions. These can often lead to lower levels of automation.
2. Customer satisfaction scores (NPS is used for this example, but CSAT or Customer Effort score can also be used) based on a simple survey being added to the flow - e.g. see Create post call surveys in Amazon Connect.

The figure below illustrates this example. There are more Contact Flows, Intents and Queues listed, but in addition there are multi-intent conversations (one example shows customers who start by checking their balance and then go onto paying a bill and then finally ordering a statement). Given that these are the actual journeys that customers are using, it is important that analytics can be provided for these flows.

Whilst the example shown here gives a birds-eye view across the full set of self-service applications, users can easily filter out Contact Flows, Channels, Bots or other items to focus on specific journeys. If automation (or containment) rates are of interest, users can drill into specific flows. To begin with, the example below shows that 72% of customer interactions are automated. Next, it shows that out of the customers who check their balance, a significant number go onto pay bills, order statements, and other intents, with the overall level of automation for this intent dropping significantly to 44%.

If users are interested in analysing how customers journeys through the application effect NPS or customer satisfaction, it is straightforward to zone in on the flows of interest. The example above shows that 46% of customers provide a promoter rating, with this increasing to 77% for mobile chat customers. Promoters account for only 35% on the voice channel, with the Transfer intent only garnering 3% - which is largely responsible for the poor voice performance.

In cases where custom segments of a Contact Flow need to be analysed (e.g., for A/B testing), Lambdas can be used to send custom data to Joulica, which allows analytics to be broken down by these custom data elements. Of course, flows can be broken down by attributes set directly within the Contact Flow or elsewhere in the Amazon Connect solution (e.g., the agent desktop).

Summary

In this article, we have detailed just some of the IVR and Virtual Assistant analytics that Joulica provides. Other capabilities include drill-down to individual journeys and higher-level metrics such as authentication rates. It is also straightforward to analyse performance over specific time periods. To find out more, get in touch by email at info@joulica.io or contact us and we can arrange a demo.