Show Setup

Tom Morgan 0:02

Welcome to ACP, the Amazon Connect podcast. This is the show that focuses on Amazon Connect and related technologies. I'm your host, Tom Morgan, and this week I'm running solo in the ACP studio. Find out more about Cloud Interact by visiting us at cloudinteract.io.

0:37

It's time for another ACP, and yes, it's just me this week. Alex is away on a special mission and so, uh, we're not able to get together to record and, logistically it was just a little bit too tricky to get together remotely as well given what he's doing. But that's okay because I've got a good plan for what we're gonna do this episode. A bit of an experiment.

AI Agent Call Plan

1:01

So we talk about AI agents a lot on this show, especially in the past six months. It really feels like that's been the focus because, I mean, that's the focus of where Amazon connect customer has been looking. So we've talked a lot about what AWS have announced, what people think, whether the demos, you know, whether they live up to the hype. But I don't think we've ever actually played you one on the show. I don't think we've ever actually really showed you how it sounds. Showed you so you can listen. And so that's what we're gonna do this episode. So we're gonna do five real calls. Four of them are demo environment calls uh, they're taken from our own demo environment that we use for customers, and then one is a clip from a video I did earlier in the year, on a conference show floor, just so you can get a sense of what that's like talking to an AI agent over lots of noise. I'm gonna talk about each one. We'll listen to it, and then I will explain what's going on underneath.

No Editing Promise

1:52

Three kind of really important things I, I think to get out the way before we start. Every recording that I'm gonna play is exactly as it comes out of Amazon Connect customer. So there'll be no editing, there'll be no gap trimming. I think, like being completely open with you, I think the tool, the software that we use to produce the podcast is gonna normalize it. But there, there'll be, there's no gap trimming and there's no-- we haven't taken anything out and there's no retakes. Everything is exactly as it gets exported from Contact Search. Okay? So if there's mistakes in there, and there are mistakes in there you'll, you know, you're gonna hear them. So the environment we are taking these from is our demo environment. It's Cloud Interacts environment that we use to demonstrate different technologies to customers. That means that all of the names, the account numbers, the medical details, everything in there is fictitious. I think the only real thing in there is my email address, but, um, don't be weird. Everything else is completely fictitious. You'll hear a couple of different voices. You'll hear my voice as the, the human side of all of the calls. For the AI agent, you'll hear some different voices and that's, that's deliberate really. Uh, th- the voice is a configurable choice. It's something you get to choose, and it depends a little bit on the region, so it depends on the uh, the country you choose, and then i- within that you get choices of different voices, there really is no one AWS voice. These are some of the things you're gonna hear.

How Agentic Voice Works

3:08

First of all, you're gonna hear what an AI agent in Amazon Connect customer actually sounds like. So it's not that kind of press one for sales. It's not a chatbot with canned replies. What we're showing you is the agentic voice capabilities. It's a large language model. It's built on top of that phone line experience. It's got things backing it, so it's got a knowledge base, which can be a company's own documents and knowledge about how the company operates, its standing operating procedures, its FAQs. It's a set of tools, so things that it can call and use to look up a booking, read a transaction list, check a calendar, send an email, that sort of stuff. And it also has a set of rules about what it's allowed to do, what it's not allowed to do, which is equally important. So it understands when it needs to hand over its call. It needs to hand it over to a human agent. It can talk, it can look things up, it can take actions, and it has rules. The second thing to tell you is about the voice. You'll hear the voice sounds really nice, and there've been some real changes to the voice in the past few months. So Amazon Connect's new agentic voices run on all new technology, and that's where that kind of natural pacing, the turn-taking comes from. So things to listen for as we're going through the different demos are the pauses, the interruptions, the talk over, how it copes when the caller changes direction, and some of the emotion. And that sounds like an odd thing to say, but just listen out, you'll hear what I mean. Some of it's a little bit gimmicky and some of it is, yeah, a little bit like eye-rolly. But actually I think it's important if you take it away from the demo and put it into a real world scenario, you actually want to get that confirmation signal that the thing you're talking to understands the seriousness or the intent of what you're talking about. And I think actually those, those emotion cues really do help to do that.

Demo Platform Context

Tom Morgan (2) 5:01

Just a note on where I got these recordings from. So Cloud Interact has a demo platform that we use to showcase the agentic AI capabilities of Amazon Connect customer to our existing customers, to potential customers, and we do that across a wide range of different industries and verticals. So we put together a demo environment to allow us to do that. They're all based roughly on the same platform and pattern. They have a dedicated knowledge base of industry-specific information. They use identity verification where it's needed. They have a set of scenarios as well that the agent can handle, and then they've got defined handoff points to a human agent, so we can demonstrate the whole kind of end-to-end story. That's how, uh, four of these clips at least were made. I wanted to explain that just so you know that they weren't made specifically for this episode. They're not curated artisan recordings. We do hundreds of these a month, and I just cherry-picked a couple just to pull them out into here so you could hear what they sounded like. And they're Exactly as they came out the demo

Airline Demo Call

Tom Morgan (2) 6:04

environment. All right, let's start with the first clip of the episode which addresses a question I get all the time actually, which is: Will it actually be able to understand me? Will it be able to understand what I'm saying? So for our first clip, we're gonna go to the airline industry. So this is all about flight times and prices. So a caller is gonna phone up and they're gonna ask about flight times and prices for a particular route. There's a bunch of follow-up information there, things like, "Do you want it direct or non-direct? Do you wanna go one way or return?" And then it's important that the agent can then present the right information and then the caller wants to go and think about it, so the agent is gonna email them. things to listen for on this recording. The first one is the email address. So email addresses are really hard. They're a classic kind of failure point for a lot of voice systems. All of those numbers and dots and underscores and domains that sound like other words as well. So listen for how it handles that. Listen for how it works and whether it captures it cleanly and what it does with it, and listen to how it confirms it as well, which is kind of a really important step. And then secondly, I would say just listen for how it delivers this information. These are kind of lists of numbers, and they can be quite dull, so listen for how the automated voice delivers that information. Okay, let's play the clip.

AI Agent 7:26

Hi, I'm an AI agent for the airline's customer service team. How can I help you today? Whether it's about a booking, a flight, baggage, or anything else related to your travel with us, I'm here to assist.

Tom Morgan 7:39

Great. Thank you. Um, I'm looking for a ticket from Stockholm to Amsterdam. Um, um, I don't know times and prices

AI Agent 7:49

Perfect. I'd be happy to help you find a flight from Stockholm to Amsterdam. Let me gather a few details so I can show you the best options. Are you looking for a one-way ticket or a return journey? And how many adults will be traveling?

Tom Morgan 8:03

Um, a, a one-way. Uh, yeah, just me, um, please.

AI Agent 8:09

Great. So that's one adult, one way from Stockholm to Amsterdam. What date would you like to travel?

Tom Morgan 8:17

Uh, the twentieth of September.

AI Agent 8:20

Perfect. And just to confirm, would you prefer a direct flight or are you open to connections?

Tom Morgan 8:27

I'd prefer a direct flight, please.

AI Agent 8:31

Excellent. Let me show you some direct flight options from Stockholm to Amsterdam on the twentieth of September. We have three departures available that day. There's an early morning flight departing at half past six, arriving at half past eight. Then we have a midday option leaving at noon and arriving at two o'clock in the afternoon. And finally, an evening departure at half past five, arriving at half past seven. Which of these times works best for you?

Tom Morgan 8:59

Um, can I get a fare for the, for the early one, please?

AI Agent 9:08

Great choice. The early morning flight departing at half past six. For that flight, economy is priced at eighty-five euros, and if you're interested in business class, that's two hundred and thirty euros. Would you like to book one of these fares or would you like to think about it?

Tom Morgan 9:25

Uh, I need to go and get approval. Um, could you email me those costs, please?

AI Agent 9:31

Of course, I'd be happy to send you an email with all the flight details and fares. What's your email address?

Tom Morgan 9:38

It's tom.morgan@cloudinteract.io

AI Agent 9:44

Perfect. Let me read that back to you to make sure I have it exactly right. That's tom.morgan@cloudinteract.io. Is that correct?

Tom Morgan 9:53

Yes, it is.

AI Agent 9:56

Brilliant. Thank you. Let me send that over to you now Done. I've sent the email to tom.morgan@cloudinteract.io with all the details: the route, date, departure and arrival times, and both the economy and business fares. Once you've had a chance to get approval, just let me know and we can move forward with the booking. Is there anything else I can help you with in the meantime?

Tom Morgan 10:24

Uh, no, that's everything. Thank you.

AI Agent 10:28

You're very welcome, Tom. Good luck with getting that approval, and we look forward to helping you book your flight to Amsterdam. Have a great day.

Airline Clip Breakdown

Tom Morgan (2) 10:36

So this clip for me, in a way, represents the kind of the run-of-the-mill, the unglamorous, the everyday. This is the stuff that is really good for doing voice automation on. So things like collecting emails and post codes and reference numbers, and then providing information. This is a place that voice systems can get really hung up on; collecting email addresses, collecting reference numbers, collecting post codes. It's where users get really frustrated because it's just such simple information. Why can't you understand it? It's where you get a lot of fall-throughs to human agents as well. And so actually, if you can take these what are really simple requests, you know, these are not long, complicated calls that require empathy and emotion or negotiation or reasoning or, any of that stuff. They're really simple, essentially requests for data. It's asking some questions and then email me the details. If we can stop those calls getting through to a human agent then that's great because it, it allows human agents a lot more time to deal with the important stuff. Did you notice as well how it pulled my name out of my email and used it at the end of the conversation? Love that. the other thing to point out about this clip, uh, that you will have noticed, I think, this was done in a demo environment. That it's a clean line, it's a quiet room. I'm, annunciating and talking fairly normally. Keep that in mind because we're gonna make it a little bit harder later on. But let's start with the easy one, and then let's get progressively harder.

Banking Demo Call

Tom Morgan (2) 12:11

Okay, let's go to our second clip. For this one, we're gonna go to the retail banking industry, and there's a lot going on in this clip. There's a few things to listen out for because this is another three-minute clip, and I'm not gonna apologize for inserting these fairly long three-minute voice recordings into your podcast episode. It's a deliberate thing, and I really... I kind of struggle with this a little bit because these are not the highly curated 30 to 60 seconds for social media. They've got the gaps in it. They've got the, the lookup times. They've got the fairly slow conversation. But I wanted to really make sure that you got the full sense of how these conversations went in the real world. So I know it's three minutes long. Close your eyes and imagine that you are reviewing a call recording from your contact center. Would you want it to sound like this? Or close your eyes and imagine you are talking to a contact center, and is this how you want the conversation to go? So retail banking. Caller calls up their bank. They don't recognize a transaction on their account. This is a fairly common occurrence. Uh, it's also a fairly tricky problem. Identity verification needs to happen, and then the agent needs to walk the customer through a list of recent transactions until they spot the one that they don't recognize. So that will happen first, so listen out for how that happens. There's also a balance check 'cause that's another very common thing people like to do. And then there's a calculation as well based on a repayment on a car loan. And because that's a fairly complicated calculation, the caller asks for the details to be emailed. Finally, there is a request to change address, which is something that the agent won't do, but it will transfer it to a human. So listen out for all of those things. Listen out for how the agent handles reading out a list like that. Listen out for the tone of the call 'cause this could be quite a stressful call. Caller's calling up because they think they might have been defrauded. And then listen out for the different things that it does with the customer's data there, like reading it versus calculating it and reasoning on it.

AI Agent 14:11

Hello. Welcome to our credit union member services. How can I help you today?

Tom Morgan 14:16

Hi. Um, I've seen, uh, like a transaction on my account that I don't recognize

AI Agent 14:23

I'd be happy to help you look into that. Before I can pull up your account information, I'll need to verify your identity for security purposes. Can you please provide your member ID and PIN?

Tom Morgan 14:33

Uh, yeah. My member ID is one two three four, and my PIN is four three two one.

AI Agent 14:40

Thank you. Let me verify those details for you Great. Your identity has been verified. Now let me pull up your recent transactions so we can find that one you're concerned about. Here are your last five transactions on your primary checking account. Today, Amazon Marketplace for forty-seven dollars and eighty-two cents. Yesterday, there's a transaction labeled SQ Star MCHT four four two nine one for twenty-eight dollars-

Tom Morgan 15:10

Yeah, that's the one, that SQ one. I don't recognize that one at all

AI Agent 15:15

No problem. Let me take a closer look at that one for you. I've pulled up some additional detail on that transaction. It's actually from Sunrise Coffee Roasters for twenty-eight dollars and fifty cents on a debit card purchase yesterday morning. The reason it initially appeared as that reference code is that Sunrise Coffee uses a third-party payment processor called Square. Until a card transaction fully settles, which usually takes one to two business days, our system shows the processor's internal reference rather than the merchant name. Once it clears, you'll see Sunrise Coffee Roasters on your statement instead. Does that look familiar now?

Tom Morgan 15:48

Oh, I remember now. Yeah, that was me. Okay, that's helpful. Thank you. Can you just confirm my account balance then, please?

AI Agent 15:56

Of course. On your primary checking ending in forty-five twenty-one, you have a balance of three thousand two hundred and forty-seven dollars and eighty-nine cents. Is there anything else I can help you with today?

Tom Morgan 16:07

Uh, yeah. I've got a car loan as well. Um, I-- Could you tell me what it would cost me to pay it off today?

AI Agent 16:16

Let me pull that figure together for you. The total amount needed to pay off your auto loan today is $8,450.16. That's made up of your outstanding principal of $8,400- You

Tom Morgan 16:27

know what? I'm not gonna remember this. Could you email that to me, please?

AI Agent 16:32

Absolutely. I'll send that right over to you now Perfect. I've sent that payoff information to your email. Is there anything else I can help you with today?

Tom Morgan 16:44

Uh, yeah, I've just moved house. Um, can you, um, help me change my address, please?

AI Agent 16:51

I'd be happy to help you with that. Unfortunately, I'm not able to update your address directly through this system. For a change like that, I'll need to connect you with one of our member services representatives who can make sure your information is updated correctly across all your accounts. Would you like me to transfer you to someone who can help with that right away?

Tom Morgan 17:09

Yes, please.

AI Agent 17:12

Perfect. Let me connect you with a member services representative who can help you update your address right away. You are being transferred to a human agent. Please

Banking Clip Breakdown

AI Agent 17:21

wait a moment.

Tom Morgan (2) 17:22

All right, so in that clip you heard quite a lot of structured data being accessed. So the actual transaction list lives in a third-party system. It gets brought in via a tool call, so that'll be an API typically, uh, wrapped up in an MCP server. It comes into the agent as a JSON, blob, and then that gets translated by the model into something that you can understand and absorb being read down a phone. That's quite important and it's important to do so in a way that you're getting over enough information but not too much as well. The loan calculation is one of those things that sounds really simple when you hear it on the demo, but actually it's really quite involved because there's a bunch of arithmetic that goes on there in calculating the loan repayment based on the, balance and the time and, and all the rest of it. A year ago I don't think we would be recommending you do this, and the, the advice there is, you know, you don't let your LLM do maths, especially not on the phone, especially not in the finance sector, and especially not to do with car loans, right? So, the... we've-- That there's a whole design decision that goes into doing that. There's some non-deterministic calculations that go on there in calculating what the repayment term is. The important thing is that the LLM knows what those are. It knows what tools to call when. It knows what information to provide as inputs to those tools. There's enough validation around the inputs as well to make sure that mistakes can't be made. And it's a whole design process that we've worked out now to really get confidence in doing things like this. I think the key is that you don't realize any of that through the conversation. The, the AI agent doesn't say, "Oh, hang on a minute. I've got to transfer you to the loan calculation bot," or or anything like that. You don't realize the, you know, the AI agent makes some tool calls in the background and can carry on and carry the flow of the conversation. Um, but it is not doing that calculation. The calculation is being done elsewhere in a non-AI-led way. It's just the AI is delivering the information. The address change is interesting, right? Because listening to that, you might think, "Well, that's a fairly ordinary request. Why can't the AI agent handle address changes?" And we do do address changes in agentic systems and in lots of other verticals. This one is a bit unique because actually changing an address on a bank account is a fraud vector. So we have guardrails on that particular demo to say, actually, if it's changing address, it needs to go through to a person. I think the only other interesting thing about this clip is just the volume of calls that this would potentially be able to handle this particular use case. Because unrecognized transactions, those calls are one of the biggest drivers in retail banking, and they're also super time-critical because of the fraud risk. So it's kind of the worst of the it's the worst of both metrics, right? So triaging that from, is this actually fraud or did you just forget something, before bringing in a human agent is important for two reasons. First of all, you can quickly make anxious customers happy again. You know, if, if they can satisfy themselves, "Actually, there isn't fraud here. This was just me forgetting something or not understanding something," you're helping them quicker, which is great. It's also obviously good for the organization as well because, um, there's some savings to be made there if you can cut down the calls that are identified as real fraud and need, need real triaging. So that was quite an involved demo, quite involved scenario.

Simple Handoff Demo

Tom Morgan (2) 20:43

Uh, we're gonna go for a much simpler one now. This one's only one minute 30. And this one is just a simple thing that it can't do. So this AI agent isn't doing anything at all. This is a completely different vertical. It's to do with bathrooms. But we've told this agent that it can't do address changes either. So when a caller phones up wanting to change their delivery address, the call gets handed off to a human agent. I'm including this one because it's ordinary. It-- this is applicable to a lot of different businesses, and a lot of businesses fall into this ordinary kind of, "Where's my order? Can I change my delivery?" They're not banks, they're not hospitals, they're just regular mid-sized businesses. But it doesn't mean you can't get loads of value and benefit out of agentic AI on your contact center. So I'm including this even though the agent isn't doing anything. Why am I doing that? Two reasons. The first one is actually the agent is doing identity verification, and that's really important. And the second thing is, is that the agent is identifying the intent from the user. So when the call does get passed off to a human agent, not only does the agent know exactly who the caller is, they also know exactly what they want to do, and that's really important. So let's play the clip.

Bathroom Products Clip

AI Agent 21:57

Hello. Welcome to our customer service team. I'm here to help you with any questions about our premium bathroom products, your orders, or anything else I can assist with. How can I help you today? Before I can assist you today, I need to verify your identity for security purposes. Could you please provide your user ID and PIN?

Tom Morgan 22:17

Yeah, my user ID is one two three four, and my PIN is four three two one.

AI Agent 22:23

Thank you. Let me verify those details for you Brilliant, Tom. Your identity has been verified. It's great to speak with you today. I can see you have an open order for a premium shower enclosure that's due to be delivered in three days, and your guarantee is all registered and active.

Address Change Handoff

AI Agent 22:46

How can I help you with your order or any of our products today?

Tom Morgan 22:50

I'd like to change the delivery address, please.

AI Agent 22:54

I understand you'd like to change the delivery address for your order. Let me see what I can help you with regarding that. I'm not able to change the delivery address directly through this system. However, I can connect you with one of our specialists who can help you update that right away. Would you like me to do that for you?

Tom Morgan 23:13

Yes, please.

AI Agent 23:16

Perfect. Let me connect you with a specialist who can help you change your delivery address. You are being transferred to a human agent. Please wait a moment.

Smarter Transfer Context

Tom Morgan (2) 23:27

So the value here isn't in the human handoff, right? It's the additional information that goes along with it. It's the fact that the call has already been identity verified, and it means that when the human person picks up the call, not only do they know with certainty who the caller is, but they also know exactly what they want to do. They want to change their address. So that means that you could automate the systems that the human agents use to bring the right records up, to bring the right change address screens up. You can also save time on the call. The human agent doesn't need to go through that IDV process, so there's a real time saving there. That's thirty to sixty seconds of every single call that gets transferred, which is a real time saving. The other thing you can do is you can take all of these calls that you know are about address changes, and you can route them to somewhere different. So if you've bring in new people into your contact center, you can maybe train them just on address changing and then just route all of these calls to them to get them started for practicing, for training and all the rest of it. And so this can be a really nice way of introducing agentic AI into your contact center. You don't have to solve every single problem, and sometimes it can feel like doing AI in the contact center is just such a massive thing to bite off and do all in one go. It doesn't need to be like that at all. You can start with essentially the AI agent not being able to do anything apart from doing the, just the identity verification and understanding what it is that the caller wants to do and then transferring. There's nothing wrong with that at all. You're still saving time. You're still making the flow of calls through your contact center more efficient.

Health Clip Setup

Tom Morgan (2) 24:59

Okay, for our fourth clip, we're gonna go to the health vertical. So this is a really important vertical. It's obviously a vertical that is critically important to get things right, and so we've got a fairly involved clip here. And there's a mistake in this clip as well that's worth listening out for. So the caller's gonna call in. They're gonna go through IDV, that identity verification process.

IDV Options Explained

Tom Morgan (2) 25:19

And just a note on our IDV process, actually, in our demo environment, 'cause I, I do get asked about this quite a lot. You'll have heard by now in our IDV process, we ask for user ID and PIN, and that's just something that we put in our demo environment. It doesn't mean that's the only way to do identity verification in Amazon Connect customer. We have customers in production where the identity process involves asking for pins or zip codes or post codes or Social Security numbers, national insurance numbers, surnames, last names, date of birth, whatever it is. Basically, however it is you do your identity verification today, that's how you can do it in Amazon Connect customer. There isn't a set box that you have to tick in Amazon Connect customer that says, "Do IDV," and that's how it works. It's not like that at all. You set it up, you define the rules, you tell it what's important, and you can call out to systems to verify that information. We only do user ID and PIN in our demo system because it's transferable across many different industries and verticals, and it's a nice easy way for us to do the demos. So I don't want you to like hear the clips and think, "Oh, this is the only way that IDV has to happen. That won't work for us, so we're out." Not like that at all. So yeah, just bear that in mind. So coming back to our health clip, we're gonna do IDV. We're gonna book an appointment through a third-party booking system, which is good 'cause it's an agent doing something rather than just telling you about something. You'll hear there's some mishearing that goes on. The agent mishears something. So listen out for how that happens, how it gets detected, how it gets corrected as well. And then there's a pivot towards the end. So, the caller mentions some side effects, and so you'll hear how the agent responds to that as well. When listening to the clip, just think about or listen for how it handles the booking. You know, a booking is essentially choosing from a lot of different options, and then having one sort of committed and confirmed. How graceful is that? How did it work? You know, that correction as well, how did that sound? How was that transfer to a human agent? I guess everybody has had to phone up and try and book a, a doctor's appointment. In the UK, that's a bit of a challenge right now. In other countries I imagine it's different. But it often can be stressful. It can lead to a lot of, like, time on hold. Listen to this clip and see if this is sounds like it might be a better way of doing

Appointment Booking Demo

Tom Morgan (2) 27:39

things.

AI Agent 27:39

Hello and welcome. I'm here to help you with any health concerns you might have. Before we get started, I need to verify your identity for security purposes. Can you please provide your user ID and PIN?

Tom Morgan 27:51

Yeah. My user ID is one two three four, and my PIN is four three two one.

AI Agent 27:59

Thank you. Let me verify those details for you right now Great. Your identity has been verified. Now I'm here to help you with any health concerns you might have. What brings you in today? What symptoms or health issues can

Tom Morgan 28:15

I help you with? Um, yeah, I missed, um, an appointment a couple of days ago. I need to schedule it again, please.

AI Agent 28:23

Of course, I can help you reschedule. Can you tell me what that appointment was for? And do you have a preferred day or time that would work better for you?

Tom Morgan 28:31

Hmm. It was a, it was a routine checkup, routine health checkup. Um, I don't really mind. Next week's looking fairly free. I prefer afternoons, I think, if possible.

AI Agent 28:42

Perfect. I can arrange that for you. Let me find some afternoon slots next week with Dr. Sarah Mitchell at Riverside Family Medicine. How about Tuesday the 16th at two thirty in the afternoon, Thursday the 18th at three fifteen, or Monday the 22nd at two o'clock? Would any of those work for you?

Mishearing and Confirmation

Tom Morgan 28:59

Um, can we do the Thursday session, please?

AI Agent 29:03

I want to make sure I have the right slot for you. When you say the first day session, do you mean Tuesday the sixteenth at two thirty in the afternoon?

Tom Morgan 29:12

No, I, I meant Thursday. Sorry

AI Agent 29:17

No problem at all. So you'd like Thursday the 18th at 3:15 in the afternoon? Let me confirm that for you Perfect. You're all set. I've booked your routine health checkup with Dr. Sarah Mitchell for Thursday the eighteenth at three fifteen in the afternoon. Your booking reference is HC seven four two nine. Please arrive about ten minutes early and bring your photo ID and insurance card. I've also emailed a confirmation with all of those details to you. Is there anything else I can help you with today? Is there anything else I can help you with today?

Medication Side Effects Escalation

Tom Morgan 29:53

Yeah, no, there is. Um, so, uh, I'm taking medication for my cholesterol, um, and I've, I've noticed that, uh, immediately after I take it, I get really bad swelling in my legs. Is that normal?

AI Agent 30:07

I'm sorry to hear you've been experiencing swelling in your legs after taking your cholesterol medication. Because this is a new side effect you're having with your medication, I want to make sure a clinician reviews this right away. Let me connect you with someone who can help you properly. You are being transferred to a human agent. Please wait a moment

Tom Morgan (2) 30:25

All right.

Booking Tooling Breakdown

Tom Morgan (2) 30:26

What did you think to that call? Let's think about the booking first. So there's a few different things going on there, because reading appointment times from a system, that's tool use retrieval. And then the booking itself is a writing back to that external system as well. That happens when the agent decides that it has enough information, so it gets told about the, the tool call to that external system, and it knows the minimum amount of information that's needed before it can make that call. And so it makes sure it gathers that, and that could be through a process of either knowing who the caller is through the IDV process or specifically asking in terms of, making the caller pick one of the available slots and all the rest of it. This is one of those things that's really important to get right. In some ways it's more important than if just the wrong information was given because you're making an actual booking or committing data in an external system as well, and that's bad because obviously if the booking's wrong, you've got bad data and all the sort of things, consequences that flow from that. But then also you're taking a booking from somebody else who can't have it as well. So lots of good reasons to make sure that's right. Lots of good design goes into making that feel like a seamless process as well. And then that misrecognized where it kind of misheard what I was saying. Now, some of that's probably on me, whether I was muffled, whether I didn't pronounce properly, doesn't matter, it happened. It's more about how it got corrected. You can kinda hear there how the model took what it thought it heard and kinda sense checked it and thought, "That doesn't sound quite right. I'm gonna confirm it." Uh, which is kind of a good sign that misrecognition is what you want. Because obviously mistakes happen. People speak with accents, there's background noise, there's people talking to other people whilst they're also on a phone. All those things happen all of the time. And so having a system that can detect that, understand when the data it's getting isn't quite right and do something with it that's, uh, that's really important. It's one of those things as well we're gonna come back to, but it's where things like evaluations and rules are really important to make sure you evaluate all of your calls that are handled by AI agents so you've got real confidence in

Guardrails and Safety Handoff

Tom Morgan (2) 32:28

what's happening. The other really important thing here are the guardrails. So we have strict guardrails on this demo that describe the things that should go to a human agent immediately, and that includes side effects of medication. You don't want that kind of conversation to be had by an LLM with its kind of improvisation and, and making stuff up. Compared to the last couple of clips, which were more about handoffs that happened because of capabilities or policies, this was a safety handoff. And it's, uh, it's really important to get right, and it's one of those ones where you want it to happen immediately without a long kind of explanation from the LLM about what's happening next. A really quick handoff, uh, is really

Amazon Connect Health Overview

Tom Morgan (2) 33:07

important here. As a reminder Amazon Connect Health is one of the four products that Amazon Connect became in April. This call I think is a great example of why healthcare got its own product line. Because yes, you can absolutely do all of this stuff in Amazon Connect customer. Amazon Connect Health has all of this plus a little bit more. It's got some default guardrails, it's got some default tool calls, it's got a integration with Epic. It's got some AI agents kind of out the box for handling prescription refills, appointments, insurance, and things like that. But this is yeah, this is an area where really good design is really important, that, you know, the, the, the rules have to be stricter, the cost of error is higher, uh, and that's why Amazon Connect Health exists. But it's, it's worth evaluating the two. There's nothing that says you can't use Amazon Connect customer for your healthcare calls, but it is worth evaluating Amazon Connect Health as well to see if it's

Noisy Conference Agent

Tom Morgan (2) 34:03

a better fit. And then finally, the last clip. This one's a little bit different from all of the others. Everything else up until now has been taken from our demo instance and I wanted to showcase something a little bit different. So this is from earlier in the year when I went to a conference and I built an AI agent to basically tell me what to do at the conference. So it had the knowledge of the conference schedule, it knew what the day was and the time, and so I could call it up and ask what should I go and do now?" And it knew what I was interested in and gave me some good, specific, actionable answers. And the reason I'm showing this clip is because that conference floor was really noisy. Really, really noisy. You know what conference floors are like, right? So, I was really interested to see how the conversation would go. Would the AI agent be able to even understand me? Would it get really confused with all the other voices in the background? Would it be able to keep up with what I was saying? So I'd like you to take a listen to this and just if you've been to a conference recently, do you think it would be useful to have something like this to call up? You know, should the-- should conference organizers deliver and provide AI agents that you can call up as part of the package as well as all the other things you get when you attend a conference? So here's the clip.

Tom Morgan 35:15

Hi there. Um, I've just registered. I've just got my badge, just picked up my badge, just got into the expo floor. Um, what should I do now? What should I prioritize this morning?

AI Agent 35:28

Great timing. It's just after half 9, so you've got a good chunk of the morning ahead. Let me ask you this: Are you here mainly for Amazon Connect sessions, or are you looking to explore more broadly? If Connect's your focus, I'd suggest heading straight to the demo pods on the expo floor. They're brilliant for getting hands-on with the tech and meeting AWS specialists and other customers. Or if you want to kick off with a session, there are a few good ones coming up this morning. What sounds more appealing right now?

Tom Morgan 35:57

Yeah, that sounds great. I'll, I'll go have a look at the pods now. Uh, what time's the first Amazon Connect session that I should, uh, be interested in?

AI Agent 36:07

Perfect. Let me pull up the Connect sessions for you so I can tell you what's coming up this morning. Perfect. So the first Amazon Connect session is a lightning talk at half 11 called AI-Powered Customer Service with Amazon Connect. It's only 15 minutes, so it's a nice quick intro. That's in the New to AWS area on floor one. After that, you've got the Bank of Ireland case study at 10 past 12 on floor three, which is brilliant if you want to hear how a big financial services player transformed their contact center. You've got a good hour to explore the demo pods now before the half 11 session. Sound good?

Tom Morgan 36:45

Perfect. Sounds great. Thanks

Robustness in Real Noise

Tom Morgan 36:46

very much.

Tom Morgan (2) 36:47

How do you think it went? I thought I was really impressed, honestly. Uh, genuinely impressed when I recorded that because it did so well. And it was really noisy. And I know it's really hard to... i'm trying to show you these clips in the most honest way I can. Obviously this is one call, one good outcome. Doesn't really prove robustness or anything. Like, it demonstrates it, right? But I think this last clip is a fairer test than the, the demos w- you know, with the no background noise. And so I think that's good. It's good to show it and, uh, I don't know. If you're at all on the fence about whether this is gonna work, I would say build one out and go try it. Do what I did. Put a phone number on it, take it out somewhere noisy. If you're worried it's not gonna work on your factory floor, just build one, take it out to the factory floor, make some test calls. Talk to us, we'll set up a demo, take it down to the factory floor. We've done this in restaurants as well for for chefs who are wanting to reorder food. We've done it in the, uh, for airlines in sort of busy airports and runways and things like that. Just go and try it, because I was really pleasantly surprised, 'cause conference halls are really noisy. Take it down to the train station, take it into the shopping center, take it wherever it is that you go to that gets

Design Testing and Metrics

Tom Morgan (2) 37:57

busy. All right, so that was five test calls, variety of different scenarios, variety of different outcomes. Couple of things we didn't talk about, the kind of back-end machinery, the stuff you won't kind of hear and it's hard to demonstrate. Each one of those operates on a set of rules a set of knowledge, uh, a set of prompts about what it can and can't do, where it needs to hand off to a human agent, what it's allowed to talk about how it should talk, you know, the tone, How it should deal with customers and all the rest of it. That's a whole bunch of kind of design and testing that goes into doing those. It's what makes good agents. It, you know, it's part of agentic design. And getting it right is what makes AI agents dependable in a way that I think people who say that they're unpredictable, people, people that say AI agents are unpredictable, I don't think fully understand. Well-designed agents with good guardrails and good prompts actually can be. So the models are probabilistic, but the scaffolding around them is deterministic, and that's your safety layer, right? So good design can create really good agents that are not only good to interact with and are great user experiences, but are also safe and that organizations can have confidence in. There's loads of tools in Amazon Connect Customer for confidence as well. The evaluations and rules that you can create in Amazon Connect Customer, we would always say really important to QM 100% of your AI agent calls with those evaluations, with those rules, so you can really understand what's going on, have the confidence, 'cause it's a big deal, right? Putting these AI agents in front of your users as the first voice that they hear when they contact your organization. So you wanna be super confident that they're doing the right thing, and those tools in Amazon Connect Customer let you do that. Those evaluations, the rules all of those things, and all the monitoring that's in place allow you to do that. In fact, since about April Connect has had an AI agent performance dashboard in it. It's got eight new metrics, three that I really like. The goal success rate, so that measures did the caller actually get what they came in for? The faithfulness score, which is the answer that they got, is it grounded in the knowledge base, or was it just made up? That's your hallucination detector. And there's also a tool selection accuracy metric, which is measuring did it call the right tool in the system? Uh, the right tool use. Did it call it at the right time in the right place? So some of the things that you heard in the clips earlier, such as that kind of duplicated question or mishearing those are kind of things that you might catch with those metrics. Definitely don't go to production without having a good measuring strategy in place, having a good way to know that the agents you're deploying are doing the right thing. Super, super important.

Agentic CX Designer Update

Tom Morgan (2) 40:57

Just before I go, we'd normally cover news items at the beginning but I'm gonna do this one at the end, uh, just because it's directly related to some of the stuff we were just talking about. So earlier this month AWS made the Agentic CX Designer generally available in Amazon Connect customer. So this has been in preview for a little while. It's the thing that came out of the NLX acquisition, um, but it's now called Agentic CX Designer. It's a no-code visual canvas for building AI self-service across voice calls and digital calls as well. AWS frame it as the clarity of a flowchart with the power of a language model. If you're familiar with building call flows, it's a similar kind of canvas, but it's really focused in on building AI self-service flows. And so if this had been available when we were building out our demo system a, you know, a year or so ago, I think we would absolutely would've used it. We're in the process of moving everything over. It's very exciting. It's a lot of capability, a lot of power, but also there's a lot of really good tools, again, for getting that confidence in testing. So a lot of extra ways to do testing before you deploy. A lot of simulations, so you can build different test scenarios and simulations and run them multiple times in an automated way. You can place automated calls. You can use large language models to place test calls to your call flow to see how it works, and then you get really good visualization about where, you know, how the call is flowing through the different flows, and then loads of diagnostic information as you go through. I think this is a really good evolution for Amazon Connect customer. I think it's interesting. So I think the no-code visual canvas, I think no-code is doing quite a lot of heavy lifting there. I think the design is still really important. So designing that, that whole flow, when to hand off, what should be in the prompts, and what the guardrails should be, that's a design thing. And so I think that's really still very important. I think this tooling makes it easier to ship something, but it will make it easier to ship something mediocre as well. So, that's just something to watch out for. You know, the tooling kind of lowers the barrier to entry, but it shouldn't lower the bar, right? You still need to go through those design steps. It's still really important to consider all of those things. This just makes it easier to build them once you've decided them.

Wrap Up and Feedback

Tom Morgan (2) 43:18

All right. This has been a really fun episode. We've looked at five different calls all on Amazon Connect. We've gone from the kind of the boring to the complicated, to looking up lots of information, to making mistakes and correcting ourselves, to being able to get things done, to understanding when it's really, we can't do things and we ne-need to get human involved, and it was still working when I was shouting at it on the conference floor. So it's not perfect. You could hear it make mistakes. The point of this episode is not for me to come here and say everything is perfect. It isn't. Is it measurable? Is it fixable? Uh, yes. Could you use it to improve how your contact center operates? Yes. All right. There's just me talking, and I've talked at you for quite a long time, so I would like you to talk to me. So I'd like some feedback from this episode, please. Of those five clips we heard, which one surprised you the most? And is there a scenario that you didn't hear that you want to hear? So I've got access to that demo environment that I pulled all the clips from, and so I don't think anyone will mind if I went and recorded some new ones. So if I get enough requests for a particular scenario, more than happy to to create it and record it, and we can bring it back to a future episode and talk about it and see how it goes. So let me know. Uh, email me at podcast@cloudinteract.io. I would like to know which of the clips surprised and interested you the most and why, and which scenario you'd like to hear next. Looking forward a little bit it's gonna start getting busy. I think the summer season is well and truly over. We've got re:Invent sort of late November, early December time. We've got Gartner's CCaaS Magic Quadrant refresh is due any moment now, I think. Uh, we've got the Amazon Connect User Group events coming throughout the year, and we are busy lining up guests for future episodes. We've got a couple of really interesting people we're talking to, so look out for that. And of course, Alex will be back with me on the

Final Sign Off

Tom Morgan (2) 45:20

next episode. As it turns out, I could talk all day, but it is time to bring this episode to an end. Thank you all for listening. Be sure to subscribe in your favorite podcast player. That way you won't miss the next episode. Whilst you're there, we would love it if you would rate and review us. If you have colleagues that you think would benefit from this content, please let them know. To find out more about how Cloud Interact can help you on your contact center journey, visit cloudinteract.io. We're wrapping this call up now, and we'll connect with you next time