The unseen risks of AI
In this episode of In the Loop, Insurance Insider’s Abbie Day is joined by Melanie Hayes, CMO, board member and co-founder at Kynd, to explore the business risks posed by AI and what they could mean for you.
Together they unpack how AI can create hidden liability exposures for businesses, where those risks might sit within your portfolio, whether silent AI is the new silent cyber, and whether AI could eventually become a line of business in its own right.
This transcript was generated using AI and may contain errors. Please check against the audio before quoting.
Abbie Day 0:04
Hello everyone and welcome to this week's episode of In the Loop, where we bring you in-depth discussions about AI and the insurance industry. I'm Abbie Day, Senior Reporter for Insurance Insider, and today we are digging into the business risks that AI poses and how it might affect our listeners. With my guest Melanie Hayes, Board Director and co-founder at Kynd, we will be discussing how AI can present a potentially unseen liability risk to businesses and where those risks could fall within your portfolio. Silent AI being the new silent cyber, and whether AI could end up standing alone as its own line of business. Let's jump right in. Melanie, welcome.
Melanie Hayes 0:50
Thank you very much, Abbie.
Abbie Day 0:51
If we could start maybe with just a quick brief intro to Kynd. Tell us a bit more about what you guys do.
Melanie Hayes 0:55
So Kynd is a cyber risk intelligence company. We started about nine, ten years ago now, which doesn't seem particularly long, but in cyber it's kind of like dog years. And there were three of us that started back in Brixton, and now we work with insurers, brokers, MGAs, reinsurers across the world, predominantly in the US and the UK. And specifically, what we do is we look at an organization from the outside in. So we look at the cyber risk affecting a company with just a domain name and instantly tell you the cyber risk that could be affecting that company, whether their emails are spoofable, whether they're vulnerable to a ransomware attack. And it's live and accurate data. And when we created Kynd, it was built specifically for the insurance industry. So we weren't cyber security trying to sort of give insurers information. We really focus on the information cyber insurers actually needed. And part of that was around silent cyber, understanding the cyber usage and the cyber risk within there. And what's really interesting is not only do we help insurers around the world now with risk collection, portfolio monitoring, and also understanding accumulation risk, but also the nascent part of it being next to AI. So AI is really sort of a natural progression to what we started as a problem in the cyber insurance industry, to problems that we have within AI and identifying AI risk.
Abbie Day 2:20
I mean, that puts us perfectly into our topic of conversation today, talking about the risks that AI can present for companies and insurers. Maybe you could just talk a bit about what kind of risks AI does present to a company using it, and maybe a bit about what lines that that would fall into. Is it a cyber risk?
Melanie Hayes 2:37
It's a really good question, and it sounds like it's really simple to answer, right? What is AI risk? What is the exposure around it? And over the last 12 months, we've been hearing so much from our clients, from insurers, reinsurers, MGAs about AI risk, which made me and everyone at Kynd really want to get under the hood of what that is. Now we're not cyber insurers, but what we do is we work with a lot of them. So we did a massive piece of research where we talked to them about what are they seeing, what are the claims, what are the losses, how is it working for them, how is it being embedded into what they're doing to really get an understanding of what the industry thinks about it. And we did this massive piece of research speaking to them. We did some live sessions where we had some live debates, which got a bit lively, I'm not going to lie. We did some webinars as well, where the themes we were talking to. And I think what was really interesting is the description of AI risk to most of this group. I cannot stress highly enough how much they didn't agree on, but what they did agree on is that AI risk to them meant it's the exposure where a business hands over a decision or an output to an AI system.
Abbie Day 3:48
Yeah.
Melanie Hayes 3:48
That risk, it's not that the risk is AI doing something wrong, it's that it makes a bad judgment call. So it's doing everything it should, but the judgment call, the decision at the end of it, isn't right, and that is where the risk lies. And I think what was also really interesting with this group is they weren't too concerned about a major AI hack or a cyber breach because of AI, some sophisticated attack. The thing they were most concerned about is not actually understanding or seeing disclosed AI usage and the signals of usage, so it being undeclared. And in terms of where AI sits, now immediately people think AI and cyber, the two just go together. And actually, you couldn't really be further from the truth.
Abbie Day 4:34
Okay.
Melanie Hayes 4:35
So the insurers that we spoke to, they talked about the harms of AI, where they see that falling. For example, within recruitment. So you've got an organization using an AI hiring tool to do screening of applicants. And in that screening, the AI excludes a certain group of people, say anyone who lived in Brighton. I'm just saying that. Using that. Now, that's not a cyber risk. That can come under employment liability, that can come under D&O. It's a risk. It's not because AI did that. If you had a manager who was discriminating in the screening process, that would also be a liability. You've got examples of copyright in IP damage. So, for example, a newspaper said that AI had been taking their articles and recreating the news based on those articles and claiming it as their own. So they sue them. So that's kind of like copyright law, copyright infringement. You've got leaked financial information or leaked personal information because people are putting things in AI that they shouldn't be through there. I think we all understand how that can possibly happen. There's all these things that AI have an influence on or actually start from AI, but actually where the loss happens or where the claim happens hits different insurance lines. E&O and tech, P&C. I mean, we're in a lovely room now. If the sprinkler went off here because agentic AI is being used to actually set the timers on here and something happened, the time changed or something, AI's not doing anything wrong, but suddenly water comes leaking through here. You've got damage to the walls, you could be hurt, I could be hurt. So we could so there's many, many different lines there that it can affect. So it's not just within cyber. In fact, cyber doesn't always cover things. So if there's a data leak or a data breach, AI may be involved in it, but it wouldn't have caused it, so it wouldn't even be covered under cyber lines. So
Abbie Day 6:24
okay. I mean, this all sounds quite reminiscent of the silent cyber of a few years ago. I've heard from speaking to people in the market differing opinions on whether this is silent cyber 2.0 or if this is something new. What's your take on how similar silent cyber and silent AI are?
Melanie Hayes 6:42
So I'm on the side of the fence where they are so similar. Okay.
Abbie Day 6:45
So
Melanie Hayes 6:46
Kynd started because of a problem in insurance, and that was we were speaking to some insurers, and they were saying to us, look, we're really interested in cyber insurance, it's a new thing for us. But what we're really struggling with is understanding their cyber usage or the risk within there, because we've got application forms which are questionnaires, they're disclosing things on there, but we know that that's there. Um but we need some tech to help us understand that and see it easily, and that's how Kynd was created to kind of combat that silent AI. And you can tell a lot from an organization from their domain name with our tech that we can do and scan really quickly. And we're seeing the same thing in AI. So Verizon did a really interesting report, and what they were saying is that in the last 12 months, AI usage within businesses quadrupled up to 45% of users. Now, this report came out two or three months ago, and obviously the research was done a bit before then, so I suspect it's even more. But of that 45%, let's just use that number, two-thirds of those users weren't using company systems, they were doing it through their own personal systems, so the organization didn't know about it, aren't able to disclose it. So that's a huge risk there because how can you underwrite something you can't see? And that's where we are with silent AI. You absolutely need to, as a starting base point, be able to identify the usage of AI or the signals of AI usage. And interestingly, that's something that we've been really focused on at Kynd, and we've recently launched our AI detection service because that's got to be the starting point. Visibility is key alongside the guide rails that organizations need to have in place.
Abbie Day 8:24
Yeah.
Melanie Hayes 8:25
And one of the things I found really interesting when we were talking to insurers and some of the debating points was around governance. And one of the people we spoke to on the underwriters said they felt AI should be treated like an employee, clear rules, setting limitations, and the organization being responsible for the output. So if you treat every AI that you use within there, then you're going to do better than if you weren't treating it like that. And I think that was a really sensible approach and advice.
Abbie Day 8:51
The topic that we've covered on Insider before of where that liability lies where something goes wrong, is it with the company using it? Is it the person using it? Is it the creators of the AI? It seems quite murky at the minute. I wanted to dig in a little bit more into this shadow IT topic you were talking about. Can you tell us a bit more about that, about staff using perhaps AI that companies don't know about yet?
Melanie Hayes 9:15
I don't think people are doing it deliberately. People want to do a good job and they want to do things that make it easier. And sometimes they're not aware that these things are AI or that they shouldn't be putting information into this.
Abbie Day 9:28
Yeah.
Melanie Hayes 9:28
And that's where it comes down to the organization to be really clear and have the guardrails in place about if we're just use Claude, for example, it's a workplace Claude. So you're allowed to link into your systems, but then you don't allow it to train for other models. That information's kept in there and it's not displayed elsewhere.
Abbie Day 9:46
Sure.
Melanie Hayes 9:46
So there's some simple things that organizations can do, but people have phones, people have different usages from it, and it happens a lot. And it's like, okay, this is the bit that needs to be understood. And this is where, for us, we can detect when there's AI usage happening, and not all AI is risky, and there's some that are more risky than others, and it's identifying which ones those are so that an organization can kind of understand the crown jewels or their data, what can and can't be shared, what shouldn't be shared, and make sure there is protection around it.
Abbie Day 10:19
Okay, one of the phrases that I've heard so often, especially these last few weeks, with kind of conferences and things like that, is governance is super important. You've mentioned it as well. Is just strict AI governance and auditability and things like that enough to minimize the AI liability risk? What role does governance play?
Melanie Hayes 10:39
Governance is really important, and if you've got good governance in place, it's absolutely a guardrail in terms of what's happening. However, things are moving at a real pace and trying to keep ahead of it and the governance catch up to it, it takes a long time for things to actually be written in, to actually come out. So at the moment, I would say governance is a bit patchy, particularly in North America. So in the US, they have a mix of rules dependent on different states. And actually, the courts are doing a lot of the heavy lifting in terms of mandating where things are. I don't know if you're aware of it, but the Air Canada case. You had an Air Canada client, and unfortunately, they suffered a bereavement. So they went onto the Air Canada website and went onto the chatbot and asked if they were to refund their flight or change their flight details because of the circumstances. Would there be any penalties? And AI bot did what it did, looked at the information, and said no, there'll be no penalties. And so the client on that basis changed the ticket. However, that wasn't the case, and that wasn't Air Canada's policy. So they refused to honour what the chatbot had said. And that went to court. And the Air Canada's argument was well, it wasn't us giving you the information, it was AI, and AI made a mistake. However, the tribunal felt differently and said you can't remove liability. It was on your website, it was your chatbot, it was that information, so they were held accountable for it. So, in terms of governance, there's courts that are doing a lot of the heavy lifting in that area. In the EU, we're actually moving quite quickly, I think, which is a good thing. And there's laws coming out. There's a new product liability law coming out in December, which is probably going to have a big impact on what can happen on there. So governance is moving at a pace and it will help. But the guardrails, the basic hygiene, those things need to be in place. So we're rushing to keep ahead, we're rushing to use this technology. Don't forget the basics. And we said this in cyber, and I'll say it again for AI, it's exactly the same thing. You need to have the basic hygiene. And I'll also go back to what the insurer said on our panel about treat it like an employee, put those same rules in place. Yeah. And you'll be in the best place that you can be.
Abbie Day 12:47
Maybe you could just add a bit more context on what the basics are for anybody who might not know already.
Melanie Hayes 12:54
So one of the things is making sure you do your updates and your patches. They're pesky, there's lots of them. They seem to come out all the time now. But make sure that they do that because there's lots of things that change that aren't supported unless you're doing the patch. And as a hacker, the thing that you're looking at, the thing that's really easy information for you to gain is which systems are no longer being supported, which ones are vulnerable. And you can see which organizations are using them. So why would they bother trying to sort of go through encryption and do all the hard work when there's things that are easily makes a company exploitable and vulnerable? So it's doing those. Stuff, passwords. It sounds so basic, but it's still something. And sometimes organizations don't make it easy. It has to be changed every two weeks, it has to be changed every month, and they don't have password managers in place. So people have to remember these things. And when you have to remember it, you do things that are easy, which also means it's easier for the cyber criminals to say
Abbie Day 13:52
password with an extra digit.
Melanie Hayes 13:53
Exactly. All the different advice that people are given, famous phrases, to be or not to be. So making it easy for staff with password managers, having security protection, also understanding which data is valuable. So those are your crown jewels, that data. Let's ring fence, let's make sure that's protected. We know how to protect that, how to do that. And training of staff. Do you know you're expecting staff to be the gatekeeper of your security and employees are, But train them, do regular training, simulated phishing attacks, different things. So people understand and are aware of what's happening out there.
Abbie Day 14:26
Amazing. And I think we kind of say human in the loop, but that is a lot of responsibility to be like, say, the gatekeeper for this brand new technology.
Melanie Hayes 14:35
100%. But people have to be. Yeah. Being aware of things, locking down your computer, not sharing details. You've got to be super wary of emails that come in because the cyber criminals are really clever. It's not like back in the day where you could immediately tell it was a spoof email.
Abbie Day 14:53
Yeah.
Melanie Hayes 14:53
We've seen examples of where people have the scripts from banks. So they're calling up people. I'll just make up a bank name, but just say Barclays, for example, that I have the script and they say, We've noticed some fraudulent spending on your account. So you've got three payments that have gone through in your account, and we just need to take you through security details to make sure. And this person has had these things spent on their account because it's the cybercriminals who have just done it. And so they'll be like, okay, let's go through this. And they're like, let's do the security questions first. Okay. And then there's an added measure of security because this is card fraud. We're going to send you an authentication code. I need you to read that out to me just to make sure that we're talking to the right person. So you get the code through, and then the person reads it out. They don't realize they're not talking to the bank.
Abbie Day 15:38
Yeah.
Melanie Hayes 15:38
And what's happened is when they're trying to access their real bank is sending them a security code to make sure it's them and they've just given it to the scammer. But it sounds like a someone who works in that bank, there's no accents, there's nothing to give you any flags that that's not happening. It has actually happened. So they're going to have
Abbie Day 15:57
starting questionnaires for the bank to make sure that they are the right person.
Melanie Hayes 16:00
Well, the first thing you do is put the phone down and then find the number and call them directly. And also a bank will never, ever, ever say, I've just sent you a code, read it back to me.
Abbie Day 16:10
That's good to know. I mean, with these companies, I this feels like it's happening already. For companies that may not know these things, have there been any claims coming through to insurers? Is it sort of an influx that people should be wary of in the next, say, year or so?
Melanie Hayes 16:26
So we're not underwriters. Yeah. And we we don't have access to claim and loss data. But what we did have access to were a whole load of senior leaders in the insurance industry, and we asked them this exact same question. And what the consensus was amongst that group is yes, there are claims, but they're not necessarily being recorded as AI. Because if you remember at the beginning of the conversation, we talked about scenarios, but they come into other different lines of insurance or different types of losses. So yes, they are coming in, but they're not currently being recorded as AI. And what they also said is we're probably receiving claims, but we just don't know it yet. That will come out a bit further along the line. Interestingly, there was a report from Spain where they actually reported the first AI attacking itself and a breach that happened because of it. So that's the first time we've seen anything like that. And that happened about four or five weeks ago. So, yes, claims are probably coming in, but they're not being recorded like that. But going forward, that may happen. What they're also in a consensus about is that the loss information isn't there yet, and that probably won't be for a while.
Abbie Day 17:33
So until that time, we are just dealing with the silent AI, I suppose, and trying to reserve for it.
Melanie Hayes 17:38
Well, we're calling it AI, but actually it's falling within other policies. It's it's how the losses happened that determines which policy it comes within. And one of the big questions and one of the big talking points in the research that we did in the webinars and the live sessions was should AI become its own line? That particularly interested us because we saw cyber become its own line. It moved from D&O, it moved from PI and became its own line. Is that going to happen with AI as well? And interestingly, the room was divided.
Abbie Day 18:11
Yes.
Melanie Hayes 18:11
Now we had some people in there who were actually starting to write AI themselves and they think absolutely this is the way to go. I'd say the majority that we spoke to didn't feel that it could become its own line at the moment because it hit so many different policies. They could write exclusions in there or they could incorporate it within different things, but they didn't feel just yet it could become its own line. And some of them are saying they just don't know enough about it yet. How can you actually talk about AI here? I what I found really interesting, there's, I don't know if you've heard about it, read about it this week, was the case where someone in the US created a load of songs using AI, right? Like hundreds of songs using AI, distributed them on the streaming services, Spotify, Apple Music, but then created a load of bots who then downloaded all those songs and kept repeat buying those songs.
Abbie Day 19:02
Oh,
Melanie Hayes 19:04
so they were created by AI, they were listened to by AI, and he generated about $8 million in about four years, I think, is the timescale. So that went to court. He pleaded guilty, so he received a jail sentence and also had to pay back eight million bucks. So people are going to be ingenious, really, in terms of how they use AI and what can be used, but also going back to the governance points and the courts doing the heavy lifting, that's fraud.
Abbie Day 19:29
Yes. It
Melanie Hayes 19:30
doesn't come under AI, doesn't come under cyber, it just comes a plain old fraud.
Abbie Day 19:34
I heard someone describe it as trying to cover AI is like trying to cover water. Or
Melanie Hayes 19:40
trying to boil the ocean, yeah.
Abbie Day 19:43
It's like water comes under so many different policies in so many different ways. Unless you can define flood. Same with AI, you have to actually know what the risk is to cover it.
Melanie Hayes 19:54
Exactly. And it's going back to that point. You need to be able to see it, you need to be able to identify it and then the risk associated with it, and then for insurers to make the call, this is more risky than this, this is what we'll price against. Those different areas. And I think something that's really topical at the moment, the PHP to shell, that's just happened this week. And that was a flaw that's been embedded for 22 years and not been discovered until we had AI. And luckily it was the good guys that discovered it. But 70% of the world's websites had this critical flaw embedded within it. So there's these risks that AI can expose, and we want to make sure we're on the side of it where we understand it and we can do things about it. But the impact of it being found from a good way was phenomenal, right? I mean, the US government's website came down because they wanted to fix this flaw. I really don't know anyone that didn't have something around this that needs to be in there, but actually identifying, well, where are these risks? How do we stay ahead of them? What do we need to look after it? So it's moving so quickly at the moment.
Abbie Day 20:57
Yes. And I've heard so much about how much impact the insurance industry has had on cyber hygiene and things like that. It seems optimistic that, or hopefully, I guess, that insurance will help make those guardrails and hygiene around AI as well.
Melanie Hayes 21:11
I think sometimes insurance gets a really bad rap, right? Insurance doesn't make anyone better when you're buying car insurance or you're doing property insurance. I think that maybe is a fair statement. But what cyber did, it stopped being an insurance that just looked at renewals because that snapshot point in time, what a company looks like from a cyber point of view, can be very different three weeks later. In the cyber world, things move by the millisecond. And at Kynd, we monitor all the time. So once we have an organization, we just need the domain name, we're constantly monitoring them. So something changes in their cyber profile, we're able to sort of alert them. So we don't just work at risk selection, we also work with the whole portfolio over the lifetime of its policy. What cyber insurance don't want to happen is we've given you a cyber policy, see you next year. They want to make sure the risk is minimized throughout the life cycle. And so they've done a great job of providing tools, services, help, not saying all their clients use it, but they should be, but they're generally helping them. So you buy cyber insurance, you have all this help available to you. And if something, God forbid, does happen, your insurers are there to actually help you with the incident, to actually help you get you over that. And I'm not sure all organisations realise how much your insurance company can actually help you.
Abbie Day 22:26
Yes. Amazing. Well, I think that's everything to cover today. Thank you so much for chatting to me, Melanie. I could talk to you about this all day long.
Melanie Hayes 22:33
Same, same, Abbie. It's a really pleasure. I know we were speaking earlier about a lot of things, but this has been really interesting. I think we both could talk for hours on this.
Abbie Day 22:40
Well, we'll wrap it up there for now, but hopefully we'll have you on again soon. Thank you so much.
Melanie Hayes 22:44
Lovely, thanks, Abbie.