Alerts aren't enough anymore #09

[00:00:00] So that's why looking at the wider behavioural analysis of an alert and looking backwards as well over time of what did this look like a week ago, what did it look like a month ago and bringing that data set together to be able to understand if this is a real attack or whether it might be just a benign positive. Difficulties arise when you've got a role that maybe bridges two departments or a senior member within a business.

[00:00:29] They're the difficult ones because clearly they might need to have access to multiple different areas. I think the clue that we need to work more on at the moment is the data science around cybersecurity. We need to be analysing these large data sets as organisations, not just as the big vendors, but as organisations so that we can have a better understanding and a better picture of what is going on in our environments at all times.

[00:00:58] But we need to bring that kind of thinking into our internal teams and looking at data to drive decisions rather than, you know, just a one-off alert, you know, a one-off alert, you know, for example.

[00:01:10] Today, you are looking at why detecting alerts isn't enough and why understanding behaviour and context matters with my guest James Samminton, now Cyber Infrastructure Director and who was Head of Cyber Security at the time of the recording. I am Michael Virgon. Welcome on my podcast Cyber All Day.

[00:01:41] By the way, if you listen in audio, you will have the timestamps in the description. John, here's the first part of my conversation with James. We can get started. So thanks for your time. Before we start, but actually, if you can introduce yourself just so we know a bit more about you. Yeah, yeah. So I'm James Hamilton, Head of Cyber Security. I've been working in the security space for about eight years now.

[00:02:05] You know, previous to that, I've been working in IT for all my working career from leaving school back in 2005, which seems like a lifetime ago now. But yeah, always had a strong passion for IT. I started out with system administration and IT technical support and found a passion into cybersecurity. I never looked back since. It's such an exciting topic and space to work in.

[00:02:34] Every day is a challenge and there's something new to be looking at every day. Yeah, last time you told me you would almost do it for free. Yes. It's good to get paid. We shouldn't share that too wide, but I absolutely genuinely love the job. And I often say that I would do this for absolutely nothing because it's so much fun. That's fine. That's fine. So what we can do, I just get started on the first part.

[00:03:00] When we spoke together about a different topic, one thing that you said, oftentimes I'm going to repeat verbatim when I wrote from you. You said in prep that the game is changing in a security operation. And specifically, you mentioned that we need to look at traditional security operation differently. So to get started with this particular topic, my first question, which is a bit broad deliberately, is what do you think specifically is changing?

[00:03:28] And as well, like on what aspects in particular should we look differently at what essentially? Yeah, I mean, I think when you start to delve into this, you know, security operations and the traditional way is changing. That's with the rise of AI products, I think, and the speed of things that are happening now. And there's various other things that are contributing to that change.

[00:03:53] But when we look at the traditional security operations analysts and the structures that were built around these teams, they're built to detect on very specific signals. They detect the detector breaking, don't they? We're looking for those breakings, a very specific action. But when we start to look at the behaviors and the techniques that are being used today and then being chained together by AI, that ability to just look for a breaking isn't as clear cut as it used to be.

[00:04:22] Now we're having to look at much wider telemetry and behavioral patterns of an actor that is accessing an environment. And what I mean by that is, you know, the abuse of identities and known good is becoming more apparent. And we're seeing these attackers, you know, stealing real credentials, real session tokens. So to be able to actually say, well, we've got a burglar that's got a balaclava on breaking in through our front door is very difficult these days.

[00:04:52] Because the person that's coming through the front door has got the key. They've got the code to the alarm and they're impersonating a real person. So to be able to actually see that through a traditional alert is becoming more and more difficult. And that's why we need to change the behavior of our security analysts and start looking at more heuristic behaviors of an attack or of a legitimate action as well.

[00:05:18] Because there are very subtle differences that we're seeing now in the way that a user will log on and it will be a valid session token. But they might take a very, very slight different route to being able to get that login. And it's something I've only spoken about just this week with a friend of mine who runs an AI startup company. We spoke exactly on that point around the differences between somebody driving down a motorway to work in the morning.

[00:05:46] But then normally in the traditional sense that we've got at the moment to spot that difference of the normal person driving down the motorway to driving down one of the side roads to get to work is quite a big difference. But the subtleties in the AI space now are very much smaller than that. And what we're looking at is you might be driving three inches to the left rather than to the right.

[00:06:08] And those pieces are very, very difficult to detect on the current alerts and structure that were built into our analysts and our monitoring solutions. So that's why looking at the wider behavioral analysis of an alert and looking backwards as well over time of what did this look like a week ago? What did it look like a month ago? And bringing that data set together to be able to understand if this is a real attack or whether it might be just a benign positive. Yeah, yeah.

[00:06:39] I've taken a few notes on what you just said. So I'm trying to make my mind what to get started. Because you know, you said often the bungler right now, they might actually have the keys. It's kind of as well as they might already be working there. You know, they're already in the proper uniform. You actually might be talking about the bungler to the guy already working there, which is actually himself or themselves, the bungler. Yeah, absolutely.

[00:07:06] I mean, the insider threat is real, isn't it? And, you know, we know that, you know, DPRK and various sort of nation states are absolutely putting people into Western companies and applying for Western jobs with that exact cause in mind. And so you're absolutely right at the internal threat to a business. And there's obviously just internal curiosity as well.

[00:07:33] You know, a user could potentially be accessing an area just because they're curious to see what's in there. It doesn't mean that they're going to do something malicious. But, you know, there's still lots of different avenues that could come from a genuine internal person that is doing something that is out of the normal.

[00:07:49] And again, I think that then that ties into looking at behaviors because, you know, a user from finance would have very little need to then start accessing production systems or doing things on a server, you know, doing a sysadmin requirement on a server. That behavioral difference becomes very easy to spot them because you're doing something, you know, every day of your life. And then all of a sudden you're doing something very slightly different.

[00:08:17] That is where we need to be spending more of our time investigating and following up on because they're the places that the attackers are now using to exploit organizations. You just mentioned right now about, you know, a different population in a company, which of course have different roles, so different behaviors, which are normal for their role. Actually, in the previous podcast, I was thinking like a side of technology. How do you define in a company what is normal?

[00:08:47] Especially because company to company, different industries, different legacy, different organization, everything can be different for each company. Especially, for example, if you speak to your peer, you know, somewhere else. Like, how do you define what is actually normal behavior in your company for different population? Do you have like a map that you guys create or like what type of process? I'm actually curious about this to understand a bit more.

[00:09:16] Yeah, I mean, it's actually quite simple, really. We just operate on a role-based access and we determine talking with the various departments, you know, what is it that you need access to and what is it you need to be able to perform your function or job. Clearly, always keeping a least privileged approach in mind. But it becomes relatively easy once you start having those conversations with departments or individuals that have got a specific task to complete.

[00:09:45] And once you've figured that out, then you can start building those profiles out to support them and give them the access they need to be able to do. And then that figures out what normal would look like. That was over a bigger data set, clearly. And you need to monitor over a long period of time to know what that normal is. But I think the bigger challenge that many, many organizations have is the permission creep that we have in a lot of organizations.

[00:10:10] For example, when somebody moves from finance to procurement or from procurement to customer services, you know, having strong controls on that review process when you have an internal mover and then going through a full permission review to say, well, you're moving out on that department now. So now you don't need that permission. You need to have this permission. And this is what normal now looks like. Really important.

[00:10:34] And that's where, you know, lots of organizations, I think, will be struggling because it's a lot of time that that takes up. You know, there's lots of movement that goes on within a business. There's lots of promotions that are going on all of the time. And all of those pieces need to be reviewed every single time to ensure that you've got a consistent and secure model going forwards or at least to be able to understand what normal is.

[00:10:58] Yeah, because you just mentioned permission creep, which made me think actually an example I had in my French podcast and a guest talking with me and my audience about PowerShell. Of course, he was saying PowerShell is super useful for IT teams. But if you don't do the hardening correctly, random users may have access as well. But for them, not the same. They don't actually have a use for it normally. However, for attackers, they're quite liking it a lot because it's a path often used.

[00:11:28] It makes me think about the map I was asking you about. Because like you said, if someone changes a role, it would be useful actually to know a bit more precisely, like you said, what permission now you have to remove or give them access or these sort of things. Yeah, and this is part of that review cycle, isn't it? Actually, when you've got those permissions mapped out against a role, then the review cycle becomes somewhat easy.

[00:11:53] Because if you are moving out of that into another role, then clearly you pick up the new permissions. I think, again, where some confusion and difficulties arise are when you've got a role that maybe bridges two departments or a senior member within a business. They're the difficult ones because clearly they might need to have access to multiple different areas. So, again, it's about being as granular as possible with the permissions that you're granting.

[00:12:19] The bigger challenge, I guess, again, is when we start to look in with IT departments and IT teams. You know, historically, we have got lots of access to everything, haven't we? We know we have to be SharePoint admins and global administrators. And by the sheer nature of having those permissions means that we can activate our own destinies and our own different routes through systems.

[00:12:41] So, again, a lot more scrutiny needs to be put onto IT teams and IT accounts, certainly where there's privilege involved. You know, earlier you mentioned some change in the behavior of the attacks. As well, you mentioned some subtle differences. Like, can you explain a bit more what you mean in particular about this?

[00:13:04] Yeah, I mean, I think this is attributed to AI and the significance of AI at the moment, not just from a business perspective to improve productivity and, you know, improve business performance, but from a security perspective as well. And it's an interesting one because AI isn't necessarily developing or creating new tactics to attack organizations with. The tactics that are used are static.

[00:13:32] An AI agent has been trained on the MITRE attack framework. So, those tactics that are in place there, it knows about. But the thing that is changing, again, I've been speaking about this week, the thing that's changing there is in the way that these tactics are and vulnerabilities in systems are being chained together at speed. And this is the thing that's changing for us now because, you know,

[00:13:57] a human wouldn't have the time to be able to chain the attacks together in the way that an AI agent is. And this is why sometimes these gaps that the AI agents are finding are such small things that we would never have seen them as analysts or penetration testers. It is only possible by that ability to crunch and look at large data sets in a very short amount of time.

[00:14:25] So, this is where I think we're seeing the biggest change in the way that attackers are exploiting organizations at the moment. And interestingly, you know, I know that the security space is doing everything it can to keep up in the AI space, in the AI race that we've got going on at the moment, certainly from a defensive space. But it's difficult as well. You know, the attackers always, you know, there's always that analogy that they use that an attacker only needs to be right once,

[00:14:54] whereas defenders need to be right every single day, 365 days a year, don't they? Yeah. So, you know, when you said it's hard to keep up, it makes me think like, I'm sure you will appreciate this yourself and being in the same security. You know, you have a lot of salespeople like myself prospecting you, offering you new products. It's kind of as well, like the more the technology evolves, especially with AI,

[00:15:18] it's like attack surface becomes wider because now you have things that you didn't know were actually a problem. So now you have potentially a solution fixing or helping you want to mitigate this threat vector. But actually, it just makes it every time more and more. You know, you think, OK, now I'm secure, fairly secure. But actually, wait, this area that you never thought about it, now actually it's a problem. So it's kind of making it bigger each time.

[00:15:46] You're always behind, of course, but it's always an issue. Yeah. And again, you know, the AI race is changing that. It's the speed of discoverability on vulnerabilities. And we see this in the statistics that are presented through monthly reports from various different vendors that, you know, vulnerabilities are on the increase. You know, that vulnerability management is becoming more and more of a challenge for businesses.

[00:16:10] And especially when we start to say that, you know, critical vulnerabilities under certainly the UK cyber essentials scheme needs to be patched within 14 days. Well, that number is growing and growing and it's just making teams more and more stretched. And it is difficult to keep up with that. So in preparation of the podcast, you said we need to bring more analytics into play and start looking at user behavior.

[00:16:34] So can you explain a bit more what did you have in mind in terms of analytics that you think would be useful, easier to expand or just to have in general? Yeah, I mean, I guess what I meant by that was, and a lot of this already exists in various different products. And I know that a lot of the vendors are working in space and have been for some time, you know, long of the days of where, you know, we used to have signature based releases.

[00:16:59] And, you know, we had to wait for the vendors to release the signatures that they'd sandboxed and then say, well, the LS is a threat. And then they would release that out to their customers and then you would be protected. I mean, long gone are those days. And, you know, we've been running on, you know, what we would say next gen antiviruses for some time now. But I think the clue that we need to work more on at the moment is the data science around cybersecurity.

[00:17:25] We need to be analyzing these large data sets as organizations, not just as the big vendors, but as organizations so that we can have a better understanding and a better picture of what is going on in our environments at all times. Okay. Yeah. Like analyzing patterns, these sort of things in the data for more specific entry organization. Absolutely. Yeah. I mean, again, this is, this is, it's not something new with this.

[00:17:49] This is what vendors have been doing for a long time, but we need to bring that kind of thinking into our internal teams and looking at data to drive decisions rather than, you know, just a one-off alert, you know, a one-off alert, you know, for example, as a security team internally, we do a number of internal tests and we can generate alerts at any point because the actions that we're taking might appear to be malicious actions.

[00:18:16] But over time, we should be able to build a behavioral model that goes, well, actually, you know, James's security team does this testing every month and we know it's going to violate alerts. So therefore, over time and over a data set, you should be able to say, well, no, that is a benign positive or a false positive. And therefore, we don't need to investigate that. These, and you can flip that on its head as well to say, right, well, this behavior of this person, and this is where it loops back into the behavior of a user doing their daily task.

[00:18:46] We should be able to pattern match that against what is normal. Yeah. You know, the funny thing is, and I'm saying this as a sales guy, always talking about user behavior in my roles. The ironic thing with this is, if the attacker manages to realize what is normal, it's actually easier to blend in because now you just do exactly what is actually normal to do for this person or for this user.

[00:19:15] Actually, it's very funny when you think about funny. It's not really funny, but it's very ironic. Actually, what you think is helping you to detect what is abnormal, different, is actually the thing that makes it easier for them to blend. Is there already inside? Yeah, it's absolutely that, isn't it? And that's what attackers are wanting to do. They want the maximum amount of time in a system where they're going unnoticed.

[00:19:40] So blending into what normal people are doing is a smart tactic. Now, I think the only thing that we do still have in our favor is that, you know, a legitimate employee wouldn't need to cause impact to a system. They wouldn't need to deploy ransomware. They wouldn't need to go and search for a privileged account. So there's still certain telltale signs that we should be able to pick up on. You know, a normal user doesn't need to elevate their permissions to a domain admin.

[00:20:10] There is no use case for that. So again, we can absolutely say, yes, they will blend in and they will do certain tasks. But at some point in that attack chain, they're going to want to initiate their impact phase or they're going to want to commit fraud or there's going to be an action that is benefiting the attacker at some point. And that's the bit where that behavior should then change and we should be able to pick up on.

[00:20:35] Yeah, but what I'm thinking of this is it makes me think like, you know, when we drift into insider threats, you said earlier that sometimes it's not even being malicious. It's just a curiosity. Sometimes it might just be this, you know, like they don't have any bad intention or sometimes it can be just negligence or very simply. They're very curious. If I open PowerShell, what happens and I've seen this on YouTube, GPT or Claude or whatever.

[00:21:03] So it's kind of like as well, you have to balance the permission that you give. Otherwise, people might do things they're not supposed to do, but just because. Yeah, they're just curious or they try. But even if they're curious, it doesn't mean that we shouldn't be following that up. Even if it's an internal employee doing something like that, it's certainly something that we should be following up and having that conversation with them. And, you know, ultimately that can benefit teams as well, because we can understand, well, is this an area that you're interested in?

[00:21:32] Is it an area that you want to expand your knowledge in and learn more in? And if you do, come and sit with the security team. Come and sit with the guys that run the infrastructure so that we can teach you about this safely rather than going off on your own and being curious. Maybe that isn't appropriate. Maybe what you should be doing is sitting with us and we will help develop you. And potentially there could be a role switch there for somebody if it's an internal person, you know, to move into a different team within the organization. And that drives career development and career progression then.

[00:22:02] In preparation of the podcast, you told me about identity-based attacks. And there was something that was brought on as well by another guest I'm preparing a podcast with. You know, the increase of this type of attacks in particular. Like, how could a company actually mitigate these risks without being too much, essentially a pain or too much, you know, give too much friction in someone's job on a daily basis?

[00:22:29] Because it's kind of like two sides of the coin. Yeah, I mean, it is increasing. And I think it's increasing because it's easy for us to blend into what a normal user does. I think when we look at our applications that we consume these days, they're not on-premise applications necessarily anymore. Everybody's using cloud now and SaaS applications. So, you know, it makes sense that the attack surface has moved more to identity-based stuff.

[00:22:56] And again, if you take that even deeper, you know, we probably started out with a lot of password spraying and brute force attempts. And then, you know, we've fixed that with putting MFA in place. And now that there's plenty of bypasses around MFA and, you know, various other things, we kind of solved that by putting conditional access policies in place to say, right, we want to analyze what you are doing when you are connecting. And are you coming from a company-joined device or is it up-to-date, whatever it might be.

[00:23:26] And then now we're seeing that obviously people are abusing tokens and session tokens that are issued. And again, now we're catching up again and we're deploying techniques in a way that we can start interrogating the session token that's in use. And, you know, there's new stuff that's coming out now where the session token for your login will be tied to your TPM chip on your device. So, again, we're making it harder all the time.

[00:23:51] Now, a lot of these changes, I think they do generally go under the radar for users and they don't really cause that much friction for them. I think the biggest one is MFA. I think it's widely adopted now and it's probably accepted as a really good thing to have in place, even in your personal lives. But I know initially that that was certainly a friction point for users. But, you know, again, we've still got to respect the basics there, haven't we?

[00:24:15] And we do a lot of work on looking at what types of all the strengths of passwords that people are using. We can't get away from passwords fully at the moment. I know passkeys are being introduced heavily now across a lot of systems. But, you know, passwords are still around. And we do need to move to a passwordless world. But I still think we're with some time off of that. And we've got to make sure that we're looking after the basics of setting strong, complex passwords, changing them somewhat regularly.

[00:24:45] And certainly after we know that we've had any issues, making sure that MFA is in place, doing everything that we can to protect these identities. On top of then looking at this behavior layer of what does a user do day to day, then we've got a fighting chance, I think. During the prep, once again, I'm going to refer always to this. You mentioned about threat actors like Conti as a group, which actually had a playbook leaked a couple of years ago, I think 2021 or 2022.

[00:25:14] And you said, so using still the same techniques, you told me we can do a lot of learning about that. You're always trying to guess when attacker is actually going on next. For example, when you look at this type of information, how do you actually think about the path that they're going when you look at the behavior they're using or leveraging in the attacks? Yeah, I mean, this is well documented and well researched from the big vendors.

[00:25:42] Microsoft do a fantastic job on their threat intelligence, as do Google and Mandian. They do fantastic work in the threat intelligence space. And we need to make sure that organizations and teams are looking at threat intelligence as being published because there's lots to be learned from the behaviors of attacks that we're seeing. Not only that, but these articles that are produced often include the indicators of compromise that we need to be looking for within our environments.

[00:26:09] And it's about taking that information, putting it into our security technology stacks to make sure that we're up to date and we're tackling this stuff. But I think we go a bit further than that. And one of the things that doesn't happen all that often is organizations between themselves don't share lots of information. You know, it's still quite a taboo subject. A security breach for a company is still a taboo subject. And we've got to break down the barriers of that.

[00:26:36] You know, there's certainly been some high profile breaches within the UK over the last couple of years. And while we've got some ideas of what those attacks entailed and the impact of those, there's still not much in the way of IOCs that have been shared or specific information. And I appreciate it's difficult. Sometimes you can't share this information, but you flip that on its head as well.

[00:27:04] And if organizations spoke more freely about indicators of compromise or learning lessons of where, well, actually we were breached because of this gap that we had, every organization would be more secure then. So it is one of those where it's a double-edged sword. I think it's very difficult to share this information. And rightly so, it should be difficult because, you know, it can be damaging to your brand and various other things. But by sharing the information, we would be helping every organization in protecting themselves.

[00:27:33] So you said companies need to share more. Obviously, they can be quite sensitive or dependent on the company. But it's kind of as well, like now, you, how can I phrase this? It's not like that you don't really trust the others. But of course, you can always control just what you can control, which is in your environment. So it's kind of like as well, now you share potentially very sensitive things. But actually, there might be leaked through other means.

[00:28:00] So it's kind of like, you know, you have to think, okay, I share this in confidence in some respect, but potentially, because I don't control the information anymore. Once it's gone, it's gone. It potentially is going to be gone in the wild if anything happens to the company or people that you share it with. Absolutely. Yeah. You're 100% right, Michael. Yeah. If the data is already leaked, then it's leaked, isn't it? And what are we trying to protect then?

[00:28:27] But there is an opportunity as security professionals that we could then protect another business from the same fate, potentially. Again, it is difficult. I know in certain circles that this does happen. Certainly, you know, tier one banks certainly have sharing groups and they're able to talk about this stuff with various different peers.

[00:28:46] But when we start looking a bit further down, you know, just, you know, manufacturing plants, schools, whatever it might be, it becomes more difficult to share that information or it's not happening as freely as probably what needs to be happening. Or at least that's my experience anyway. What type of information do you think would be useful to share between companies in particular that maybe is not done often enough? Yeah. I mean, again, it's difficult because, you know, I haven't really thought about it that much.

[00:29:16] And what if we were in that incident, what would we be able to share with an external company? But, you know, there must be some level of information that we can provide to other organizations in order to make them more secure. So, I appreciate it doesn't help the organization that's been impacted. But, again, we've all got responsibilities that we should be trying to make this a more secure world together rather than, you know, just looking after ourselves. We've got to do more to work together as security professionals. Yeah.

[00:29:46] When we prepped, you told me, obviously, you have different roles. So, you said IT technician, you've been a data security officer, infrastructure security manager, and now head of cybersecurity. So, very diverse, which I always find very interesting when I look at my guests. You said specifically that you have the luxury of being able to see a number of different areas, different angles, and the benefit it brings.

[00:30:12] When we think about behavior, behavior as a whole, right? How much do you think is based on the technology helping you want to detect those behaviors? It actually, on the person themselves, their experience, maybe their guts, to actually notice something a bit fishy, a bit, you know? Maybe the technology might tell you something, or maybe they might not. But at least you feel like, eh, there's something here. It's an interesting question, actually.

[00:30:40] And I think it's a balance. It has to be a balance between tooling and what that is telling us and your gut. I mean, I've been in countless situations now through my career where, you know, the alert or the paper is telling you to do one action. But your gut and your knowledge of being in situations like that in the past tell you to do a completely different thing. And I always do err on my experience as an analyst and as an IT professional, as a security professional.

[00:31:09] And I always err on my knowledge and my gut feel above and beyond an alert. Now, a lot of the time they do match. But in those instances where they don't match, which is very few, I will always rely on my gut of what we should do next rather than what the alert is telling me to do.

[00:31:27] So, yeah.

[00:31:57] And even in its traditional form of, you know, alert cues and dealing with the next alert in the queue, it does work. And there's still a place for that. But we need to be pulling more information into this. We need to be leveraging AI more to be able to look at bigger data sets in a smaller amount of time. And we need to make sure that we're continually improving this process.

[00:32:21] You know, we've got to focus on what is working, what is not working and how do we fix what is not working and make it better for the future. Because this is a continuous improvement loop that needs to happen all of the time. So, yeah.