“Install the Google Malware” – The AI Post

Reflections on trust, tools and the reality of automation

This blogpost was written by Jon Cumiskey, Head of Information Security. The views expressed are personal and do not necessarily align with the views of SEP2.

As someone who has been inherently sceptical of AI in the past (“it’s just a bunch of IF statements, it’s just a marketing buzzword”) seeing the rise of chatbots, fake-imagery and hallucinations has been an interesting one for me. What I am writing here is an opinion from a particular vantage point: at SEP2 we want to help as many organisations be as secure as possible, which is a vantage point that I hope is common enough to be relatable.

It is interesting to see the shifting sands in this part of the technological world and get a good view of what benefits and challenges are emerging, especially when viewed through our security lens. This blogpost is not meant to be over-critical to any specific vendors and is merely a way of addressing one point of view in a snapshot of time.

I was in a meeting with our CEO Paul Starr and a customer a few weeks back where Paul decided to try Microsoft Copilot out for summarising the meeting notes for us. We were discussing malware protection and compensating controls in Google Cloud with the customer at the time. Copilot succinctly summarised the point as “We will install the Google Malware”, missing out some key parts of the sentence. This type of stuff is fun to laugh at, and then manually correct, until it makes a big enough error, or until it gets smart enough to grow tired of our nit-picking and berating and puts us out to pasture (aka. Minimum Basic Income). Aside from that, it did a good job in summarising a 2 hour discussion into a couple of sentences.

One of the older idioms that really stuck with me from the Machine Learning days is that a key challenge was “understandability” (i.e. why the model made the decision that it made). The amount of utility that a tool can have relies on scrutiny, being able to understand its reasoning and knowing the level of fallibility and/or trustworthiness it has. This almost sounds like working with people. While that sounds incredibly obvious, the horror stories of people mis-judging that trust level will no doubt continue.

Gemini

At SEP2, we’ve been getting a view of the publicly available AI-powered functionality from Google’s Gemini model (formerly known as Duet) through the Chronicle SIEM/SOAR platform on the SecOps Enterprise packages. While this platform has cut down our engineering time significantly and given us a scalable platform to thrive on, I want to know how we can deliver more to our customers. Google are releasing more features and capabilities soon for this too and anyone who was at Google Next knows that they are a little bit excited about this particular topic.

At current, Gemini exists in two places in Chronicle – a SIEM Search Helper and a SOAR AI Investigation Widget.

SIEM Search Helper

This has a very simple modus operandi, which is to help you write structured UDM (Unified Data Model) queries in Chronicle without having to know the specific syntax of UDM.

So, let’s see a few examples.

Below, I’ve asked Chronicle how many emails I sent last Friday:

So, looking at this as someone who knows Chronicle quite well, my reaction is “Hmmm, not quite.” It hasn’t necessarily managed to resolve my name into my username/email format correctly and it hasn’t necessarily selected the correct UDM field for my username. Also, what’s going on with the date there? I wrote this on the 17^th April, so last Friday was the 12^th April, not the 24^th February, as it has given me.

Let’s try another one: how many users browsed sep2.co.uk this month?

Yep, that looks good, I’m happy there. The date range selected isn’t quite what I asked for, but I get its logic.

And finally, a real hail Mary: how many machines executed the EICAR test string?

A swing and a miss, as they say!

So where does this leave us in the SOC workflow? Next up, SOAR Investigation.

SOAR Investigation Widget

The SOAR AI Investigation Widget seems to be a great start. Cases are being neatly summarised and there are some quite succinct, if maybe unadventurous recommendations.

Who’s going to get the most out of this? The less experienced/technical and those who need the most guidance are going to benefit the most right now for sure. You don’t know what stones you need to turn over? Great – it’ll help massively there.

I can’t see it replacing yet the golden “SOC spidey-sense.” While writing this from inside our SOC I had overheard an Analyst talking with great passion to a colleague about how they don’t like the look of a certain event in a customer’s environment. Lo and behold they’ve got themselves a true positive.

A slightly dry and uncreative end result is definitely better than one that is hallucination prone or overloading the end user with data. So, it feels like Google have made the right call for now in keeping it un-controversial in its findings.

Outside of the Google Tools

An AI tool needs to be readily available to have utility – it needs to either be able to do a job no other tool can do, or be at least as easy as using another tool to do the job.

One use case I have is that I need to convert human readable datetime to Unix/Epoch timestamps quite regularly. I would love to have a Neuralink implant to allow me to do this via mental arithmetic, but for now I am using unixtimestamp.com.

Thinking about this 30-second-a-go time-sink, the Island Enterprise Browser we use internally has an AI chat bot side bar. I’ve not used that yet, so let us see if that can help me?

Oh, OK. So, you’re not solving that problem for me then.

Summary

What are we left with? Time savers, not game changers, is what it feels like right now. The truth of the matter is that this technology will likely realise its full utility and potential at the point that it has become “boring” and “taken for granted,” at which point the hype bandwagon will have left for the next Big Thing. The key challenge is being able to trust it. Trust takes time to build, which can only be done through consistent accuracy. Finer minds than me know this and are no doubt solving this problem, model by model. Currently, it is “Our Eyes” that are performing the scrutiny, not the “A Eye.”

Closing Comments

What we are in here is a bit of a gold-rush for the next stage of computational dominance and using technology to make people’s lives easier. Interestingly, many of the different tools from different vendors are displaying many of the same challenges, which is encouraging, as competition always feels like a good thing. It has also introduced a new and enjoyable “drunk toddler” aspect to computing, which is always fun.

In any case, I am sure that for the benefit of anyone who reads this post in 6 months time (or even perhaps 6 weeks time), this post will have aged like fine milk.

Maybe, as one current upside, we’re now past the high watermark of security vendors abusing the term AI to avoid scrutiny on their (potentially underbaked) black-box detection methodology? If we’ve got that out of all of this, then the whole thing feels like a massive success.

As one last point, I decided to try and generate an amusing image for this article using Free Midjourney, which feels like it summarises my points nicely.