The Double-Edged Sword of Generative AI

Forget phishing attacks as we know it. In a world where technology looks and sounds more human than ever before – spling errors and poor grammar, are no longer the giveaway of a fraudulent email or cyberattack.

Now add the risk of being deceived by a deepfake, and receiving voice notes or texts that emulate speech, then these AI-powered tools enable the creation of hyper-realistic phishing content and ransomware that can disrupt entire sectors. Cybercriminals can arm themselves with a new and formidable weapon: generative AI.

This powerful technology, which has changed the British business and technology landscape in a way I’ve not seen for decades, holds immense promise for organizations to improve productivity and cyber defenses. But as these advanced technologies become more embedded in business operations, they also expand the attack surface and provide cybercriminals with sophisticated new tools to exploit.

Cybercriminals Gaining the Upper Hand

According to a report from the World Economic Forum produced in collaboration with Accenture, more than half of business executives believe that attackers will have the upper hand over defenders over the next two years.

With the ability to purchase malicious large language models on the dark web, attackers are able to craft more deceptive and damaging cyberattacks than before.

The threat is very real. There has been a staggering 76% increase in ransomware attacks since the end of 2022 targeting critical sectors such as manufacturing, education, and healthcare.

Furthermore, the advent of deepfakes which are capable of emulating company executives to authorize fraudulent transactions now represents a significant financial threat. How do we train employees as a first line of defense in this new world? And how can cyber security professionals get ahead?

Generative AI Security Vulnerabilities and How to Prepare

Firstly, generative AI introduces specific vulnerabilities that can complicate the cybersecurity landscape. As organizations scale generative AI solutions, they face more risks of model disruption, prompt injection, training data exposure, theft, and manipulation.

These vulnerabilities therefore call for the development of new security capabilities such as shadow AI discovery, LLM prompt and response filtering and specialized AI workload integration tests — areas where many organizations are currently underprepared.

The good news is that AI isn’t new. Many cyber security professionals have already been using AI, and machine learning, to find patterns in data and identify unusual activities for humans to quickly intervene and investigate. AI-powered red teaming and penetration testing are also becoming increasingly common, allowing organizations to test their defenses more frequently and become more resilient.

By fully leveraging generative AI, cyber security teams can turn the tables on attackers. For example, generative AI intelligence can be integrated into security operations to improve and speed up incident detection and response. Security analysts can also use generative AI algorithms that automatically scan code and provide rich insights to better understand malicious scripts and other threats.

Of course, this will also create new checks and balances for code that teams use to build their defenses and protect organizations. Teams will now need to integrate generative AI security into their governance frameworks. This involves establishing clear policies and processes that are informed by the latest cyber intelligence and are aligned with regulations, like the upcoming EU AI Act.

Nonetheless, organizations will need to be mindful that the proliferation of security tools can be increasingly difficult to navigate. At a time when organizations can have as many as 50 tools deployed at the same time, they are crying out for consolidation. As the threat landscape escalates again, it means organizations must swiftly adjust their security strategies again, deepen their risk assessments, and deploy technology they trust.

Conclusion

A new era in cybersecurity is being shaped by the acceleration of generative AI, and we find ourselves in a fine balancing act. The same technology that holds the promise of revolutionizing our defenses can also be exploited by cybercriminals. The surge in ransomware attacks and advent of AI-powered deepfakes and phishing are a clear signal of the escalating threat.

Cybersecurity professionals will need to stay ahead by developing new skills, tools, and strategies to protect their organizations from cyber attackers armed with generative AI. As we continue to navigate this evolving landscape, the role of cybersecurity professionals is more crucial than ever in safeguarding our future.