Machine Learning in Cybersecurity | Cyber Risk
By 2026, more than 80% of enterprises will have used generative artificial intelligence (“GenAI”) APIs, models and/or deployed GenAI-enabled application in production environments. With this fast pace of adoption, it is no wonder that artificial intelligence (AI) application security tools are already in use by 34% of organizations, a number that will no doubt increase.
Applications are required to go through rigorous security testing to ensure that they do not introduce security risks compromising the application or its users. How do you ensure that an AI system has been thoroughly tested when the risk landscape is evolving as rapidly as the technology? Machine learning is critical in many new AI systems. The security evaluation of these systems is different from other types of system security reviews. A thorough security review starts with understanding of components involved in the system.
Following is an introduction to some machine learning concepts relevant for cybersecurity professionals. This material will be useful for security practitioners, including DevOps engineers, looking for an introduction to AI security. It also serves as a relevant background for our other AI focused articles – see “AI Security Risks and Recommendations: Demystifying the AI Box of Magic”.
What Is Machine Learning?
Machine learning (ML) is a subset of AI that empowers computers to execute tasks without explicit programming. It is built upon algorithms and statistical models designed to recognize patterns and relationships in data, iteratively improving performance. This iterative learning process involves training models with labeled data, enabling them to adjust internal parameters to minimize the difference between predicted and actual outputs. Through this approach, models learn to generalize from training data, facilitating accurate predictions on new or unseen data. ML encompasses various techniques, including supervised learning, unsupervised learning and reinforcement learning, each tailored to different data scenarios. Supervised learning trains models on labeled datasets, while unsupervised learning infers patterns from unlabeled data. Reinforcement learning enables agents to make sequential decisions by interacting with the environment and receiving feedback. An effective ML model can automate tasks, make predictions and uncover insights across diverse domains.
Large language models (LLM) are a specific type of machine learning that has led AI industry due to its capacity to understand and generate human-like text. LLMs, such as OpenAI’s GPT series, are trained on vast amount of text data and use deep learning architectures, typically based on transformer models. These models learn to predict the next word or token in a sequence, based on the context provided by preceding words. What distinguishes LLMs is their ability to capture complex linguistic structures, semantics and context, enabling them to generate contextually relevant text responses. LLMs have demonstrated proficiency in various language-related tasks, including language translation, text summarization, question answering and text generation.
Whether you use custom trained ML models running on dedicated inference infrastructure or integrate with third-party LLM provider, ML can open up many cybersecurity risks, and create weaknesses in the overall security architecture.
What Is A Machine Learning Model?
A ML model is a file that has been created by using patterns of previously analyzed data. The model uses its previous learnings to predict things for any new data that is passed in the model. The model file contains weights structured as vectors, which can be interpreted as an extremely complicated graph that plots new data. More practically, a model is a raw file on a filesystem that can be used for inference.
Some model files can be packaged as model archives. Model archives contain the model file as well as code which can be used to transform or tokenize data before interacting with the model. The code is used to transform input data into vectors that are compatible with the model, and output from the model back into the desired format.
A model can be created by using one of the many different ML libraries. Some of the most popular libraries are:
Understanding the components and risks associated with a ML framework and model format, is part of the due diligence that should be done to ensure vulnerabilities or unintended consequences, which are not being incurred due to security flaws in ML deployment.
