Report: Generative AI bots are susceptible to user manipulation

A report by Immersive Labs reveals that generative AI bots can be manipulated by users of any skill level — not just cyber experts. The main security concern that the report focuses on is generative AI’s susceptibility to prompt injection attacks, or attacks in which users unput specific commands in order to prompt chatbots into disclosing sensitive information. When done successfully, prompt injection attacks may expose organizations to data leaks. 

Through an analysis of prompt injection tests, the report found that 88% of participants were able to prompt a generative AI bot into disclosing sensitive information in at least one level of the test. Furthermore, 17% of participants were able to extract information from the generative AI bot across all test levels. 

Based on these results, the report established the following key takeaways: 

• Human ingenuity can still exceed the abilities of generative AI. Humans are able to devise clever ways to trick generative AI bots, such as prompting them to reveal sensitive information via poems or stories. 
• Non-cybersecurity professionals (including those unfamiliar with prompt injection attacks) are able to leverage creative measures to manipulate bots, suggesting that exploiting generative AI in the wild may be easier than initially expected. 
• Security leaders need to prepare their organizations to respond to prompt injection attacks.