Four Security Questions to Ask Your Enterprise Generative AI Provider

Generative artificial intelligence (GenAI) is a transformative technology that is quickly becoming the focal point of many enterprise IT strategies. As part of that effort, security teams are working to identify, develop, and implement best practices for securing GenAI use in the enterprise. This requires not only a review of internal IT security practices that account for GenAI, but also a strong understanding of what role GenAI providers play in supporting secure enterprise use. Best practices in this area are constantly evolving, but there are four foundational questions enterprise security teams should be asking to get the conversation started.

Will My Data Remain Private?

GenAI providers should have clearly documented privacy policies. Ideally, customers should be able to retain control of their information and not have it used to train foundational models or shared with other customers without their explicit permission.

Can I Trust the Content Created by GenAI?

Like humans, GenAI will sometimes get things wrong. But while perfection cannot be expected, transparency and accountability should. This can be accomplished in three ways: Use authoritative data sources to foster accuracy, provide visibility into reasoning and sources to maintain transparency, and provide a mechanism for user feedback to support continuous improvement. In this way, providers can help maintain the credibility of the content the tools create. 

Will You Help Us Maintain a Safe, Responsible Usage Environment?

Enterprise security teams have an obligation to ensure safe and responsible GenAI use within their organizations. AI providers should be able to support their efforts in a number of ways.

For example, one area of concern is user overreliance on the technology. GenAI is meant to assist workers in their daily tasks, not to replace the actual workers. As such, users should be encouraged to think critically about the information they are being served by AI. Providers can help promote the right amount of user scrutiny by visibly citing sources and using carefully considered language that reinforces thoughtful usage.

Another risk, perhaps less common, is hostile misuse by insiders. This would include attempts to engage GenAI in harmful actions, such as generating dangerous code. AI providers can help mitigate this type of risk by including safety protocols in their system design and setting clear boundaries on what GenAI can and cannot do. 

Was This GenAI Technology Designed With Security in Mind?

Like other types of enterprise software, GenAI technology should be designed and developed with security in mind, and technology providers should document and share their security development practices. Further, security development life cycles should be adapted to account for new threat vectors introduced by GenAI. This includes actions like updating threat-modeling requirements to address AI and machine learning-specific threats and implementing strict input validation and sanitization of user-provided prompts.

AI-aware red teaming can also be a powerful security enhancement, allowing providers to look for exploitable vulnerabilities, the generation of potentially harmful content and other such issues. Red teaming has the advantage of being highly adaptive and can be used both before and after product release — an essential benefit in maintaining the security of a rapidly evolving technology like GenAI. 

Shared Responsibility

These questions can help enterprise security teams gain a vital understanding of their GenAI providers’ efforts across four foundational areas of protection: data privacy and ownership, transparency and accountability, user guidance and policy, and secure design and development. 

And while these are an excellent starting point, a number of promising industry-level initiatives also are poised to help ensure the safe and responsible development and usage of GenAI that should further expand our understanding of secure AI considerations. However, one thing is clear: Leading providers of GenAI technology understand their role in this shared responsibility and are willing to provide information on their efforts to advance safe, secure, and trustworthy AI. So go ahead and get that conversation started today.

— Read more Partner Perspectives from Microsoft Security