How to craft a responsible generative AI strategy

Concerns with generative AI fall into four broad categories: hallucinations and inaccuracies, intellectual property rights violations, data privacy and security concerns, and bias and harmful content.

Hallucinations

Large language models (LLMs) tend to hallucinate, or generate factually incorrect content. This is problematic in both employee-facing and customer-facing use cases, especially when users are not aware of the possibility that generative AI tools will produce false outputs.

Intellectual property rights violations

Two main issues arise regarding generative AI and copyright violations. First, the data used to train generative AI models might be governed by open source licenses or protected by copyrights. This could lead to legal challenges if the creators of the content used as training data did not give permission for such use. Various high-profile lawsuits against AI companies, such as the New York Times’ case against OpenAI and Microsoft, are progressing through courts across the world.

The second issue focuses on the content generated by the AI model, rather than the data used to train it — specifically, whether generative AI outputs might infringe on intellectual property rights. For example, if a generative AI application produces text, images or audio that closely resembles an existing copyrighted work present in its training data, this could lead to potential plagiarism accusations.

Data privacy and security concerns

Introducing generative AI into an organization poses significant data privacy and security risks. Often, the data used to train generative AI models is not publicly disclosed. If private user or enterprise data is included in the training corpus, a model’s output could inadvertently reveal this sensitive information. Additionally, shadow AI — when employees use generative AI applications independently without the organization’s explicit knowledge — can result in the accidental release of private data into a generative AI model, making it accessible to others.

Generative AI applications can be deployed on premises, in the cloud or in hybrid environments. Depending on the deployment type, data privacy might be compromised if input prompts and model outputs are not subject to the same stringent data privacy policies normally applied to enterprise data. Furthermore, generative AI introduces new attack surfaces, and malicious actors can exploit models via techniques such as prompt injection.

Bias and harmful content

AI-generated outputs can exhibit a variety of biases that do not align with antidiscrimination laws or an enterprise’s brand values, such as stereotyping or prejudice against marginalized groups. In some cases, generative AI models can produce harmful content. Misleading, offensive or biased outputs can cause direct harm to users and result in increased liability for the organization.

There are also concerns about the energy required to power generative AI, not only during the training phase but also during use at inference time. Generative AI’s energy consumption can be significantly higher than traditional enterprise technologies, raising questions about environmental impact and sustainability.