Mistral releases its first generative AI model for code

Mistral, the French AI startup backed by Microsoft and reportedly valued at $6 billion, has released its first generative AI model for coding, dubbed Codestral.

Codestral — which is designed to help developers write and interact with code, like the many other code-generating models out there — was trained on a data set of over 80 programming languages, including Python, Java, C++ and JavaScript, explains Mistral in a blog post. Codestral can complete coding functions, write tests and “fill in” partial code, as well as answer questions about a codebase in English.

Mistral describes the model as “open,” but that’s up for debate. The startup’s license prohibits the use of Codestral and its outputs for any commercial activities. There’s a carve-out for “development,” but even that’s significantly caveated — the license goes on to explicitly ban “any internal usage by employees in the context of the company’s business activities.”

The reason could be that Codestral was trained partly on copyrighted content. Mistral didn’t confirm or deny in the blog post, but it wouldn’t be surprising exactly — there’s evidence that the startup’s previous training data sets contained copyrighted data.

Codestral might not be worth the trouble, in any case. Weighing in at 22GB, the model requires a beefy PC in order to run. And it’s barely ahead of Meta’s Llama 3 model on popular coding benchmarks.

While impractical for most developers and incremental in terms of performance improvements, Codestral is sure to fuel the debate over the wisdom of relying on code-generating models as programming assistants.

Developers are embracing generative AI tools for at least some coding tasks. In a Stack Overflow poll from June 2023, 44% of developers said that they use AI tools in their development process now while 26% plan to soon. Yet these tools have obvious flaws.

An analysis of more than 150 million lines of code committed to project repos over the past several years by GitClear found that generative AI dev tools are resulting in more mistaken code being pushed to codebases. Elsewhere, security researchers have warned that such tools can amplify existing bugs and security issues in software projects; over half of the answers OpenAI’s ChatGPT gives to programming questions are wrong, according to a study from Purdue.

That won’t stop companies like Mistral and others from attempting to monetize — and gain mindshare with — their models. This morning, Mistral launched a hosted version of Codestral on its Le Chat conversational AI platform as well as its paid API. Mistral says it’s also worked to build Codestral into app frameworks and development environments like LlamaIndex, LangChain, Continue.dev and Tabnine.