The market for small AI

On Tuesday, Microsoft introduced its smallest AI model to date, called the Phi-3-mini. While large language models have grabbed a lot of the headlines, a number of companies are working on creating smaller language models. That generally means they’re trained with less data, so they may know less. But they also require less processing power, so they’re often cheaper and can work on smaller devices.

You can think of generative artificial intelligence systems as having different-sized brains. University of Virginia economics professor Anton Korinek says the large language models are experts — they know a lot about a lot of different things. 

“Then there are also much smaller models. Their brain is more like the size of a dog brain or something like that,” said Korinek.

But sometimes you don’t need or want your AI system to be all-knowing. Take an automaker who wants an AI chatbot for dealerships to ask questions about different car models, said Ohio State engineering Dean Ayanna Howard.

“You want it to be fast, you want it to be quick,” said Howard.

And you want it to know a lot about the benefits of cloth versus leather seats, but you don’t need to take up its computer brain space by training it on things like health care. 

“So we’re starting to see a lot of companies adopt these much more specialized models,” she said.

Howard said smaller AI systems also let companies store their data locally. “It doesn’t become part of the ecosystem so that your competitors have access to your data.”

Smaller models also require less computing power. Gil Luria of D.A. Davidson said that’s a giant money saver.

“The cost for using these models will be considerably smaller, which will make it possible for us to have a lot more applications,” said Luria.

For example, Luria said we could see more phone apps that summarize all your emails or easily make a restaurant reservation based on where your calendar says you’re going later.

Essentially, a smaller AI model whose data set is you.