“Good Enough” Is Good Enough.
When Databricks launched DBRX, its flagship AI foundation model, earlier this year it was accompanied by a press release touting how much better it performed on a host of benchmarks compared to competitors. In particular, it beat OpenAI’s GPT 3.5, a model that was released two years earlier, an eternity in AI years.
At the time I asked Databricks CEO Ali Ghodsi why he was comparing his brand new model to something launched in March of 2022 as opposed to the more recent GPT 4 or GPT 4 Turbo. As an answer, Ghodsi pulled up the pricing page on OpenAI’s website. The price per million tokens for GPT 4: $120. A million DBRX tokens on the other hand, about $6.
The idea of DBRX, said Ghodsi, is to let customers “have your cake and eat it too.” It may not be approaching sentience, but it can reliably complete your specific business task, Ghodsi says. What’s more, it only cost Databricks about $10 million to train it instead of a reported $100 million for GPT-4.
“Companies that build chatbots like ChatGPT or Claude, they give you general knowledge, ask it about World War II, or who won the election in 2020,” Ghodsi said, “we are not in that business at all.”
Databricks is just one of many companies embracing smaller, more affordable models.
At the annual Google I/O event on Tuesday, the company unveiled Gemini 1.5 Flash , touted as “faster and even more cost-effective.” This new, leaner model will cost users only 35 cents per million tokens, compared with $7 per million for the Gemini 1.5 Pro model.
Meanwhile Microsoft and Apple recently announced smaller, cheaper models as they push toward a generative AI product that could one day run entirely on your phone. Even OpenAI, which usually differentiates itself by its cutting-edge research, announced this week a new version of GPT-4 that it describes as “much faster and 50% cheaper.”
In the early days of the AI arms race, the competition was over who could build the most powerful models, with the highest number of parameters, the largest context windows, and the most impressive demo reels. But it remains to be seen if state-of-the-art is actually a good business model.
After all, most large companies deploying generative AI aren’t necessarily looking for a human-like AGI to propel us into the next stage of consciousness; they’re trying to automate customer service calls or file expense reports faster.
Now, it seems many AI developers would rather be VHS than LaserDisc; perhaps not at the bleeding edge of theoretical research but providing real utility at an attractive price and making a killing in the process.
“The general kind of consensus is that if you make these models bigger and train on more data, it’ll get better and better, but we don’t know when that ends, right?” said the founder and CEO of one seed-stage startup that’s training new AI models, “but then, you know, as businesses, what you also want to do is what is good enough.”
As AI models get bigger and gobble up more money and computing resources, there are often diminishing returns in terms of performance. Researchers at Morgan Stanley recently estimated that if GPT-5 ends up being 5-10x times larger than GPT-4, it will likely have cost 25-100x the computing power to train.
Cohere, a once and perhaps future OpenAI competitor, recently announced Command-R, described in the press release as a cheaper alternative to GPT-4 Turbo for “key business-critical capabilities.”
It’s the company’s latest offering in what COO Martin Kon calls the “emerging category of scalable models.” He says Cohere is laser-focused on enterprises looking to roll out AI features on a massive scale. In that world, he says, affordability and reliability are more important than having the most advanced technology.
“We don’t focus on the car show prototype; we focus on the production models,” Kon said.
There is, of course, the hundred billion-dollar Gorilla in the room, which is the imminent launch of GPT-5. OpenAI is expected to unveil its new model in the coming months, and, one way or another, it will likely define the AI conversation going forward. Most agree that it will probably be far more advanced than existing models, but it’s an open question whether it will be impressive enough to repeat the zeitgeist-exploding launch of ChatGPT that set the AI revolution in motion. If not, it may further validate the argument that good enough is the way to go.
