Generative AI investment: Trends and ROI considerations

Generative AI investment is a focus of attention for processor manufacturers and cloud providers, as recently seen in Microsoft Corp.’s adoption of AMD Inc.’s MI300X AI chips in the Azure platform.

Yet this interest among major industry players to compete with Nvidia’s H100 GPU processor for generative artificial intelligence business is a subplot in what may be the bigger story. Some businesses may not be realizing the return on investment or results to justify adoption of GPU-based technologies.

“Right now, there’s a shortage of GPUs because everybody is rushing and buying them from the market … generative AI being the primary driver of that,” said David Linthicum, principal analyst for theCUBE Research. “I think the story here is we may be overly focused on processors as the enablers for generative AI and other AI systems. The larger issue is that we may have some trouble in finding some of the use cases out there.”

The AI Insights and Innovation podcast, from theCUBE, SiliconANGLE Media’s livestreaming studio, is the new go-to podcast for the latest news, trends and insights in artificial intelligence, including generative AI. In this debut edition, theCUBE’s Linthicum discusses how a lack of use cases may force enterprises to reevaluate significant investment in high-end processors and generative AI systems.

Chipmakers compete for AI market share in generative AI investment

The recent AMD/Microsoft announcement highlighted the changed competitive landscape as chipmakers maneuver for market share against the success of Nvidia Corp., which has capitalized on the demand for generative AI. Nvidia’s skyrocketing sales and backlog for chips have contributed to a singular focus on how companies can best leverage AI technology.

“All the cloud conferences have become generative AI conferences, and enterprises have reacted to that,” Linthicum said. “That’s why we’re hoarding GPUs and the Nvidia chips and we’re talking about different processors that are out there. All this stuff is going on when we’re not addressing the core issues as to why the generative AI systems need to exist, the ability to find use cases and the ability to solve business problems. That really needs to be the focus.”

That focus may become increasingly more important as enterprises begin to evaluate whether the ROI on generative AI systems is truly as robust as expected.

“There doesn’t seem to be as much ROI on leveraging generative AI systems as people thought, and that’s causing a bit of a dilemma,” Linthicum noted. “CIOs are taking a look at where the current enterprise is … the use cases that are there, the ways in which generative AI can add value and ways in which it’s probably not a good fit. We’re going through this iteration right now as we’re adopting this technology.”

There is some evidence that business leaders are finding use for generative AI within their organizations. A Harris poll released at the end of 2023 found that 54% of respondents reported improved productivity as a benefit from the technology. Yet the question remains whether the high cost of processors and other related AI technologies will ultimately deliver expected results.

“Certainly, generative AI is going to add some value, but it’s not necessarily going to be enough value to cover the development costs and the risk associated with building systems using the specialized technology and, of course, the higher end, higher powered chips,” Linthicum said. “My advice is this needs to be about the use cases. If we can’t find use cases there, then we shouldn’t force fit generative AI or any other technology for that matter into those problems because we’re just making things worse.”

Here is the complete discussion from David Linthicum, part of the AI Insights and Innovation podcast series on theCUBE:

Image: pinglabel /Canva