A Microsoft VC’s warning about dubious, and rising, corporate artificial intelligence claims – NBC 5 Dallas-Fort Worth

• Analytics firm FactSet dug through recent earnings call transcripts of S&P 500 companies and found that use of the term AI far exceeds the five-year average.
• “It actually isn’t that hard to put AI into a slide deck,” says Michael Stewart, managing partner at Microsoft’s venture arm M12.

The artificial intelligence landscape is teeming with players, and they’re not all legitimate. Some are practicing something called “AI washing,” which Securities and Exchange Commission chair Gary Gensler explained in a video includes “false claims to investors by those purporting to use those new technologies.”

In the financial space, the SEC fined two investment advisers, Delphia (USA) Inc. and Global Predictions Inc., a total of $400,000 in March for what the SEC described as marketing AI-enabled investment predictions to their clients when they were not actually using this technology. (The companies paid the penalties and accepted SEC orders without admitting or denying the Commission’s findings.) 

It’s not just the financial services space that is seeing this phenomenon. Analytics firm FactSet dug through earnings call transcripts of S&P 500 companies for the three months ending in mid-March and found that 179 companies used the term “AI,” which exceeds the five-year average of 73 companies. While it’s not clear which of these companies are being disingenuous about the depth of AI technology in their operations, experts say tacking on the acronym has become  commonplace across industries.

“It actually isn’t that hard to put AI into a slide deck, or to even just use AI because you can use it very easily through any of the platforms without having to build it into the business,” said Michael Stewart, managing partner at Microsoft’s venture arm M12, who focuses on AI, gaming, and deep tech. “Yet, there’s no sustainable competitive advantage to that,” he added.

In reality, what AI washing leads to is a breakdown of confidence between vendors and their consumers, enterprise partners and investors.

“The parties at risk are really the people on the back end not understanding the technology,” said Timothy Bates, professor of practice at University of Michigan-Flint College of Innovation & Technology with an expertise in AI and other emerging technology (and formerly chief technology officer at Lenovo and General Motors).

It’s not just companies claiming AI use when they have none. Bates says so-called button-pushing applications are AI washing, too. AI learns by ingesting information and receiving varying inputs, or prompts. “When you ask the same question over and over again, which is a button push, it doesn’t build the database,” Bates said, adding that third-party applications built on generic natural language processing models are not effective over the long term.

For example, when a law firm buys an AI assistant to replace a human, it may stop working in a matter of months if it’s not based on a unique, unbiased database that’s specifically trained on law and actively learning.

A ‘good’ corporate AI litmus test

Toby Coulthard, chief product officer at Phrasee, a generative AI solution for enterprise clients (including Sephora and Macy’s), says to be wary of any business that uses the term AI broadly.

“AI is such an ambiguous term,” Coulthard said. Instead, he says the businesses should define what type of AI they use and specify how they’re employing it.

Additionally, Coulthard says to take note of when a company first began talking about its AI usage. “Seeing if a business was talking about AI prior to ChatGPT is a good litmus test,” he said. And it’s also a positive sign, he says, if they’re talking about what they won’t do with AI and employing some kind of ethics policy.

“The more you see a business speak more verbose around what they do with AI and what they don’t do with AI, it’s going to be a lot more of an accurate representation,” Coulthard said.

Bates says to look at the model the company uses for an AI offering. “It’s really digging into the company itself and finding out, have they generated their own models [or] are they completely dependent on a third-party model?” And if they are dependent on a third-party model, uncover their service-level agreement or key performance indicators over the next one or two years, he said. “These prompts that are being sold as AI companies, they have to be maintained and monitored and adjusted in order to work.”

Microsoft’s VC approach to AI scrutiny

Stewart and the other AI-focused partners at M12 scrutinize startups by using what they call the four D’s: data, dividends, distribution, and delight.

“If you don’t, as a startup, have access to your customer’s most important data that relates to what the AI will operate on, then any other competitor can have access to the same data,” Stewart said.

For the dividends piece, it can be helpful to identify whether the output of the AI is the work product itself. In other words, is it contributing to the bottom line? When possible, involved parties can look at the gross profitability of any kind of AI business, knowing that one advantage of the technology is extremely high profit margins. Stewart says even 80–90% gross profitability for a fully AI company with limited human intervention is standard.

While the distribution and delight elements of M12’s investment analysis process aren’t directly related to AI washing, these factors play a key role in determining the sustainable longevity of a startup (understanding that young businesses tend to evolve as they mature).

With thousands of AI startups in the market, the winning ones “need strong and stable distribution channels to cut through the noise and ensure they can reach customers in the first place,” M12 says. Meanwhile, “creating consistently delightful user experiences is key to creating the love that keeps them coming back for more.”

AI washing is just another form of jumping on the technological bandwagon. Unlike AI evolutions that came before, natural language processing technology is salable and marketable in unique ways. Because of this, investors and customers have a vested stake in the technology, which makes obfuscation more likely.

So far, the SEC has focused primarily on investment advisors and broker dealers, but AI washing is a marketplace-wide concern. “The last thing we want to do with our fund, and involve Microsoft with, is the use of a technology that goes outside of guardrails,” Stewart said. “If it’s a sham, if the AI is not actually what’s producing the magic, and we’ve missed that, that’s something we need to go back and change our whole diligence process around.”

Already, M12 erased the chalkboard once in 2022 and developed all new guidelines for analyzing AI startups after finding many computer vision companies were more superficial than they claimed to be.

“The hype surrounding AI can attract investors, inflate stock prices, and draw consumer interest, giving these companies a temporary edge,” Bates said. Regulatory bodies are likely to step up scrutiny and enforcement, he added, as too many resources go towards superficial AI claims and not enough towards tangible progress in the field.