Janet Yellen warns AI in finance poses ‘significant risks’

CNN
 — 

Treasury Secretary Janet Yellen is expected to warn bankers and tech executives on Thursday that while artificial intelligence could open the door to vast rewards for the financial system, the technology also threatens to introduce new dangers, according to speech excerpts shared first with CNN.

Yellen’s speech at a conference on financial stability represents her most extensive remarks to date on AI.

It comes as investors scramble to get a piece of the AI boom, tech giants embark on an AI arms race and regulators worry about what could go wrong.

“The tremendous opportunities and significant risks associated with the use of AI by financial companies has moved this issue toward the top” of financial regulators’ agendas, Yellen plans to say in the prepared remarks to be delivered at the conference, which is being held on Thursday at the US Treasury Department and on Friday at the Brookings Institution. The event, including Yellen’s remarks, are set to be live-streamed.

On the opportunity side, Yellen will note how AI has already been used by investors to support forecasting and portfolio management and by banks to fight fraud and support customer service, according to the prepared remarks.

Going forward, Yellen is expected to say AI’s “rapid evolution” could pave the way to make financial services cheaper and easier to access, pointing specifically to advances in natural language processing, image recognition and generative AI.

AI chatbots, including OpenAI’s ChatGPT and Google’s Gemini, have captivated users with their abilities. The latest AI tools can almost instantly conjure up images, spit out song lyrics and some can even generate movie-quality videos.

Yellen herself has experimented with AI chatbots, a Treasury official told CNN.

“I know all of you here also recognize that the use of AI by financial institutions comes with risks alongside these opportunities,” Yellen said in the prepared remarks.

A Treasury official told CNN that the conference is expected to include a mix of regulators, tech executives, asset managers, insurers, academics, civil society organizations and small and large banks.

Last December, a team of leading US regulators led by Yellen warned for the first time that AI poses a risk to the financial system. The Financial Stability Oversight Council, a SWAT team of regulators formed after the 2008 financial crisis, formally designated AI as an “emerging vulnerability.”

In her speech, Yellen plans to spell out what could go wrong in financial markets.

For instance, Yellen will note that the “complexity and opacity” of AI models could cause problems.

The problem is that many AI models operate as a “black box,” meaning their inner workings are impenetrable to outsiders.

If Wall Street firms are relying on mysterious AI models, regulators will struggle to understand how safe their systems truly are.

Yellen plans to also cite “inadequate risk management frameworks” around AI risks and “interconnections that emerge as many market participants rely on the same data and models.”

In other words, if many investors are all relying on tools that produce the same outcome, it could cause crowded market positions that exacerbate market moves, both to the upside or downside.

Likewise, Yellen will say there is a “concentration” risk linked to the fact that there are only a few companies providing AI models. That means if one goes down, many Wall Street firms could be impacted.

Another concern: AI models can at times churn out biased results. And that could be a major problem in the real world, especially when it comes to making financial decisions like who gets a loan.

“Insufficient or faulty data could also perpetuate or introduce new biases in financial decision making,” Yellen will say, according to the prepared remarks.

One major challenge facing AI that Yellen does not address in her speech excerpts: so-called hallucinations. AI models have a history of making stuff up, often in a convincing way.

Such hallucinations have previously gotten lawyers in trouble when they cited case law that did not exist.

Yellen is expected to emphasize that regulators plan to continue to monitor AI’s impact on financial stability.

Given how fast AI is evolving, Yellen will say that regulators and the industry can use scenario analysis to better understand “future vulnerabilities” and what can be done to “enhance resilience.”

A Treasury official told CNN that FSOC is working to connect the dots on how AI could pose a threat to the financial system, including by ramping up efforts to monitor how it’s already being used.

Of course, US officials themselves are leaning on AI to do their jobs.

For instance, the IRS announced last September that it has started deploying AI to detect tax cheats.

And in February, the Treasury Department disclosed it has quietly deployed AI to catch bad guys trying to steal from taxpayers.

Treasury plans to do even more with AI in the future.

“We’ve engaged with the public and private sectors on using AI to detect some of the greatest risks we face, from money laundering, to terrorist financing, to sanctions evasion,” Yellen plans to say.