Who will prevail in ChatGPT vs the Sellside?
Unlock the Editor’s Digest for free
Roula Khalaf, Editor of the FT, selects her favourite stories in this weekly newsletter.
Analyst angst is real. You spend years sweating over some of the world’s toughest exams, with the odds stacked against passing, only to discover a computer can do the job better.
A draft paper found that generative AI, which mimics (or in this case apparently exceeds) human ability, outperformed financial analysts when it came to predicting the direction of future earnings — and by a comfortable 8 percentage point margin too, said Alex Kim, Maximilian Muhn and Valeri Nikolaev of the University of Chicago.
The real surprise was that the large language model trounced its biped peers without recourse to all the context that informs earnings — sectoral issues and economic backdrop, say, or changes of personnel.
Instead, it was simply fed stripped down and standardised balance sheets and income statements. Company names and years were anonymised: after all, even a basic model could crawl through news cuttings and “predict” that Nvidia will increase earnings this year.
Analysts, in contrast, have plenty to peruse in order to inform their decisions: earnings calls, management discussion and outlook as well as knowledge of the sector and the environment in which it is operating.
ChatGPT’s secret sauce is chain of thought reasoning. The model learns to do the maths — operating efficiency, leverage ratio — and identify trends. These become the building blocks to work out whether future earnings will rise or fall next year.
Certainly some of this is basic stuff. Even 19th century calculators were capable of dividing numbers. Spotting trends is as easy as CMD-F: try plugging free cash flow into any number of tech newbies and check out all the negative numbers.
Yet ChatGPT’s record on the basis of slim pickings bodes well for a model furnished with more contextual data. Banks, hedge funds and wealth managers are already harnessing AI. Some fund managers are already incorporating genAI in investment decision making. Mostly, however, these complementary tools take over the grunt work: transcribing earnings calls, summarising data, analysing text, eliminating human error and saving copious amounts of time.
So analysts need not start packing their boxes yet. The model’s outperformance is not just a story of superior intelligence. The benchmark is the median forecasts: plenty of analysts do better.
In any case, as with most jobs, there will be room for humans alongside the machines. That at any rate is the view of the diplomatic ChatGPT, which concludes “A hybrid approach that leverages the strengths of both AI and human analysts often provides the most accurate and comprehensive forecasts.”
