ChatGPT

GPT-4 Outperforms Human Analysts in Financial Predictions, Study Finds

May 25, 2024 at 5:28:30 AM

TL;DR New research from the University of Chicago shows GPT-4 can analyze financial statements with accuracy surpassing professional analysts. Using anonymized data and "chain-of-thought" prompts, GPT-4 achieved a 60% accuracy in predicting future earnings, higher than the 53-57% range of human analysts. This suggests LLMs like GPT-4 could play a central role in financial decision-making, despite challenges in numerical analysis.

GPT-4 Outperforms Human Analysts in Financial Predictions, Study Finds

Researchers from the University of Chicago have shown that large language models (LLMs) like GPT-4 can perform financial statement analysis with accuracy that rivals or surpasses professional analysts. Their findings, published in a working paper titled “Financial Statement Analysis with Large Language Models,” suggest significant implications for the future of financial analysis and decision-making.

Key points include:

Performance of GPT-4: The model was tested on predicting future earnings growth using standardized, anonymized balance sheets and income statements. GPT-4 outperformed human analysts, achieving an accuracy score of 0.604 and an F1 score of 0.609.
Chain-of-thought prompts: Researchers used "chain-of-thought" prompts to guide GPT-4's reasoning, emulating the analytical process of financial analysts. This approach helped GPT-4 achieve a 60% accuracy in predicting future earnings, higher than the 53-57% range of human analysts.
Narrative insights: The LLM's predictions did not rely on its training memory but on generating useful narrative insights about a company’s future performance.
Numerical analysis challenge: Despite the traditional challenge of numerical analysis for language models, GPT-4 demonstrated significant capabilities in this domain.
Benchmark concerns: Some experts noted that the ANN model used as a benchmark might not represent the state-of-the-art in quantitative finance.
Disruptive potential: The ability of a general-purpose language model to match specialized ML models and exceed human experts highlights the disruptive potential of LLMs in financial analysis.
Interactive web application: An interactive web application showcasing GPT-4’s capabilities was created, though users are advised to independently verify its accuracy.

The study concludes that LLMs like GPT-4 may play a central role in financial decision-making due to their vast knowledge base and pattern recognition abilities. While human expertise remains crucial, tools like GPT-4 could significantly augment and streamline the work of financial analysts, potentially transforming the field in the future.