The List / Reports, Filings, Accounting, and Risk / paper
Bloated Disclosures: Can ChatGPT Help Investors Process Information?
Abstract
Generative AI tools such as ChatGPT can fundamentally change the way investors process information. We probe the economic usefulness of these tools in summarizing complex corporate disclosures using the stock market as a laboratory. The unconstrained summaries are remarkably shorter compared to the originals, whereas their information content is amplified. When a document has a positive (negative) sentiment, its summary becomes more positive (negative). Importantly, the summaries are more effective at explaining stock market reactions to the disclosed information. Motivated by these findings, we propose a measure of information ``bloat."We show that bloated disclosure is associated with adverse capital market consequences, such as lower price efficiency and higher information asymmetry. Finally, we show that the model is effective at constructing targeted summaries that identify firms'(non-)financial performance. Collectively, our results indicate that generative AI adds considerable value for investors with information processing constraints.
In one sentence
Summary
The paper addresses the challenge of information overload in complex corporate disclosures, such as Management Discussion and Analysis (MD&A) sections and earnings conference call transcripts. It investigates whether generative AI tools, specifically GPT-3.5 Turbo, can effectively distill relevant information for investors with limited attention. The authors position this within the context of disclosure quality, obfuscation, and the economic value of AI in financial analysis, aiming to determine if AI summaries retain or enhance the informational content of the original documents compared to human processing constraints. The methodology involves generating unconstrained and targeted (financial or ESG) summaries for a large sample of MD&As and conference calls from 2009 to 2020. The authors compare the sentiment and informativeness of these summaries against the original texts by regressing short-window abnormal stock returns on sentiment scores derived from both. They also introduce a 'Bloat' measure, defined as the relative reduction in document length after summarization, and analyze its determinants and capital market consequences using proxies for price efficiency and information asymmetry. The study includes robustness checks with out-of-sample data and variance decompositions to isolate firm-specific effects. Key findings indicate that GPT-3.5 summaries are significantly shorter (25-30% of original length) yet exhibit amplified sentiment and superior explanatory power for stock market reactions. The 'Bloat' metric reveals that firms with higher bloat experience adverse market outcomes, including lower price efficiency and higher information asymmetry. Targeted summaries for ESG topics show increasing market relevance over time. Limitations include the use of GPT-3.5, which may differ from newer models, and the reliance on sentiment-based metrics which might not capture all nuances of complex financial data. The study suggests AI can mitigate information processing costs but highlights risks associated with opaque or bloated disclosures.
Research problem
The paper investigates whether generative AI, specifically GPT-3.5, can effectively summarize complex corporate disclosures (MD&A and earnings calls) to aid investors with limited attention. It addresses the problem of information overload and potential obfuscation in financial reporting, questioning if AI summaries retain or enhance the informational content of original texts. The study also explores the economic consequences of 'information bloat' and the utility of targeted AI summaries for specific themes like ESG.
Core contributions
- Demonstrates that GPT-3.5 summaries of corporate disclosures are significantly shorter yet retain and amplify sentiment compared to original texts.
- Shows that AI-generated summary sentiment is more informative in explaining short-window stock market reactions than original document sentiment.
- Introduces a novel 'Bloat' metric based on the relative length reduction from AI summarization, capturing redundant or irrelevant information.
- Establishes that higher disclosure bloat is associated with adverse capital market consequences, including lower price efficiency and higher information asymmetry.
- Validates the use of targeted AI prompts to extract specific information (e.g., ESG performance) and shows its increasing market relevance over time.
Method
- Used GPT-3.5 Turbo to generate unconstrained summaries of MD&A and conference call transcripts.
- Applied targeted prompts to generate summaries focused on financial performance and ESG activities.
- Calculated sentiment scores using Loughran and McDonald dictionaries for both raw and summarized texts.
- Regressed short-window abnormal stock returns on sentiment scores to test informativeness.
- Defined 'Bloat' as (n - n*)/n, where n is original length and n* is optimal summary length approximated by GPT output.
- Performed variance decomposition to analyze sources of variation in the Bloat measure.
- Regressed capital market friction proxies (PIN, bid-ask spreads, volatility) on the Bloat measure.
- Conducted out-of-sample tests using post-2021 data to validate robustness.
Results & findings
- GPT-3.5 summaries are approximately 25% (MD&A) and 30% (conference calls) of the original document length.
- Summary sentiment is more pronounced (more positive/negative) than original sentiment, suggesting AI filters out hedging language.
- Summary sentiment has significantly higher explanatory power for stock market reactions than original sentiment.
- The 'Bloat' measure is primarily firm-specific, with 70% of variation attributed to firm-level factors.
- Higher bloat is associated with lower price efficiency and higher information asymmetry (PIN, bid-ask spreads).
- Bloat is higher for firms with losses, negative sentiment, and negative stock market reactions, supporting obfuscation hypotheses.
- ESG-specific summaries show an increasing time trend in market informativeness, reflecting growing investor focus on ESG risks.
Data & experiments
- Sample of MD&A sections and earnings conference call transcripts from 2009 to 2020, representing a random 20% of the population.
- Generation of unconstrained and targeted (financial/ESG) summaries using GPT-3.5 Turbo with a temperature of 0.5.
- Regression analysis of two-day cumulative abnormal returns on sentiment scores from both raw and summarized documents.
- Variance decomposition of the Bloat measure to assess time, industry, and firm-specific variations.
- Analysis of capital market consequences using proxies for informed trading (PIN), abnormal bid-ask spreads, and post-filing volatility.
- Out-of-sample robustness checks using data from post-2021 periods to ensure results are not due to overfitting.
Limitations & caveats
- The study uses GPT-3.5, which may differ in performance from newer or more specialized financial LLMs.
- Sentiment analysis relies on dictionary-based methods, which may not capture all nuances of AI-generated text.
- The 'Bloat' measure is based on length reduction, which might not perfectly align with semantic redundancy.
- Results are based on historical data (2009-2020), and market reactions to AI summaries may evolve.
- The study assumes the stock market efficiently processes information, which may not hold in all contexts.
- Potential biases in AI summaries due to training data limitations or prompt sensitivity are not fully explored.
Curator notes
Discovery status: round3_promoted. Source links: Can Large Language Models beat wall street? Evaluating GPT-4’s impact on financial decision-making with MarketSenseAI; FinBERT : A Large Language Model for Extracting Information from Financial Text†