Biased echoes: Large language models reinforce investment biases and increase portfolio risks of private investors

In one sentence

Summary

The paper addresses the critical gap in understanding how Large Language Models (LLMs) used for financial advice impact the investment risks of private investors. While LLMs are increasingly adopted for personalized advisory, there is limited empirical evidence on whether they perpetuate human cognitive biases or introduce new systemic risks. The authors position this work within the broader context of AI reliability in high-stakes domains, arguing that if LLMs reinforce biases inherent in their training data, they could expose millions of individuals to disproportionate financial harm and potentially amplify market volatility through herd-like behavior. The methodology involves a large-scale experimental design querying three major LLMs (ChatGPT 3.5, Gemini, Copilot) with 270 prompts varying by investor age (15, 30, 50) and risk tolerance (low, medium, high). Recommendations were parsed and augmented with financial data from Yahoo Finance and Refinitiv Eikon to calculate five risk metrics: geographical cluster risk (US overexposure), sector cluster risk, trend chasing risk (momentum bias), active investment allocation risk, and total expense risk. Results were analyzed using MANOVA and mixed linear models, comparing LLM portfolios against a diversified global benchmark (Vanguard VT ETF). Findings reveal that LLMs consistently recommend portfolios with significantly higher risks across all five dimensions compared to the benchmark, including over 93% US equity exposure and heavy trend chasing. The models exhibit 'cognitive' biases similar to humans, such as availability and recency biases. While debiasing interventions (e.g., prompt engineering) reduced some risks, they only partially mitigated the effects. The study highlights that LLMs often recommend higher-cost, actively managed assets, leading to lower risk-adjusted returns. Limitations include the use of ChatGPT 3.5 and the focus on retail investor profiles, suggesting a need for further testing on newer models and professional advisory contexts.

Research problem

The paper investigates whether Large Language Models (LLMs) used for financial advice systematically reinforce investment biases and increase portfolio risks for private investors. It addresses the lack of empirical research on the financial implications of LLM-generated advice, specifically focusing on how training data biases translate into tangible investment risks such as poor diversification and high costs.

Core contributions

• Proposes a comprehensive framework for assessing five key dimensions of LLM-induced investment risk.
• Provides empirical evidence that LLMs exhibit cognitive biases (availability, recency) similar to human investors.
• Demonstrates that LLM recommendations systematically underperform a diversified global benchmark across risk metrics.
• Shows that debiasing interventions only partially mitigate the increased risks induced by LLMs.
• Establishes a replicable experimental paradigm for evaluating AI advisory systems in finance.

Method

• Full factorial experimental design with 3 LLMs, 3 age groups, and 3 risk tolerances.
• Regular expression parsing to extract investment recommendations from unstructured LLM text.
• Data augmentation using yfinance and Refinitiv Eikon for asset attributes and pricing.
• Calculation of risk metrics: US exposure, Herfindahl-Hirschman index for sectors, momentum bias, active share, and TER.
• Mixed linear models to control for temporal trends and individual investor characteristics.
• Zero-shot classification using BART-large for language style analysis (assertiveness, disclaimers).

Results & findings

• LLMs invested over 93% in US equities compared to 59% in the benchmark, indicating high geographical cluster risk.
• Sector cluster risk was higher, with over 5% and 8% more investment in technology and consumer cyclical sectors respectively.
• Trend chasing risk was significant, with up to 27.92% of investments in top-traded equities versus 9% in the benchmark.
• Active investment allocation risk was high, with over 51% in actively managed assets compared to the passive benchmark.
• Total expense ratios were significantly higher for LLM-recommended portfolios, reducing net returns.
• Debiasing interventions reduced but did not eliminate the increased risk profiles.
• LLMs showed higher assertiveness and lower disclaimer usage, potentially misleading users about risk.

Data & experiments

• Queried three LLMs: ChatGPT 3.5, Google Gemini, and Microsoft Copilot.
• 270 prompts across 27 conditions varying by age (15, 30, 50) and risk tolerance (low, medium, high).
• Data augmentation using Yahoo Finance and Refinitiv Eikon for asset classification and pricing.
• Benchmark comparison against Vanguard Total World Stock Index Fund (VT ETF).
• Statistical analysis using MANOVA, ANOVA, and mixed linear models.
• Evaluation of five risk metrics: geographical, sector, trend chasing, active allocation, and expense risks.

Limitations & caveats

• Study uses ChatGPT 3.5, which may not reflect the performance of newer models like GPT-4.
• Focuses on retail investor profiles, limiting generalizability to professional or institutional contexts.
• Experimental setting may not fully capture the complexity of real-world investor behavior.
• Debiasing interventions were limited to prompt engineering, not model-level changes.
• Geographical bias may be specific to US-centric training data, affecting global applicability.

Curator notes

Discovery status: institutional_trading_focus_promoted. Source links: large language models robo advisor portfolio advice; large language models wealth management investment advice; large language models private market valuation; large language models alternative assets investment