The List / Trading, Investment, and Portfolio Management / paper
Task-Adaptive Large Language Models to Generate Human-Persuasive Investment Reports
Abstract
The increasing interest in stock market investment in Indonesia has highlighted a significant challenge for retail investors: the difficulty of analyzing lengthy and complex corporate annual reports. These documents, essential for fundamental analysis, are often hundreds of pages long and contain detailed narrative sections that require considerable time and effort to comprehend. This research addresses this issue by developing an automatic summarization system using a Large Language Model (LLM) to generate concise and insightful summaries of such reports. The primary objective was to develop and evaluate an LLM-based system specifically adapted for the structure and content of annual reports. The method involved creating a tailored dataset comprising 2,008 narrative text excerpts and their corresponding manual summaries sourced from the annual reports of companies listed on the Indonesia Stock Exchange (IDX). The open-source Llama-3.2-3B-Instruct model was then fine-tuned using the Parameter-Efficient Fine-Tuning (PEFT) technique, specifically Low-Rank Adaptation (LoRA). The research results demonstrated a significant improvement in the model's performance after fine-tuning. Quantitative evaluation using ROUGE metrics showed a relative increase of 18.63% in ROUGE-1, 44.45% in ROUGE-2, and 33.83% in ROUGE-L compared to the base model. Qualitative analysis confirmed that the fine-tuned model was capable of generating informative and relevant summaries aligned with the context of annual report analysis. In conclusion, this study demonstrates that fine-tuning LLMs with document-specific data is an effective approach for specialized tasks such as annual report summarization.
In one sentence
Summary
The paper addresses the challenge of generating investment reports from earnings call transcripts that are not only factually accurate but also persuasive enough to guide human investment decisions across different time horizons. It positions this task within the FinNLP 2025 Earnings2Insights shared task, highlighting the limitation of traditional metrics that prioritize content similarity over decision-making effectiveness. The research aims to bridge the gap between automated text generation and real-world financial analysis by incorporating investor psychology and multi-perspective reasoning into the LLM pipeline. The proposed method is a two-stage framework using GPT-4o. Stage one acts as a 'Data Extractor,' employing an 8-category sentiment classification system to analyze Q&A pairs for tone, confidence, and evasion, outputting structured signals. Stage two acts as a 'Cognitive Reasoner,' utilizing a six-dimensional analysis framework (financial performance, business fundamentals, risk, outlook, personal factors, and conviction scoring) to synthesize the extracted data. The system explicitly models investor personalities to tailor recommendations for growth, risk-aware, and other profiles, providing conviction scores and position sizing advice for 1-day, 1-week, and 1-month horizons. Experiments were conducted on 64 earnings call transcripts from the shared task dataset. The system ranked 1st out of 12 teams in human-evaluation average Likert scores and 2nd in automated Likert scores. It achieved a win rate of 0.881 against professional analyst reports. The results demonstrate that integrating sentiment nuance and investor personality modeling significantly enhances the persuasiveness and utility of AI-generated reports. Limitations include reliance on GPT-4o's capabilities and the subjective nature of human evaluation, though the framework offers a robust template for decision-oriented financial NLP.
Research problem
The paper addresses the difficulty of converting complex, lengthy earnings call transcripts into concise, persuasive investment reports that can effectively guide human investors. It highlights that existing LLM approaches often optimize for factual accuracy or similarity to reference summaries rather than the persuasive effectiveness required for real-world decision-making. The core problem is generating reports that account for diverse investor personalities and provide actionable guidance across multiple time frames.
Core contributions
- A two-stage task-adaptive LLM framework combining sentiment extraction with cognitive reasoning for report generation.
- An 8-category investment-centric sentiment classification system that captures management confidence and evasion.
- A six-dimensional analysis framework covering financials, fundamentals, risk, outlook, personal factors, and conviction scoring.
- Integration of investor personality modeling to tailor reports for different investor profiles (e.g., growth vs. risk-aware).
- Explicit conviction scoring with position sizing recommendations tailored to 1-day, 1-week, and 1-month horizons.
Method
- Stage 1: Data Extractor uses prompt engineering to analyze Q&A pairs.
- Sentiment classification into 8 categories (Bullish, Optimistic, Cautious, etc.).
- Extraction of confidence levels (1-5) and question dodge scores (0-3).
- Stage 2: Cognitive Reasoner synthesizes extracted data using a 6D framework.
- Analysis of financial performance, business fundamentals, and risk.
- Integration of investor personality types to tailor recommendations.
- Generation of conviction scores (0-100%) and position sizing advice.
- Output includes LONG/SHORT recommendations for 1-day, 1-week, 1-month.
Results & findings
- Ranked 1st out of 12 teams in human average Likert Score.
- Ranked 2nd in automated average Likert Score.
- Achieved a Win Rate of 0.881 against professional analyst reports.
- Human evaluation accuracy: 0.589 (1-day), 0.542 (1-week), 0.424 (1-month).
- Automated Likert Score average: 4.903.
- Human Likert Score average: 5.96 (Usefulness: 6.13, Clarity: 6.02).
- Framework successfully quantifies uncertainty via conviction scoring.
- Demonstrates superior persuasiveness compared to baseline approaches.
Data & experiments
- Dataset: 64 earnings call transcripts from the FinNLP 2025 Earnings2Insights shared task (40 ECTSum, 24 Professional subset).
- Model: GPT-4o used for both data extraction and cognitive reasoning stages.
- Evaluation: Human evaluation using Likert scores (persuasiveness, logic, usefulness) and accuracy across time frames.
- Automated metrics: Likert scores and Win Rate vs. Analyst Report.
- Comparison: Ranked against 11 other teams in the shared task.
Limitations & caveats
- Relies on GPT-4o, which may have latency and cost implications for production.
- Human evaluation is subjective and may vary between annotators.
- Accuracy drops significantly for longer time horizons (1-month: 0.424).
- Framework is specific to earnings call transcripts, not general financial news.
- Conviction scores are model-generated and may not reflect true market uncertainty.
- Does not incorporate real-time market data or external news sources.
Curator notes
Focuses on persuasion and task adaptation rather than only automatic text metrics.