A Fused Large Language Model for Predicting Startup Success

In one sentence

Summary

The paper addresses the challenge of predicting startup success to support venture capital investment decisions, a task complicated by high failure rates and the reliance on subjective investor judgment. It positions the use of online venture capital platform data, specifically Crunchbase, as a scalable alternative to proprietary databases or manual scorecards. The core problem is leveraging both structured fundamental variables (e.g., founder education, funding history) and unstructured textual self-descriptions of business models and innovations to improve prediction accuracy beyond what is possible with structured data alone. The methodology employs a tailored fused large language model architecture. Textual self-descriptions are processed using a pre-trained BERT model to generate 768-dimensional document embeddings, which are then concatenated with normalized fundamental variables. This fused vector is fed into various machine learning classifiers, including logistic regression, elastic net, random forest, and neural networks. The study utilizes a dataset of 20,172 startup profiles from Crunchbase, defining success via events like IPOs, acquisitions, or subsequent funding rounds. Performance is evaluated using balanced accuracy, AUROC, and a calculated Return on Investment (ROI) metric derived from estimated valuations and investment costs, with rigorous out-of-sample testing and sensitivity analyses across sectors and startup ages. Results indicate that the fused model significantly outperforms baselines using only fundamental variables (balanced accuracy 74.33% vs. 72.00%) and traditional text representations like bag-of-words or GloVe. The inclusion of textual self-descriptions adds statistically significant predictive power, translating to a 40.61 percentage point increase in estimated ROI for the investment portfolio. The paper demonstrates that while fundamental variables provide a strong baseline, the semantic information in self-descriptions captures latent factors like business model clarity and tone that enhance decision-making. Limitations include the reliance on self-reported data which may be biased, the approximation of financial metrics due to lack of public valuation data, and the focus on US-based startups, potentially limiting generalizability to other markets or early-stage pre-profile ventures.

Research problem

The paper addresses the difficulty venture capitalists face in identifying successful startups amidst high failure rates. It seeks to determine if textual self-descriptions on VC platforms, when combined with fundamental structured data via a fused large language model, can significantly improve the prediction of startup success compared to using structured data alone or traditional text mining methods.

Core contributions

• Development of a fused large language model architecture that combines BERT-derived text embeddings with structured fundamental variables for startup success prediction.
• Empirical evidence demonstrating the significant predictive value of textual self-descriptions on VC platforms beyond traditional fundamental metrics.
• Quantification of the economic value of improved prediction accuracy, showing a substantial increase in estimated investment ROI.
• Comprehensive benchmarking against multiple baselines including manual feature extraction, bag-of-words, GloVe, and models using only fundamental variables.
• Provision of a decision support tool framework for automating and enhancing the initial screening process in venture capital due diligence.

Method

• Text Processing: Lowercasing and tokenization using WordPiece algorithm for textual self-descriptions.
• Embedding Generation: Use of pre-trained BERT (uncased, 12 layers) to generate 768-dimensional [CLS] token embeddings.
• Data Fusion: Concatenation of BERT embeddings with normalized fundamental variables into a single input vector.
• Classification: Training of multiple classifiers (logistic regression, elastic net, random forest, neural network) on the fused vector.
• Optimization: Hyperparameter tuning for all classifiers; use of convex loss functions and regularization (L1/L2) to prevent overfitting.
• Evaluation: Stratified cross-validation with 5 splits; calculation of AUROC and ROI based on true/false positives and estimated valuations.
• Statistical Testing: McNemar’s test to confirm statistical significance of performance improvements from adding textual data.

Results & findings

• Fused LLM (BERT + Fundamentals) achieved 74.33% balanced accuracy, outperforming fundamentals-only (72.00%).
• Textual self-descriptions contributed a statistically significant improvement in prediction performance (p < 0.01).
• Incorporating text increased estimated investment ROI by 40.61 percentage points compared to using only fundamental variables.
• BERT embeddings significantly outperformed traditional text representations like bag-of-words and GloVe when fused with fundamentals.
• Neural network classifier performed best among the final classifiers in the fused model setup.
• Out-of-time validation confirmed the model's efficacy in predicting success for startups from periods outside the training data.
• Models using only textual self-descriptions performed slightly worse than those using only fundamental variables, highlighting the need for fusion.

Data & experiments

• Dataset: 20,172 online startup profiles from Crunchbase, including structured fundamental variables and textual self-descriptions.
• Success Definition: Binary classification based on occurrence of success events (IPO, acquisition, or external funding rounds).
• Experimental Design: Stratified 80/20 train-test split repeated five times; out-of-time validation for robustness.
• Baselines: Logistic regression, elastic net, random forest, neural network; text baselines include manual features, bag-of-words, and GloVe.
• Metrics: Balanced accuracy, precision, recall, F1-score, AUROC, AUCPR, and estimated Return on Investment (ROI).
• Sensitivity Analysis: Performance evaluation across different business sectors, startup ages, and classifier types.

Limitations & caveats

• Reliance on self-reported textual data from Crunchbase, which may be biased or strategically crafted by founders.
• Approximation of financial metrics (valuations, investment costs) due to lack of public data, potentially affecting ROI accuracy.
• Focus on US-based startups, limiting generalizability to other geographic markets or regulatory environments.
• Definition of success is binary and event-based, potentially missing nuanced performance metrics or long-term viability.
• The model's performance may degrade if the distribution of startup characteristics or VC platform data changes over time.
• BERT embeddings capture semantics but may not fully capture complex strategic nuances or external market conditions.

Curator notes

Discovery status: institutional_trading_round2_promoted. Source links: The Gaining Paths to Investment Success: Information-Driven LLM Graph Reasoning for Venture Capital Prediction