Application of Startup Success Prediction Models and Business Document Extraction Using Large Language Models to Enhance Due Diligence Efficiency

In one sentence

Summary

The paper addresses the inefficiency and high failure rates in venture capital due diligence, which traditionally takes 4-6 weeks and relies on manual, error-prone processes. The authors aim to enhance decision-making accuracy and speed by developing an AI-assisted system that automates data extraction and risk assessment for early-stage startups. The research is grounded in a case study of an Indonesian corporate venture capital firm, highlighting the need for tools that can handle incomplete information and provide comprehensive business analysis. The proposed solution follows the Team Data Science Process (TDSP) and integrates multiple technologies. It uses GPT-4 via the OpenAI API to extract and analyze unstructured data from pitch decks and financial reports, supplemented by the Google Search API for competitor and market trend analysis. For predictive modeling, the system employs an XGBoost classifier trained on a Kaggle dataset of 923 startups to predict success or failure based on funding and demographic features. The system is deployed as a web dashboard using React, Express.js, and Flask, allowing users to upload documents and receive structured analysis and predictions. Experimental results show that the XGBoost model outperformed other tree-based models, achieving 84% accuracy and an F1 score of 88.23% through K-fold cross-validation. User acceptance testing with eight experienced investors yielded a high satisfaction score of 4.50/5.00, indicating strong approval of the system's usability and relevance. However, the authors note limitations, including GPT-4's struggles with complex mathematical reasoning and multi-page financial documents, as well as data security risks associated with third-party LLM APIs. The system is positioned as a supplementary tool rather than a replacement for expert judgment.

Research problem

Venture capital due diligence is a time-consuming (4-6 weeks) and labor-intensive process prone to human error and information asymmetry. Investors struggle to efficiently analyze unstructured data (pitch decks, financials) and assess startup success probabilities amidst high failure rates. There is a need for automated, AI-driven tools to enhance the speed, accuracy, and comprehensiveness of investment analysis.

Core contributions

• Development of an integrated due diligence dashboard combining LLM-based document analysis and ML-based startup success prediction.
• Implementation of a prompt engineering pipeline using GPT-4 to extract structured insights from unstructured pitch decks and financial reports.
• Application of the Team Data Science Process (TDSP) to design and deploy a user-centric VC investment tool.
• Empirical evaluation of the system's effectiveness through customer acceptance testing with experienced venture capital investors.
• Demonstration of XGBoost's superiority over other tree-based models for binary startup success classification in a private markets context.

Method

• Team Data Science Process (TDSP) methodology for iterative system development and customer acceptance.
• GPT-4 API integration for natural language processing of pitch decks and financial reports.
• Google Search API for scraping external market trends and competitor information.
• XGBoost classifier training on Kaggle startup dataset with feature engineering and K-fold cross-validation.
• Flask backend for serving ML predictions and Express.js frontend for user interaction.
• User acceptance testing using Likert-scale questionnaires to evaluate usability, accuracy, and satisfaction.

Results & findings

• XGBoost achieved the highest performance among tested models with 84.00% accuracy, 85.30% precision, 91.58% recall, and 88.23% F1 score.
• The system reduced manual data collection and analysis efforts, aiming to shorten the 4-6 week due diligence cycle.
• Customer acceptance testing resulted in an overall satisfaction score of 4.50 out of 5.00.
• Ease of use scored 4.56/5.00, and system appearance scored 4.63/5.00, indicating high user-friendliness.
• Analysis accuracy scored 4.38/5.00, suggesting room for improvement in LLM extraction reliability.
• Feature relevance scored 4.40/5.00, confirming the utility of the provided analytical modules for investors.

Data & experiments

• Startup Success Prediction dataset from Kaggle (Crunchbase data) with 923 rows and 49 features (funding, industry, location).
• Real-world pitch decks and financial reports (PDF/Excel) uploaded by users for document analysis.
• External market and competitor data gathered via Google Search API integration.
• Comparative model evaluation of Random Forest, XGBoost, LightGBM, AdaBoost, and Decision Tree using K-fold cross-validation.
• User acceptance testing with 8 experienced investment analysts/VC professionals assessing usability, accuracy, and satisfaction.

Limitations & caveats

• GPT-4 struggles with complex mathematical problems and reasoning tasks in financial analysis.
• System currently only extracts data from the first page of financial reports, limiting multi-page document accuracy.
• Data security risks exist due to reliance on third-party LLM APIs and potential for prompt injection attacks.
• Analysis results are supplementary and cannot replace expert judgment for final investment decisions.
• Model trained on Kaggle data may not fully represent the specific local market context of the case study firm.
• Small sample size for user acceptance testing (n=8) limits the generalizability of satisfaction results.

Curator notes

Discovery status: institutional_trading_focus_promoted. Source links: large language models private equity due diligence