Leveraging LLMs to Streamline the Review of Public Funding Applications

Basic Information

• Paper ID: 2510.09674
• Title: Leveraging LLMs to Streamline the Review of Public Funding Applications
• Authors: João D.S. Marques, André V. Duarte, André Carvalho, Gil Rocha, Bruno Martins, Arlindo L. Oliveira
• Classification: cs.CY cs.AI
• Publication Date: October 8, 2025 (arXiv preprint)
• Paper Link: https://arxiv.org/abs/2510.09674

Abstract

Annually, the European Union and its member states invest millions of euros in funding various development initiatives. However, the volume of applications received by these programs continues to increase, and due to limited human resources, this often creates significant bottlenecks in the evaluation process. This study details the practical deployment of AI-assisted assessment in two government initiative pipelines: (i) business applications for international business expansion, and (ii) citizen reimbursement applications for energy-efficient home improvement investments. Although these two scenarios involve different assessment procedures, the research demonstrates that AI effectively improves processing efficiency and reduces workload for both application types. Specifically, in the citizen reimbursement initiative, the solution increased reviewer productivity by 20.1% while maintaining a negligible false positive rate based on test set observations. These improvements reduced total assessment time by over two months, demonstrating the impact of AI-driven automation in large-scale evaluation workflows.

Research Background and Motivation

Problem Definition

The core problem addressed by this research is the efficiency bottleneck in evaluating European Union public funding projects. With the surge in application volumes, traditional manual evaluation methods can no longer meet processing demands, resulting in prolonged evaluation cycles, decreased applicant satisfaction, and ultimately undermining public confidence in the efficiency of these initiatives.

Problem Significance

Public funding projects are crucial tools for driving economic growth, sustainable development, and innovation. Low evaluation efficiency not only affects the timeliness of fund allocation but may also cause high-quality projects to miss opportunities, impacting the achievement of overall policy objectives.

Limitations of Existing Methods

Traditional document review relies on rule-based natural language processing and optical character recognition technologies, which perform well in controlled environments but are highly sensitive to changes in document structure and content, making them difficult to maintain and scale to broader applications.

Research Motivation

The emergence of Large Language Models (LLMs) provides unprecedented flexibility and adaptability for automating document processing. This research aims to explore how to leverage LLMs to improve the efficiency and consistency of public funding application evaluation while ensuring human oversight.

Core Contributions

1. Real-World Deployment Experience Report: First report of successful deployment of two AI-assisted document evaluation systems, demonstrating how automation can accelerate application analysis while ensuring decision integrity through human oversight.
2. Practical Effectiveness Verification: Achieved 20.1% reviewer productivity improvement in the ReClaim initiative, reducing total assessment time by over two months.
3. Best Practices Summary: Based on real-world deployment experience, provides best practices and key lessons learned for integrating AI models into similar environments.
4. Dual-Scenario Validation: Validates the generalizability of AI-assisted evaluation through two different types of government initiatives (business internationalization applications and citizen energy-efficiency renovation reimbursements).

Methodology Details

Task Definition

The research involves two distinct tasks:

1. IExp Task: Comprehensive evaluation of business internationalization applications, including document summarization generation, internal consistency detection, and preliminary scoring
2. ReClaim Task: Document verification for citizen energy-efficiency renovation reimbursement applications, primarily checking consistency between application information and supporting documents

System Architecture

IExp System Architecture

• Input: Business application documents averaging 30,000 tokens (over 50 pages)
• Core Model: GPT-4o
• Processing Pipeline:
  1. Document segmentation and filtering to avoid LLM context overload
  2. Identification of key fields for each task based on evaluation team expertise
  3. Automation of six most time-consuming evaluation tasks
• Output: Application summaries, consistency reports, preliminary scores, and justifications

ReClaim System Architecture

• Input: Approximately 80,000 applications, each with an average of 11 supporting documents
• Hybrid Processing Pipeline:
  1. Document Standardization: Supporting only widely-used file formats (PDF, ZIP, PNG, etc.)
  2. XML Conversion: Converting user form fields into structured XML format
  3. VLM Information Extraction: Using GPT-4o to parse unstructured supporting documents
  4. Automated Consistency Checking: Comparing extracted information with applicant-reported values
• Output: Pre-populated verification checklists flagging items requiring manual review

Technical Innovations

1. Human-Machine Collaboration Design: System outputs serve only as recommendations, ensuring human reviewers always maintain oversight and accountability.
2. Task-Specific Optimization: Customized solutions for different types of evaluation tasks.
3. Cost-Benefit Balance: Cost control achieved through targeted inputs and task prioritization.
4. GDPR Compliance: Data processing entirely within EU boundaries, stored on encrypted local disks.

Experimental Setup

Datasets

1. IExp Dataset:
   • Proof of concept: 50 applications from previous calls
   • Current evaluation: 11 applications using AI tool support
   • Activity classification: 764 historical applications
2. ReClaim Dataset:
   • Total applications: Approximately 80,000
   • Test set: 200 samples uniformly distributed across types
   • Total documents: Approximately 880,000 documents

Evaluation Metrics

1. IExp Metrics:
   • Summary alignment: Cosine similarity, ROUGE-L, BLEU, METEOR
   • Activity classification consistency: Consistency level between reviewers and LLM
2. ReClaim Metrics:
   • Productivity improvement: Percentage reduction in processing time
   • Automated verification rate: Proportion of fields requiring no manual verification
   • Accuracy: Proportions of correct, minor errors, false positives, false negatives, and reading errors

Comparison Methods

• Model Selection: Blind comparison of GPT-4o vs Gemini-1.5 Pro
• Processing Approach: Comparison of AI-assisted vs purely manual processing

Experimental Results

Main Results

IExp System Results

1. Significant Summary Alignment Improvement:
   • Cosine similarity improved from 0.77 to 0.99
   • ROUGE-L, BLEU, and METEOR metrics all improved from below 0.35 to above 0.9
2. Activity Classification Consistency:
   • LLM-reviewer consistency approximately 70%
   • LLM-applicant consistency higher

ReClaim System Results

1. Productivity Improvement: Reviewer productivity increased approximately 20%
2. Automated Verification Performance:
   • Overall automated verification rate: 76%
   • Verification rates by section: Eligibility check 84%, public core 76%, type review 67%
3. Accuracy Analysis:
   • Correct: 88%
   • Minor errors: 5%
   • False positives: 0%
   • False negatives: 3%
   • Reading errors: 4%

System Impact Analysis

Positive impacts following AI system deployment:

• Clarification requests/applications: Decreased from 2.13 to 2.05
• Applicant appeal rate: Decreased from 25.8% to 20.4%

User Feedback

1. IExp Task: Evaluators estimated AI assistance could accelerate the review process by up to 30%
2. ReClaim Task: Feedback was polarized
   • Reviewers involved in development expressed strong appreciation
   • Experienced reviewers estimated time savings up to 40%
   • Some reviewers lost confidence after encountering errors

Related Work

Traditional Document Processing Methods

Traditional automated document review relies on rule-based NLP and OCR techniques, performing well in controlled environments but sensitive to document structure variations and difficult to maintain.

LLM-Driven Document Processing

• Legal Domain: LLM tools capable of rapidly reviewing and extracting various legal texts
• Human Resources: Evolution from basic keyword analysis to complex candidate-role matching
• Public Administration: Transition from traditional machine learning solutions to generative AI and LLM integration

Human-Machine Collaboration Trends

Due to failure cases caused by bias, insufficient transparency, or over-reliance on unsupervised automation, most organizations now embed explicit human-machine collaborative review at critical decision points.

Conclusions and Discussion

Main Conclusions

1. Technical Feasibility: LLMs have reached sufficient maturity to significantly support application review processes.
2. Significant Efficiency Gains: In appropriately integrated human-machine collaboration pipelines, LLMs can substantially accelerate evaluation workflows.
3. Improved Consistency: AI assistance helps improve uniformity in reviewer outputs.

Key Lessons Learned

Organizational and Regulatory Barriers

• Bureaucracy is often the primary cause of delays and reduced solution quality
• Third-party platform ownership limitations restrict system modification capabilities
• Strict GDPR requirements narrow the range of viable models
• Complex multi-step authorization workflows delay data access

Polarized Adoption Patterns

• Reviewers often divide into two groups: those willing to use tools and focused on their advantages, and those becoming very cautious or critical when systems make errors
• Effective change management is critical for successful implementation

High Practical Application Potential

• Large-scale deployment speed far exceeds manual evaluation
• ReClaim system processed approximately 80,000 applications in less than three weeks
• As models continue to improve, fully automated evaluation becomes increasingly feasible

Limitations

1. IExp System: Limited by inability to access historical applications or external databases
2. ReClaim System: Faces challenges with inconsistent document formats and low-quality file submissions
3. Applicable Scope: Approximately 10% of documents excluded from automated parsing due to unsupported formats

In-Depth Evaluation

Strengths

1. Practical Deployment Value: Among few studies reporting real-world LLM deployment experiences, providing important practical guidance
2. Comprehensive Evaluation Framework: Comprehensive evaluation dimensions spanning from technical metrics to user feedback, efficiency improvements to system impacts
3. Dual-Scenario Validation: Validates method generalizability through two different application scenarios
4. Honest Experience Sharing: Objectively reports challenges and failures encountered during deployment

Weaknesses

1. Limited Technical Innovation: Primarily application of existing LLM technology, lacking algorithmic-level innovation
2. Limited Evaluation Scale: Relatively small test sets, particularly 11 samples for IExp task
3. Unknown Long-Term Effects: Only three months of deployment time; long-term effects and stability require further verification
4. Insufficient Cost-Benefit Analysis: Lacks detailed cost-benefit analysis and ROI calculations

Impact

1. Policy-Making Reference: Provides important reference for government agencies adopting AI technology
2. Practical Guidance Value: Offers valuable experience for AI deployment in similar scenarios
3. Cross-Domain Application: Methods generalizable to other fields requiring large-scale document processing

Applicable Scenarios

1. Government Agencies: Various application approvals and document review processes
2. Financial Institutions: Loan applications and compliance reviews
3. Educational Institutions: Application material review and academic evaluation
4. Corporate Organizations: Internal document review and supplier evaluation

References

The paper cites multiple important references, including:

• OpenAI GPT-4o system card (2024)
• EU Artificial Intelligence Act related documents
• Research on LLM applications across various domains
• Best practices research on human-machine collaboration and responsible AI deployment

Overall Assessment: This is an applied research paper with significant practical value. While relatively limited in technical innovation, its real-world deployment experience and comprehensive effectiveness evaluation provide valuable reference for AI applications in the public sector. The paper's honesty and practicality make it an important contribution to the field.