Is Research Software Science a Metascience?

Basic Information

• Paper ID: 2509.13436
• Title: Is Research Software Science a Metascience?
• Authors: Evan Eisinger, Michael A. Heroux
• Classification: cs.SE (Software Engineering)
• Publication Date: October 10, 2025 (arXiv v2)
• Paper Link: https://arxiv.org/abs/2509.13436

Abstract

As scientific research increasingly relies on computational methods, the reliability of scientific results depends on the quality, reproducibility, and transparency of research software. Ensuring these characteristics is critical to scientific integrity and discovery. This paper explores whether Research Software Science (RSS)—empirical research on how research software is developed and used—should be considered a form of metascience (the science of science). This classification is important because it may affect the recognition, funding, and integration of RSS into research improvement efforts. The paper defines metascience and RSS, compares their principles and objectives, and examines their areas of overlap. Arguments supporting the classification emphasize shared commitments to reproducibility, transparency, and empirical study of research processes. Counter-arguments describe RSS as a specialized field focused on tools rather than the broader scientific enterprise.

Research Background and Motivation

Problem Definition

The core research question addressed is: Should Research Software Science (RSS) be classified as a form of metascience?

Significance

1. Rise of Computational Science: Modern scientific research increasingly depends on computational methods, making research software critical infrastructure for scientific discovery
2. Reproducibility Crisis: The reproducibility crisis facing the scientific community is largely related to the quality, transparency, and reproducibility of research software
3. Resource Allocation Impact: The classification of RSS will directly affect its academic recognition, funding opportunities, and status within research institutions
4. Disciplinary Development: Clarifying the disciplinary nature of RSS facilitates its theoretical development and standardization of practical applications

Limitations of Existing Research

1. Fuzzy Definitions: The boundaries between metascience and RSS remain unclear
2. Weak Theoretical Foundation: Lack of systematic theoretical analysis of RSS's disciplinary characteristics
3. Insufficient Practical Guidance: Existing research primarily focuses on technical aspects, with insufficient analysis of RSS as a scientific discipline

Research Motivation

The authors argue that with the proliferation of scientific computing, there is a need for scientific study of the software tools that support scientific discovery, and such research itself may constitute a contribution to the "science of science."

Core Contributions

1. Conceptual Clarification: Systematically defines the connotations and extensions of metascience and research software science
2. Theoretical Analysis: Analyzes the relationship between RSS and metascience from an epistemological perspective
3. Multi-dimensional Comparison: Compares the two fields across multiple dimensions including methodology, objectives, and principles
4. Classification Framework: Proposes a theoretical framework for determining whether RSS belongs to metascience
5. Practical Guidance: Provides theoretical support for RSS disciplinary development

Methodology Details

Research Methods

The paper employs conceptual analysis and theoretical comparison through the following steps:

1. Literature Review Method: Systematically reviews definitions, characteristics, and development of metascience and RSS
2. Comparative Analysis Method: Compares similarities and differences across multiple dimensions
3. Argument Analysis Method: Presents arguments both supporting and opposing RSS classification as metascience
4. Synthesis and Judgment Method: Provides classification conclusions based on different metascience definitions

Analytical Framework

Definitional Dimensions of Metascience

• Broad Definition: Any empirical effort aimed at improving science
• Narrow Definition: Research focused on systematic and epistemological structures

Characteristic Analysis of RSS

• Technical Dimension: Software modeling, simulation, and data analysis capabilities
• Social Dimension: Team collaboration, workflows, and organizational factors
• Cognitive Dimension: Learning processes and knowledge acquisition mechanisms

Comparison Criteria

1. Research Object: Whether studying the scientific process itself
2. Methodology: Whether employing empirical scientific methods
3. Goal Orientation: Whether aimed at improving scientific practice
4. Scope of Impact: Whether possessing interdisciplinary influence

Core Arguments

Evidence Supporting RSS as Metascience

1. Common Research Methods

• Empirical Orientation: Both employ observation, experimentation, data analysis, and other empirical methods
• Scientific Rigor: Both apply scientific methods to the research process itself, not merely to natural phenomena
• Systematic Research: Both focus on systematic improvement of research processes

2. Overlapping Principles

Reproducibility:

• Metascience views the reproducibility crisis as a core concern
• RSS directly contributes to computational reproducibility through documentation, version control, containerized environments, and other practices

Transparency:

• Metascience advocates research transparency, including data and code sharing
• RSS supports these goals through open science principles and TOP guidelines

Advancement of Scientific Discovery:

• RSS aims to advance scientific discovery by improving software
• This aligns with metascience's goal of improving scientific efficiency and impact

3. Interdisciplinary Integration

RSS requires collaboration with cognitive scientists and social scientists, reflecting the holistic nature of metascience.

Evidence Against RSS as Metascience

1. Distinction in Research Objects

• Tools vs. Processes: RSS primarily studies software tools, while metascience studies scientific processes themselves
• Artifacts vs. Epistemic Structures: Software is an artifact, while metascience focuses on the epistemological structures of knowledge production

2. Disciplinary Positioning Differences

• Specialized Field: RSS may be more appropriately classified as a specialized branch of computer science or software engineering
• Application Orientation: RSS focuses more on practical improvements rather than systematic study of scientific knowledge

3. Limited Scope of Impact

RSS primarily impacts computational science, while metascience has broader interdisciplinary influence.

Experimental Design

Analytical Methods

The paper employs theoretical analysis rather than empirical experimentation, validating arguments through:

1. Textual Evidence: Citing authoritative definitions and research from relevant fields
2. Logical Reasoning: Establishing argument chains through deductive and inductive reasoning
3. Case Analysis: Illustrating RSS characteristics through concrete practical examples
4. Comparative Analysis: Comparing with established metascience branches

Evaluation Criteria

• Definitional Consistency: Conformity with core definitions of metascience
• Methodological Alignment: Adoption of similar research methods
• Goal Alignment: Service to similar academic objectives
• Impact Assessment: Production of corresponding academic impact

Main Findings

Key Conclusions

1. Definition Dependency: Whether RSS belongs to metascience largely depends on how metascience is defined
2. Partial Overlap: RSS and metascience exhibit significant overlap across multiple dimensions but also clear distinctions
3. Practical Contribution: RSS directly contributes to metascience objectives in computational reproducibility
4. Interdisciplinary Nature: RSS embodies cross-cutting characteristics of technical, social, and cognitive dimensions

Classification Recommendations

Based on Broad Metascience Definition: RSS can be viewed as a branch of metascience because it improves scientific practice through empirical methods.

Based on Narrow Metascience Definition: RSS is more appropriately viewed as an independent interdisciplinary field closely related to metascience.

Practical Implications

Regardless of classification, RSS should:

1. Gain Academic Recognition: Be recognized as an independent scientific discipline
2. Secure Funding Support: Enjoy dedicated research funding channels
3. Enhance Institutional Status: Achieve appropriate standing within research institutions
4. Maintain Scientific Rigor: Ensure research tools meet standards for scientific discovery

Related Work

Metascience Research

1. Five Major Areas: Methodology, reporting, reproducibility, evaluation, and incentive mechanisms
2. Core Institutions: Center for Open Science, Metascience Alliance
3. Research Focus: Reproducibility crisis, peer review, data science of science

Research Software Engineering

1. RSE Movement: Rise of research software engineers
2. Practice-Oriented Approach: Focus on practical improvements in software development
3. Tool Development: Building better research software tools

Scientization of Software Research

1. Empirical Software Engineering: Applying empirical methods to software engineering research
2. Software Metascience: Emerging research on software impact science
3. AI-Driven Development: Research on artificial intelligence applications in software development

Conclusions and Discussion

Main Conclusions

1. Conditional Classification: Whether RSS belongs to metascience depends on the breadth of metascience's definition
2. Independent Value: RSS possesses independent disciplinary value regardless of metascience classification
3. Practical Importance: RSS is important for improving scientific practice
4. Development Necessity: Scientific rigor must be applied to research software development

Limitations

1. Theoretical Analysis Constraints: Lack of empirical data supporting theoretical analysis
2. Definitional Controversy: Metascience's own definition remains in development
3. Insufficient Practical Verification: The practical effectiveness of RSS as a discipline requires further verification
4. Fuzzy Scope Definition: The precise scope and boundaries of RSS require clarification

Future Directions

1. Empirical Research: Conduct more empirical RSS research projects
2. Theory Building: Further develop RSS's theoretical framework
3. Practical Application: Promote RSS practices in more research institutions
4. Interdisciplinary Collaboration: Strengthen collaboration with cognitive science and social science

In-Depth Evaluation

Strengths

1. Problem Importance: Addresses core issues in contemporary scientific development
2. Theoretical Depth: Provides in-depth epistemological analysis of disciplinary characteristics
3. Logical Rigor: Clear argument structure with balanced pro and con evidence
4. Practical Value: Provides theoretical guidance for RSS disciplinary development
5. Clear Writing: Well-defined concepts and well-organized presentation

Weaknesses

1. Empirical Absence: Primarily relies on theoretical analysis lacking empirical data support
2. Insufficient Cases: Lacks concrete RSS practice case analysis
3. Lack of Quantitative Analysis: No quantitative comparative analysis provided
4. Limited International Perspective: Primarily based on English-language literature with possible geographic limitations
5. Insufficient Historical Analysis: Relatively brief analysis of RSS development history

Impact Assessment

1. Academic Contribution: Provides important theoretical foundation for RSS disciplinary development
2. Policy Impact: May influence research institutions' recognition and support for software work
3. Practical Guidance: Provides RSS practitioners with theoretical basis for disciplinary identity
4. Interdisciplinary Value: Promotes dialogue between software engineering and philosophy of science

Applicable Scenarios

1. Disciplinary Development: Applicable to RSS theoretical construction and development planning
2. Policy Making: Provides reference for research funding agencies and higher education institutions
3. Practical Guidance: Provides research software developers with disciplinary cognitive frameworks
4. Education and Training: Applicable to graduate education and training in related fields

References

The paper cites 14 important references covering core literature from metascience, research software science, software engineering, and other fields, providing a solid foundation for theoretical analysis. Key sources include:

• Center for Open Science official documentation on the Metascience Alliance
• Michael A. Heroux's foundational work on research software science
• Core theoretical literature in metascience
• Related research in software engineering and computational reproducibility

Overall Assessment: This is a high-quality theoretical analysis paper with significant importance for understanding and advancing the disciplinary development of research software science. While it has limitations in empirical analysis, its theoretical contributions and practical guidance value are noteworthy.