eye2vec: Learning Distributed Representations of Eye Movement for Program Comprehension Analysis

Basic Information

• Paper ID: 2510.11722
• Title: eye2vec: Learning Distributed Representations of Eye Movement for Program Comprehension Analysis
• Authors: Haruhiko Yoshioka, Kazumasa Shimari, Hidetake Uwano, Kenichi Matsumoto
• Classification: cs.SE (Software Engineering)
• Publication Date/Venue: ETRA '25 (2025 Symposium on Eye Tracking Research and Applications), May 26–29, 2025, Tokyo, Japan
• Paper Link: https://arxiv.org/abs/2510.11722

Abstract

This paper proposes eye2vec, an infrastructure for analyzing eye movement behavior of software developers while reading source code. In traditional eye-tracking research on program comprehension, researchers must pre-select analysis targets (such as control flow or syntactic elements) and then develop analysis methods to extract appropriate metrics from fixation points in source code. Researchers can define areas of interest (AOI) at different levels, such as words, lines, or code blocks, and these differences lead to varying results. Furthermore, the interpretation of word/line fixations varies depending on the analysis purpose. Consequently, eye-tracking analysis is a challenging task that relies on time-consuming manual work by researchers. eye2vec uses distributed representations to represent consecutive fixation points as transitions between syntactic elements. Distributed representations facilitate the adoption of diverse data analysis approaches with rich semantic interpretations.

Research Background and Motivation

Problem Definition

Traditional eye-tracking analysis in program comprehension research faces several key challenges:

1. Pre-selection of Analysis Targets: Researchers must determine in advance the specific syntactic elements or control flow to be analyzed, a choice that largely depends on researcher experience
2. Subjectivity in AOI Definition: Different definitions of areas of interest (word-level, line-level, code block-level) lead to different analysis results
3. Limitations of Single Semantic Analysis: Existing research typically focuses on a single meaning, making it difficult to simultaneously analyze multiple semantic levels
4. Inefficiency of Manual Analysis: Traditional methods require substantial human intervention and time consumption

Research Significance

Understanding developer eye movement patterns is important for:

• Extracting understanding patterns and strategies of expert developers
• Improving development efficiency
• Enhancing developer education
• Optimizing code readability design

This has significant implications. For example, in method summarization tasks, expert developers tend to generate summaries based on the most frequently read lines of code.

Limitations of Existing Methods

1. Dependence on Researcher Experience: The choice of analysis methods has strong subjectivity
2. Missing Semantic Information: Traditional coordinate or line number-based mapping lacks semantic understanding
3. Difficulty in Multi-dimensional Analysis: Challenging to simultaneously consider syntactic elements and structural meaning
4. Low Automation Level: Requires substantial manual feature engineering

Core Contributions

1. Proposed eye2vec Infrastructure: The first analysis framework combining eye-tracking data with distributed representations of source code
2. Achieved Semantic-level Eye Movement Analysis: Through syntactic element mapping rather than coordinate mapping, supporting multi-level semantic interpretation
3. Enabled Automated Multi-dimensional Analysis: Utilizing distributed representations to capture multiple semantic aspects, supporting machine learning and deep learning methods
4. Provided Two Key Application Scenarios: Data mining analysis support and machine learning-based label prediction

Methodology Details

Task Definition

Input: Eye movement coordinate data recorded while developers read source code and corresponding source code Output: Distributed vectors representing eye movement features (eye vectors) Objective: Transform coordinate-based eye movement data into semantically meaningful distributed representations

Model Architecture

eye2vec comprises the following core modules:

1. Coordinate Line/Column Converter

• Function: Converts coordinates recorded by eye-tracking devices into line and column numbers in source code
• Implementation: Uses the iTrace tool for coordinate-to-code-position mapping

2. code2vec Integration

• Function: Extracts path contexts from source code
• Mechanism: Path contexts refer to paths between two words on the abstract syntax tree (AST)
• Output: Path contexts and their corresponding pre-trained embedding vectors

3. Syntax Tree/Eye Linker

• Function: Associates line and column numbers with path contexts
• Processing: Converts eye movement data into vectors of path contexts and fixation counts
• Innovation: Transforms fixation counts into proportions, reflecting developer characteristics

4. Vector Compressor

• Function: Aggregates vectors and fixation counts to generate the final eye vector
• Weighting Mechanism: Applies weights to embedding vectors based on fixation frequency
• Output: Final distributed representation characterizing eye movement features

Technical Innovations

1. Semantic-level Eye Movement Representation

• Innovation: Transition from coordinate mapping to syntactic element mapping
• Advantage: Enables interpretation of eye movement patterns in complex code structures (loops, branches)

2. Path Context Weighting

• Mechanism: Weights path context vectors based on fixation frequency
• Effect: Highlights important semantic relationships that developers focus on

3. Multi-level Semantic Capture

• Capability: Simultaneously supports high-level semantic analysis (understanding strategies) and low-level analysis (variable operation tracking)
• Implementation: Integrates relationships among multiple path contexts through distributed representations

Experimental Setup

Data Input

• Eye Movement Data: Fixation point coordinates when developers read source code
• Source Code: Corresponding program source files
• Constraints: Current version accepts only eye movement data input without associating personal information

Processing Pipeline

1. Eye movement coordinates → Code line/column positions
2. Source code → AST path contexts
3. Associate position information with path contexts
4. Generate weighted distributed representations

Experimental Results

Application Scenario Validation

1. Data Mining Analysis Support

• Capability: Supports exploratory data mining analysis
• Advantage: Provides richer semantic information compared to traditional coordinate-based analysis
• Application: Investigating potential significance of trends in groups of semantically similar syntactic elements

2. Machine Learning Label Prediction

• Task: Predict developer characteristics based on eye movement data
• Prediction Targets:
  • Developer proficiency in specific programming languages
  • Program comprehension level
• Educational Application: Provides adaptive teaching support for individual developers

Technical Validation

• Vector Space Similarity: Capable of measuring similarity between understanding patterns
• Automation Level: Supports automated analysis using machine learning and deep learning techniques
• Multi-dimensional Analysis: Simultaneously considers multiple semantic levels

Related Work

Traditional Eye-Tracking Analysis Research

1. Abid et al. (2019): Eye movement analysis in method summarization tasks, discovering that experts generate summaries based on frequently read lines
2. Lin et al. (2016): Tracking cognitive processes during program debugging
3. Sharafi et al. (2022): Research on developer code navigation strategies

Technical Foundations

1. code2vec (Alon et al. 2019): Method for learning distributed representations of code
2. iTrace (Guarnera et al. 2018): Eye-tracking infrastructure in development environments

Advantages of This Work

• Semantic Understanding: Transcends traditional coordinate mapping, providing semantic-level analysis
• Automation: Reduces dependence on researcher experience
• Multi-dimensionality: Supports simultaneous analysis of multiple semantic levels

Conclusions and Discussion

Main Conclusions

1. eye2vec successfully achieves semantic-level distributed representation of eye movement data
2. The method supports automated multi-dimensional semantic analysis
3. Provides new infrastructure and analysis paradigm for eye-tracking research

Limitations

1. Privacy Protection Constraints: Current version accepts only eye movement data without associating personal information
2. Dependence on Pre-trained Models: Requires pre-trained code representation models such as code2vec
3. Insufficient Validation: Lacks large-scale empirical validation and systematic comparison with traditional methods

Future Directions

1. Tool Development: Build machine learning and deep learning application tools based on eye2vec
2. Automatic Pattern Extraction: Support researchers in efficiently extracting useful understanding patterns
3. Extended Application Scenarios: Explore applications in more program comprehension tasks

In-depth Evaluation

Strengths

1. Methodological Innovation

• Paradigm Shift: Important transition from coordinate mapping to semantic mapping
• Technical Integration: Skillfully combines eye-tracking and code representation learning
• Automation Enhancement: Significantly reduces manual intervention requirements

2. Theoretical Contributions

• Multi-level Analysis: Simultaneously supports high-level and low-level semantic analysis
• Distributed Representation: Introduces modern representation learning methods to eye movement analysis
• Scalability: Provides generic infrastructure for subsequent research

3. Practical Value

• Educational Application: Supports personalized developer education
• Efficiency Improvement: Automated analysis reduces research costs
• Standardization: Provides unified analysis framework

Weaknesses

1. Insufficient Experimental Validation

• Lack of Comparative Experiments: No systematic performance comparison with traditional methods
• Limited Validation Data: Lacks large-scale dataset validation
• Missing Quantitative Results: Lacks specific performance improvement data

2. Incomplete Technical Details

• Missing Parameter Settings: Key hyperparameters not detailed
• Lacking Complexity Analysis: No analysis of computational complexity and efficiency
• Unknown Robustness: Adaptability to different code types not validated

3. Limited Application Scenarios

• Privacy Constraints: Current design limits certain application scenarios
• Strong Dependencies: Depends on multiple external tools and pre-trained models
• Unknown Generalization: Performance on different programming languages not validated

Impact

1. Academic Contribution

• New Research Direction: Opens new pathways for eye-tracking research
• Methodological Innovation: Provides reusable analysis framework
• Cross-disciplinary Integration: Connects software engineering and cognitive science

2. Practical Potential

• Tool Development: Provides foundation for related tool development
• Standard Establishment: Likely to become standard method in the field
• Industrial Application: Has application prospects in developer training and code optimization

Applicable Scenarios

1. Research Applications

• Program Comprehension Research: Provides new tools for cognitive process research
• Developer Behavior Analysis: Supports large-scale developer behavior research
• Educational Research: Programming education effectiveness evaluation and optimization

2. Practical Applications

• IDE Optimization: Improves integrated development environment user experience
• Code Review: Assists code readability assessment
• Training Systems: Developer skill assessment and personalized training

References

Core References

1. Alon et al. (2019): Code2Vec: Learning Distributed Representations of Code - Provides technical foundation for code distributed representations
2. Guarnera et al. (2018): ITrace: Eye Tracking Infrastructure for Development Environments - Provides technical support for eye-tracking
3. Abid et al. (2019): Developer reading behavior while summarizing Java methods - Demonstrates application value of eye movement analysis in program comprehension

Related Literature

• Lin et al. (2016): Cognitive process tracking in program debugging
• Sharafi et al. (2022): Research on developer code navigation strategies

Overall Assessment: This is an innovative infrastructure paper that proposes a novel method combining eye movement data with code semantic representations. While experimental validation requires strengthening, its technical innovation and application potential merit attention. This method provides a new technical pathway for eye movement analysis research in program comprehension, with potential to advance related fields.