Two Explorative Studies on Tangible Augmented Reality for Neurodevelopmental Disorders

Basic Information

• Paper ID: 2510.14598
• Title: Two Explorative Studies on Tangible Augmented Reality for Neurodevelopmental Disorders
• Authors: Francesco Vona (University of Applied Sciences Hamm-Lippstadt), Giulia Valcamonica (Politecnico di Milano), Franca Garzotto (Politecnico di Milano)
• Classification: cs.HC (Human-Computer Interaction)
• Publication Date: October 16, 2025
• Paper Link: https://arxiv.org/abs/2510.14598

Abstract

This paper explores two Tangible Augmented Reality (TAR) applications—Holomarket and Along the Oceanic Flow (ATOF)—and evaluates their usability and likeability among individuals with neurodevelopmental disorders (NDD) through two exploratory studies. Holomarket simulates a supermarket shopping experience to help users develop fundamental daily living skills; ATOF transforms the user's environment into an oceanic world, enhancing motor coordination and cognitive skills through dolphin-shaped smart objects. Despite certain usability challenges and hardware limitations, positive feedback indicates significant potential for TAR in future therapeutic interventions.

Research Background and Motivation

Problem Definition

This research addresses challenges faced by individuals with neurodevelopmental disorders (NDD) in daily living skills training and therapeutic interventions. NDD encompasses autism spectrum disorder (ASD), attention-deficit/hyperactivity disorder (ADHD), and intellectual disabilities, which affect cognitive, social, and motor functions.

Problem Significance

1. Urgent therapeutic need: NDD impacts individuals' daily living abilities, requiring personalized therapeutic interventions
2. Limitations of traditional approaches: Existing AR/VR therapeutic methods lack tangible interaction, while NDD populations particularly benefit from multisensory engagement
3. Technological development opportunities: Rapid advancement in immersive technologies opens new frontiers for therapeutic interventions

Limitations of Existing Approaches

• AR and VR therapies lack tangible interaction elements
• Absence of specialized design principles for NDD populations
• Insufficient integration of physical and digital interaction

Research Motivation

By combining TAR technology with smart objects, this research aims to create intuitive, highly engaging therapeutic experiences for NDD populations, addressing the gap in haptic feedback present in traditional AR therapies.

Core Contributions

1. Pioneering research: To the authors' knowledge, this is the first study exploring TAR applications in NDD populations
2. Dual application development: Two TAR application systems targeting different skill training domains were developed
3. Empirical evaluation: Exploratory studies validated the feasibility and acceptance of TAR in NDD treatment
4. Design guidance: Provided key insights and design principles for TAR application development
5. Interdisciplinary collaboration: Demonstrated outcomes of collaborative design involving psychologists, care providers, and engineers

Methodology Details

Task Definition

This research defined two distinct therapeutic tasks:

1. Daily living skills training: Developing shopping, arithmetic, and money handling abilities through Holomarket
2. Motor-cognitive training: Enhancing motor coordination and cognitive skills through ATOF

System Architecture

Holomarket System

• Core components:
  • Microsoft HoloLens as AR display device
  • Holographic food shelves and shopping lists
  • RFID-equipped smart checkout counter
  • RFID-tagged food props and currency
• Interaction modes:
  • Each Price Mode: Memory game combined with shopping experience
  • Total Price Mode: Total price calculation and payment training
• Technical implementation: Developed using Unity, Visual Studio, Vuforia, and Mixed Reality Toolkit (MRTK)

ATOF System

• Core components:
  • Immersive oceanic-themed environment
  • Dolphin-shaped smart objects as interaction tools
  • Spatial mapping and environmental adaptation
• Game modes:
  • Infinity Mode: Collecting bubble rings while avoiding obstacles
  • Adventure Mode: Story-driven progressive tasks

Technical Innovation Points

1. TAR paradigm integration: Seamless combination of AR technology with physical smart objects
2. Environmental adaptability: Spatial mapping technology adapts to different physical environments
3. Personalized customization: Supports task difficulty and parameter adjustment based on user capabilities
4. Multisensory feedback: Combines visual, tactile, and auditory feedback to enhance immersion

Experimental Setup

Participants

ATOF Study

• Sample size: 11 NDD patients
• Gender distribution: 5 males (45.5%), 6 females (54.5%)
• Age: Mean 30.09 years (SD=6.24), range 20-44 years
• Diagnostic distribution:
  • Moderate intellectual disability: 45.5%
  • Severe intellectual disability: 27.3%
  • Genetic syndrome: 18.2%
  • Autism spectrum disorder: 9.1%

Holomarket Study

• Sample size: 5 patients with mild cognitive impairment
• Gender distribution: 2 males (40%), 3 females (60%)
• Age: Mean 36.0 years (SD=6.2)

Evaluation Metrics

A 7-point Likert scale was employed to assess two dimensions:

1. Likeability: Users' enjoyment of the application
2. Usability: Application ease-of-use and intuitiveness of interaction mechanisms

Experimental Procedure

ATOF Experiment

• Duration: 2 days
• Day 1: Room exploration and standard tutorial
• Day 2: Advanced tutorial and questionnaire administration
• Environment: 5×4 meter gymnasium

Holomarket Experiment

• Duration: Single session
• Task configuration: 15 food shelf types, 4 shopping list items
• Environment: 3×5 meter central corridor

Experimental Results

ATOF Main Results

Quantitative Results

• Mean likeability score: M = 5.2 (SD = 1.21), out of 7
• Mean usability score: M = 5.4 (SD = 1.36), out of 7

Qualitative Observations

• Most users demonstrated strong interest
• Users expressed desire to try different task configurations, demonstrating the importance of personalization
• Users showed good acceptance of HoloLens without confusion or fear
• Some users reported feeling "transported to another interesting world"

Holomarket Main Results

Quantitative Results

• Mean likeability score: 51.75/70 (SD = 6.94)
• Mean usability score: 63.5/70 (SD = 5.68)

Key Findings

• Users generally appreciated the virtual supermarket environment
• Connection between shopping lists and shelf items received positive feedback
• Pinch gesture operation proved difficult, requiring external assistance
• HoloLens wearing comfort issues necessitated frequent adjustment

Common Challenges

1. Interaction difficulties: Virtual object manipulation lacked intuitiveness
2. Hardware limitations: HoloLens field-of-view restrictions and wearing discomfort
3. Unclear feedback: System feedback mechanisms required improvement
4. Gesture recognition: Pinch-and-drag actions felt unnatural for users

Related Work

TAR Technology Development

• TAR applications in education, therapy, and design domains
• Tangible hologram concept: fusion of virtual objects with physical augmentation
• Capacitive 3D-printed objects enabling remote interaction

NDD Therapeutic Technology

• AR/VR applications in NDD treatment: improvements in motor skills, cognitive function, and social interaction
• Polipo tangible toy: physical interaction enhancing cognitive and social development
• HoloLens 2 pilot studies with severe ASD patients

Research Gaps

Despite promising AR/VR applications in NDD treatment, TAR solutions specifically designed for NDD needs remain lacking.

Conclusions and Discussion

Main Conclusions

1. TAR potential validated: Both applications demonstrated potential as NDD therapeutic tools
2. High user acceptance: Participants generally enjoyed the TAR experience and found it engaging
3. Technical feasibility: TAR paradigm successfully integrated physical and digital interaction
4. Personalization requirements: Customization proved crucial for NDD therapeutic applications

Design Experience Summary

Insights from ATOF

1. Target population: TAR applications proved most beneficial for moderately impaired users but showed promise for severely impaired users
2. Customization importance: Task adjustment based on individual capabilities proved essential for enhancing usability and likeability
3. Shared experience value: Multi-user participation and real-time therapist monitoring enhanced engagement

Insights from Holomarket

1. Realism value: Virtual supermarket environment's authenticity received user appreciation
2. Interaction design challenges: More intuitive interaction methods needed to replace complex gestures
3. Hardware adaptation requirements: Device comfort and field-of-view limitations require technological improvements

Limitations

1. Sample size constraints: Small sample sizes limited statistical analysis and result generalization
2. Short research duration: Unable to assess long-term benefits or improvements
3. Hardware constraints: HoloLens technical limitations affected user experience
4. Interaction complexity: Gesture interaction proved overly complex for the target user population

Future Directions

Technical Improvements

1. Upgrade to HoloLens 2: Improved gesture recognition and interaction intuitiveness
2. Enhanced data collection: Detailed performance analysis metrics
3. Collaborative experiences: Multi-device coordination and cooperative problem-solving tasks

Research Plans

1. Long-term studies: 6-month controlled studies evaluating TAR application effectiveness
2. Expanded scale: Collaboration with three Milan therapeutic centers for larger samples
3. Comparative evaluation: TAR intervention effectiveness compared to traditional therapeutic methods

In-Depth Evaluation

Strengths

1. Strong innovation: First systematic exploration of TAR applications in NDD treatment
2. High practical value: Addresses the actual problem of lacking haptic feedback in AR therapy
3. Rigorous design: Employs co-design methodology integrating professional expertise and user needs
4. Comprehensive evaluation: Assessment from both usability and likeability dimensions
5. Interdisciplinary collaboration: Demonstrates effective integration of technology and medical fields

Limitations

1. Small sample size: Restricts statistical power and generalizability of results
2. Lack of control group: Unable to directly compare with traditional therapeutic methods
3. Short-term evaluation: Cannot assess long-term therapeutic effects
4. Strong technology dependence: Limited by current AR hardware technological constraints
5. High individual variability: Participant heterogeneity increases data analysis complexity

Impact

Academic Contributions

• Pioneered research direction for TAR in NDD treatment
• Provided design framework and evaluation methods for subsequent research
• Filled research gap in haptic interaction within AR therapy

Practical Value

• Offered therapeutic institutions new intervention tool options
• Validated feasibility of immersive technology applications in special populations
• Provided user need insights for related product development

Reproducibility

• Provided detailed technical implementation descriptions
• Use of open-source tools and platforms enhances reproducibility
• Clear experimental procedures facilitate replication by other researchers

Applicable Scenarios

1. Therapeutic institutions: Day care centers and rehabilitation hospitals for supplementary therapy
2. Educational environments: Special education schools for skills training
3. Home use: Home therapeutic support under professional guidance
4. Research applications: NDD treatment effectiveness evaluation and intervention method research

References

This paper cites 18 relevant references, primarily covering:

• TAR technology development and application research
• AR/VR applications in NDD treatment
• Tangible interface and smart object design
• Neurodevelopmental disorder diagnosis and treatment standards

Important references include DSM-5 diagnostic criteria, systematic reviews of AR in autism intervention, and research on tangible digital narrative tools.

Overall Assessment: This is a groundbreaking HCI research paper that systematically explores TAR technology applications in neurodevelopmental disorder treatment for the first time. Despite limitations in sample size and research duration, its innovative technical approach, rigorous design methodology, and positive preliminary results establish an important foundation for field development. The paper's interdisciplinary nature and practical value confer significant academic and social significance.