Sensorimotor Contingencies and The Sensorimotor Approach to Cognition

Basic Information

• Paper ID: 2510.14227
• Title: Sensorimotor Contingencies and The Sensorimotor Approach to Cognition
• Author: Denizhan Pak
• Classification: q-bio.NC (Neuroscience)
• Publication Date: November 11, 2023
• Paper Link: https://arxiv.org/abs/2510.14227

Abstract

The 4E cognition perspective attempts to replace long-standing assumptions in traditional cognitive science. Among the most radical shifts is the rejection of the "sandwich model" of cognition, which posits that mental processes are located between action and perception. Overturning such a long-held assumption requires an accessible theoretical alternative with solid experimental support. A unifying thread among the emerging 4E camp is their shared conviction that sensorimotor contingencies (SMCs) constitute such an alternative.

Research Background and Motivation

Core Issues

1. Challenges to Traditional Cognitive Science: The "sandwich model" in traditional cognitive science treats mental processes as an independent layer situated between perception and action. However, mounting evidence suggests that perception, cognition, and action cannot be so cleanly separated.
2. Challenges from Neuroscientific Evidence: Research has revealed that receptive fields of deep visual stream neurons in the brain change according to action context. For example, when an arm is positioned differently, the same neuron's response pattern to visual stimuli changes, and this can even extend to tool locations during tool use.
3. Support from Behavioral Evidence: The classic Held-Hein experiment demonstrated that passive movement is insufficient to support perceptual learning; active action is required for cats to recognize edges in visual cliff experiments. Similar results have been confirmed in human perceptual adaptation and sensory substitution experiments.
4. Inspiration from Artificial Intelligence: AI research has shown that many complex tasks (such as object classification, navigation, etc.) do not require abstract object or scene representations; simple sensorimotor loops are sufficient.

Research Motivation

This paper aims to provide a unified analytical unit for 4E cognitive science to replace the traditional concept of passive cognitive representation, and to demonstrate how sensorimotor contingencies (SMCs) can become a new framework for understanding cognition.

Core Contributions

1. Proposed a comprehensive theoretical framework for SMCs: Unified the conceptual systems of sensorimotor constraints and sensorimotor habits
2. Established mathematical formalization of SMCs theory: Provided mathematical foundations for both linear and nonlinear SMCs theories
3. Constructed conceptual models of SM environments and SM habitats: Provided new theoretical tools for understanding agent-environment systems
4. Provided computational models of attunement: Explored different physiological space models and their effects on attunement mechanisms
5. Integrated philosophical and scientific perspectives: Connected scientific research on SMCs with philosophy of mind

Methodological Details

Core Concept Definitions

Sensorimotor Contingencies (SMCs)

SMCs are quantifiable regularities between actions and their sensory consequences. For example, the softness of a sponge derives from the action of squeezing it with one's hand; softness can only be perceived through the squeezing action.

Two Categories of SMCs

1. Sensorimotor Constraints (SM Constraints): Direct physical consequences of actions, determined by physical laws
2. Sensorimotor Habits (SM Habits): Action results unfolded over time, involving action/perception cycles

System Components

• SM Environment: The environment containing the agent, containing SM constraints
• Agent: Composed of physiological space (brain and body), sensory systems, and motor systems
• SM Habitat: The lifetime trajectory of an agent in the SM environment

Mathematical Formalization

Linear SMCs Theory

• Agents possess vectors representing current sensory states
• At each time step, an action (represented by a matrix) is selected to change the sensory state
• Multiplication of action matrices and sensory vectors represents SM constraints
• Sequential vector-matrix multiplications over time constitute SM habits

Nonlinear Extensions

• Accounts for the nonlinear characteristics of real sensory and motor systems
• J.L. Gordon's work laid the foundation for nonlinear SMCs theory

Sensory Volume Concept

• Visual System: The visual cone emanating from the eyes constitutes a three-dimensional sensory volume
• Electric Field Sensing: The cylindrical electric field surrounding electric fish
• General Principle: Sensory volume size correlates with decision time, affecting neural network complexity

Experimental Case Analysis

Honeybee Navigation Research

Researchers: Mandyam Srinivasan et al.

Experimental Design:

1. Placed vertical markers in corridors with manipulated spacing
2. Controlled bee flight trajectories by varying marker spacing
3. Fixed color saturation to control contrast variation

Findings:

• Honeybees use optical flow SM environments for visual navigation
• Rely on marker spacing rather than contrast for trajectory control
• Based on edge expansion SM constraint mechanisms

Four-Step Research Pattern:

1. Identify SM habits (entering the hive)
2. Determine SM environment (optical flow)
3. Identify utilized SM constraints (edge expansion)
4. Establish physiological mechanism models

Object Segmentation Research

Researchers: Doris Tsao et al.

Contributions:

• Proposed segmentation algorithms based on SM constraints
• Discovered that different species employ different strategies: primates use edge discontinuity, while tree shrews and rats use texture strategies
• Demonstrated the value of cross-species comparison of SM environments

Three Principles of SM Habitats

1. Low-Dimensional Structure Principle

• Eigenbehaviors: Animal movements can be predicted using a small number of abstract dimensions
• Eigenmodes: Brain activity exhibits core networks describable in low dimensions
• Significance: SM habitats possess more structure than superficially apparent, exhibiting common dimensionality reduction characteristics

2. Syllabic Structure Principle

• Behavioral Syllables: Animal behavior consists of sequences of semi-fixed small action patterns
• Brain Attractors: The brain navigates an energy landscape, falling into attractor basins corresponding to specific sensorimotor habits
• Universality: Syllabic structure is a universal feature of adaptive behavior

3. Cascade Principle

• Definition: Signature events where various phenomena co-occur, leading to significant long-term system changes
• Infant Development: Coordination between caregiver gaze, vocalization, and infant SM habits forms the foundation for name learning and joint attention development
• Neural Avalanches: Cascade phenomena also exist in physiological and brain domains

Computational Models of Attunement Mechanisms

Reinforcement Learning Model

• Core Idea: Agents learn action-sensation pairings by maximizing rewards
• Finding: Complex-bodied multi-task agents naturally develop network-structured SM habitat structures
• Philosophical Implication: Treats rewards as environmental inputs distinct from sensations

Intrinsic Motivation Model

• Core Idea: Rewards derive from physiological processes themselves rather than the environment
• Mechanism: Learning through minimization of prediction error
• Theoretical Basis: Statistical mechanics interpretation of the free energy principle

Homeostatic Control Model

• Core Idea: Maintaining internal balance rather than optimizing external objectives
• Egbert-Barandiaran Model: Agents learn to bounce balls and avoid diamonds because balls provide predictable interaction patterns
• Kelso-Fuchs Model: Models infant arms as self-sustaining oscillators, predicting two stable play patterns

Philosophical Considerations

Historical Origins

• O'Regan and Noë: Introduced SMCs concepts to address the hard problem of consciousness
• MacKay: Emphasized redundant information from action consequences in perception
• Ryle: Proposed that cognitive thimbles do not depend on mental representations but rather on attunement to retinal image changes

Debates on the Nature of Knowledge

Core Question: Do humans possess knowledge about the world?

Traditional View: Cognitive representations form the basis of world knowledge

SMCs View:

• Abstract knowledge is built upon low-level SM habits
• Knowledge is not about the world but about how we interact with it
• If attunement can be achieved without world representation, then knowledge in the traditional sense may not exist

Conclusions and Discussion

Main Conclusions

1. Theoretical Unification: SMCs provides a unified analytical unit for 4E cognitive science, shifting focus from disembodied computational minds to physical organisms adapted to environmental constraints
2. Interdisciplinary Bridge: SMCs demonstrates how philosophy can provide concepts that are rigorously researchable and testable by science
3. Empirical Support: Multiple studies ranging from honeybee navigation to infant development validate the effectiveness of the SMCs framework

Limitations

1. Mathematical Theory Constraints: Current assumptions presume only agent actions change sensory states; integration of dynamic environments remains an open research area
2. Nonlinearity Challenges: The nonlinear characteristics of real sensory and motor systems require more general theoretical frameworks
3. Complete Representation Difficulties: Complete characterization of any organism's SM habitat has not been achieved and may be impossible

Future Directions

1. Technological Development:
   • VR technology for SM environment manipulation and research
   • Automated tools for SM habit analysis and identification
   • Controlled rearing experiments combining real-time recording with VR
2. Theoretical Extensions:
   • Development of mathematical techniques for SM environments
   • Cross-species research on eigenmodes
   • Computational modeling of attunement mechanisms
3. Philosophical Integration: Continued dialogue between scientists and philosophers to clarify the positioning of knowledge between SM environments and physiological space

In-Depth Evaluation

Strengths

1. Theoretical Innovation: Provides a novel theoretical framework for cognitive science, challenging traditional representationalist views
2. Interdisciplinary Integration: Successfully integrates insights from neuroscience, ethology, artificial intelligence, and philosophy
3. Empirical Foundation: Based on extensive experimental evidence, supported by neural recordings to behavioral experiments
4. Mathematical Rigor: Provides formalized mathematical theoretical frameworks, making concepts operationalizable and testable
5. Clarity of Exposition: Complex concepts are clearly explained, terminology is precisely defined, and logical structure is transparent

Shortcomings

1. Theoretical Completeness: Nonlinear SMCs theory remains under development, limiting application in complex real-world scenarios
2. Experimental Validation: While multiple cases provide support, more systematic experimental validation across different species and contexts is needed
3. Computational Complexity: The high-dimensionality of SM habitats makes complete modeling practically difficult
4. Philosophical Controversy: Philosophical positions on the nature of knowledge remain contested, and scientific experiments cannot fully resolve them

Impact and Significance

1. Academic Contribution: Provides important theoretical tools and research methods for 4E cognitive science
2. Practical Value: Holds potential applications for robotics, artificial intelligence, and neuroengineering
3. Paradigm Shift: May promote a paradigm shift in cognitive science from representationalism toward embodied cognition

Applicable Domains

1. Neuroscience Research: Understanding sensorimotor integration and neural plasticity
2. Developmental Psychology: Studying cognitive development and learning mechanisms in infants and young children
3. Robotics: Designing more natural sensorimotor control systems
4. Rehabilitation Medicine: Developing treatment methods based on sensorimotor principles
5. Human-Computer Interaction: Designing more intuitive interfaces and virtual reality systems

References

The paper cites 41 important references spanning from classic Held-Hein experiments to cutting-edge computational neuroscience research, reflecting the deep theoretical foundations and broad empirical support in the field. Key references include the pioneering work of O'Regan & Noë (2001), the dynamical systems interpretation by Buhrmann et al. (2013), and numerous empirical studies in neuroscience and ethology.