NeuroPINNs: Neuroscience Inspired Physics Informed Neural Networks

Basic Information

• Paper ID: 2511.06081
• Title: NeuroPINNs: Neuroscience Inspired Physics Informed Neural Networks
• Authors: Shailesh Garg, Souvik Chakraborty (Indian Institute of Technology Delhi)
• Classification: physics.comp-ph
• Publication Date: November 11, 2025 (arXiv preprint)
• Paper Link: https://arxiv.org/abs/2511.06081

Abstract

This paper proposes NeuroPINNs, a neuroscience-inspired extension of Physics-Informed Neural Networks (PINNs) that achieves energy-efficient partial differential equation (PDE) solving through biologically-motivated spiking neuron models. Unlike traditional PINNs that rely on continuous activation functions and incur high computational and energy costs, NeuroPINNs leverage Variable Spiking Neurons (VSNs) to enable sparse, event-driven communication. This makes them particularly suitable for deployment on neuromorphic hardware and resource-constrained scenarios such as embedded and edge devices. However, the core challenge lies in reconciling the discontinuous dynamics of spiking neurons with the smooth residual loss functions required by PINNs. To overcome this, the authors employ a novel stochastic projection method inspired by upscaling theory, which faithfully captures spiking behavior while remaining compatible with gradient-based optimization. Experiments demonstrate that NeuroPINNs significantly reduce communication and energy requirements while maintaining high accuracy.

Research Background and Motivation

1. Problem Statement

Partial differential equations (PDEs) are fundamental tools for modeling physical phenomena in science and engineering. While traditional numerical methods (such as finite element and finite difference methods) are accurate, they are computationally expensive. Physics-Informed Neural Networks (PINNs) have emerged as a promising alternative, solving equations by embedding physical laws into the neural network learning process. However, PINNs inherit the inherent drawbacks of deep learning: high energy consumption, which limits their scalability in edge computing and resource-constrained environments.

2. Problem Significance

• Energy Crisis: Training and inference of deep learning models require substantial energy, limiting their application in mobile devices, IoT, and embedded systems
• Real-time Computation Demands: Many scientific computing scenarios require fast, low-power PDE solvers
• Rise of Neuromorphic Hardware: Novel neuromorphic chips (such as Loihi 2) provide event-driven, low-power computing platforms but require adapted algorithms

3. Limitations of Existing Approaches

• Traditional PINNs: Use continuous activation functions with all neurons continuously active, resulting in high energy consumption
• Conversion-based Spiking PINNs (CPINNs): Train conventional neural networks first then convert to spiking networks, requiring long spike sequences for competitive performance and introducing approximation errors during conversion
• Leaky Integrate-and-Fire (LIF) Neurons: Perform well on classification tasks but poorly on regression tasks (the core of PDE solving)
• Existing Spiking PINN Variants: Fail to effectively address discontinuity issues in residual loss computation

4. Research Motivation

Inspired by the energy efficiency of biological neurons, the authors propose incorporating spiking neural networks into the PINNs framework, reducing energy consumption through sparse, event-driven communication mechanisms while maintaining solution accuracy.

Core Contributions

1. Proposes NeuroPINNs Framework: First native integration of Variable Spiking Neurons (VSNs) into PINNs architecture for energy-efficient PDE solving
2. Innovative Gradient Computation Strategy:
   • Employs Stochastic Projection (SP) method to compute gradients required for PDE residual loss, avoiding systematic bias introduction
   • Uses surrogate gradient backpropagation for parameter updates, ensuring computational feasibility
3. VSNs for Regression Tasks: Replaces LIF neurons with graded spiking (non-binary) VSNs, demonstrating superior performance on regression tasks
4. Native Training Paradigm: Direct training in the spiking domain without conversion, avoiding approximation errors and requiring shorter spike sequences
5. Comprehensive Experimental Validation: Validates effectiveness on four representative PDE problems (including regular and irregular domains) and three-dimensional linear elastic micromechanics applications
6. Energy Consumption Analysis: Theoretical and experimental evidence demonstrates significant energy savings (approximately 50-70% of synaptic operation energy) when spike activity is below 100%

Methodology Details

Task Definition

Solving general-form partial differential equations:

Where:

• $u$: Field variable to be solved, defined on spatial domain $\Omega$ and time interval $(0, T]$
• $\mathcal{N}$: Nonlinear differential operator (parameterized by $\alpha_N$)
• $\mathcal{B}$: Boundary operator (parameterized by $\alpha_B$)
• Initial condition: $u(x, 0) = f_i(x)$
• Boundary condition: $\mathcal{B}(u(x, t); \alpha_B) = f_b(x, t)$

Model Architecture

1. Network Representation

NeuroPINN uses a deep neural network to approximate the field variable:

where $w$ denotes trainable parameters.

2. Variable Spiking Neurons (VSNs)

VSN dynamics are defined as:

Key Characteristics:

• $M_{\bar{t}}$: Accumulated membrane potential
• $Th$: Threshold (trainable)
• $\beta_l$: Leak factor (trainable)
• $\sigma(\cdot)$: Continuous activation function
• Produces graded spikes (non-binary), combining advantages of sparse communication and continuous activation

3. Loss Function

Total loss comprises three components:

Technical Innovations

1. Stochastic Projection (SP) Method

Core Challenge: The discontinuous dynamics of VSNs make direct gradient computation infeasible. Using surrogate gradients to compute loss distorts PDE solving.

Solution: Employ stochastic projection method inspired by upscaling theory:

Theoretical Foundation:

• Represent field variable measurement at point $z \neq x$ as: $u(z) = u(x) + \Delta\eta$ where $\Delta\eta$ is zero-mean noise representing microscale fluctuations
• Through Bayesian filtering and Kallianpur-Striebel formula, obtain non-local gradient estimates:

Practical Computation: At collocation point $x_p$, compute gradient using neighboring points $\{x_i^{(n)}\}_{i=1}^{N_t}$:

Advantages:

• Independent of backpropagation
• Preserves network behavior integrity
• Faithfully captures spiking dynamics

2. Dual Gradient Strategy

• SP Method: Computes PDE gradients in residual loss ($\partial u/\partial x$, etc.)
• Surrogate Gradient Backpropagation: Computes loss gradients with respect to parameters ($\partial \mathcal{L}/\partial w$) for parameter updates

This separation ensures balance between physical accuracy and computational feasibility.

3. Energy Efficiency Analysis

Based on SpiNNaker2 energy estimates, synaptic operation energy consumption:

Traditional ANN (100% activation): $E_{ANN} = 12E \cdot n_1 \cdot n_2$

VSN Layer: $E_{VSN-Syn} = 12E \cdot n_1 \cdot n_2 \cdot N_{avg\_spikes}$

where $E$ is energy per addition operation. Energy savings are achieved when average spike count $N_{avg\_spikes} < 1$.

Training Algorithm

Algorithm 1:

1. Sample collocation points, boundary points, and initial points
2. Obtain network predictions
3. Compute first and higher-order derivatives using SP method (Equation 13)
4. Compute total loss $\mathcal{L}_{Total}$
5. Compute $\partial \mathcal{L}/\partial w$ using surrogate gradient backpropagation
6. Update parameters using Adam optimizer

Experimental Setup

Datasets

Standard PDE Problems (E-I to E-IV)

1. E-I: Burgers Equation (one-dimensional time-dependent)
   • Domain: $x \in [0, 1], t \in (0, 1]$
   • Initial condition: $u(x, 0) = \sin(\pi x/l)$
   • Boundary condition: $u(0, t) = u(1, t) = 0$
   • Collocation points: 2601, boundary+initial points: 400
   • Test grid: 201×201
2. E-II: Heat Equation (one-dimensional time-dependent)
   • Domain: $x \in [0, 1], t \in (0, 1]$
   • Source term: $100\sin(\pi x/l)$
   • Collocation points: 2601, boundary+initial points: 400
3. E-III: Poisson Equation (L-shaped domain)
   • Domain: $x, y \in [-1, 1]$ (L-shaped irregular domain)
   • Collocation points: 1935, boundary points: 600
4. E-IV: Poisson Equation (star-shaped domain)
   • Domain: $x, y \in [-1.5, 1.5]$ (star-shaped domain)
   • Analytical solution: $u(x, y) = \exp(-(2x^2 + 4y^2)) + 0.5$
   • Collocation points: 4595, boundary points: 600

Three-Dimensional Linear Elastic Micromechanics Application

• Synthetic Polycrystalline Microstructure:
  • Cubic domain $\Omega \subset \mathbb{R}^3$, voxel resolution: $32^3$
  • Voronoi tessellation generating approximately 15 grains
  • Random crystal orientation per grain (uniform distribution on SO(3))
  • Material: Tetragonal TiAl
• Training Set: 5000 independent microstructure realizations
• Network Architecture: 3D residual convolutional neural network (ResNet)

Evaluation Metrics

Relative L2 Error (percentage):

3D Application: Mean relative error

Energy Metrics:

• Spiking Activity percentage
• Energy ratio $E_r$ (actual energy consumption / 100% activation energy consumption)

Comparison Methods

1. SP-PINN: Traditional PINN using SP method (baseline)
2. SB-NeuroPINN: NeuroPINN using surrogate gradients for loss computation
3. CPINN: Spiking PINN using conversion strategy (256 spike timesteps)

Implementation Details

• Network Configuration (E-I, E-II): 5 layers, 2-40-80-40-1, activation: tanh
• Network Configuration (E-III): 5 layers, 2-40-80-40-1, activation: ELU
• Network Configuration (E-IV): 5 layers, 2-40-120-40-1, activation: tanh
• VSN Configuration: First two activation layers replaced with VSN layers
• Optimizer: Adam
• Spike Timesteps (STS): Default 1 (NeuroPINN), 256 (CPINN)
• Input Encoding: Direct input (no rate or temporal encoding required)

Experimental Results

Main Results

Table 2: Relative L2 Error (%) for Various Models

*DNC = Did Not Converge

Key Findings:

1. NeuroPINN vs SP-PINN: Errors are comparable (E-I: 1.04% vs 0.25%), even superior on E-III (2.70% vs 3.67%)
2. NeuroPINN vs SB-NeuroPINN: Significantly outperforms surrogate gradient loss computation version, validating SP method necessity
3. NeuroPINN vs CPINN: Substantially outperforms conversion strategy (E-I: 1.04% vs 11.87%), with CPINN failing to converge on irregular domains
4. Irregular Domain Performance: NeuroPINN successfully converges on both L-shaped and star-shaped domains, demonstrating robustness

Energy Efficiency Analysis

Figure 6: Spiking Activity Percentage

Spike activity in two VSN layers across examples:

Conclusions:

• Spike activity in all examples far below 100%
• E-IV achieves approximately 70% energy savings (synaptic operations)
• Actual deployment on neuromorphic hardware would yield even greater savings

Multiple Spike Timesteps (STS) Experiments

Table 3: Performance with 2 STS

Findings:

• Using 2 STS further reduces error (E-I: 1.04%→0.49%)
• Cumulative spike activity remains far below 100%
• Demonstrates that multiple STS can improve accuracy while maintaining energy efficiency

Three-Dimensional Micromechanics Application

von-Mises Equivalent Stress Prediction

Observations:

• NeuroPINN maintains reasonable accuracy on complex 3D problems
• Spiking activity <21% in 10 of 12 spiking layers
• Demonstrates applicability potential in practical scientific computing

Ablation Studies

SP Method vs Surrogate Gradients

Comparison of SB-NeuroPINN with NeuroPINN (Table 2):

• On E-I, SB-NeuroPINN error increases to 6.70% (NeuroPINN: 1.04%)
• On E-III, SB-NeuroPINN fails to converge
• Conclusion: SP method is critical for maintaining physical accuracy

Native Training vs Conversion Strategy

Comparison with CPINN:

• CPINN requires 256 STS, NeuroPINN requires only 1-2
• CPINN fails on irregular domains
• Conclusion: Native training is more efficient and robust

Case Analysis

Figures 2-5 show predictions and absolute errors for each method:

• NeuroPINN Predictions: Highly consistent with ground truth, uniformly distributed errors
• SB-NeuroPINN: Exhibits notable deviations in complex regions
• CPINN: Significantly larger errors with non-uniform spatial distribution

Related Work

1. Physics-Informed Neural Networks (PINNs)

• Original PINNs Raissi et al., 2019: Embed physical laws into loss functions
• Application Domains: Heat transfer Cai et al., 2021, power systems Huang & Wang, 2022, soft tissue modeling Liu et al., 2020, fluid mechanics Cai et al., 2021

2. Spiking Neural Networks (SNNs)

• LIF Neurons: Perform well on classification tasks but limited on regression
• VSNs Garg & Chakraborty, 2023-2025: Graded spikes, suitable for regression tasks
• Energy Efficiency Research: Davidson & Furber, 2021; Dampfhoffer et al., 2022

3. Spiking PINN Variants

• Tandale & Stoffel, 2024: Use LIF and recurrent LIF, but don't address residual loss computation
• Theilman et al., 2024 (CPINN): Conversion strategy requiring long spike sequences and approximation assumptions
• This Work's Advantages: Native training + SP method, no approximation errors

4. Gradient Computation Methods

• Stochastic Projection Method Nowruzpour et al., 2019; Navaneeth & Chakraborty, 2023: Derivative-free upscaling theory
• Surrogate Gradient Learning: Widely used in SNN training but distorts physical information

Conclusions and Discussion

Main Conclusions

1. Effectiveness Validation: NeuroPINNs perform well across four PDE benchmarks and 3D practical applications, with accuracy approaching traditional PINNs
2. Energy Efficiency: Achieves approximately 50-70% synaptic operation energy savings through sparse spike communication
3. Method Superiority:
   • Outperforms SB-NeuroPINN using surrogate gradients for loss computation
   • Significantly outperforms conversion-based CPINN
   • Demonstrates robustness on irregular domains
4. Multi-STS Benefits: Using multiple spike timesteps further improves accuracy while maintaining energy efficiency
5. Native Training Advantages: Direct training in spiking domain is more reliable than conversion strategies

Limitations

1. Slightly Lower Accuracy: Compared to traditional SP-PINN, NeuroPINN shows slightly higher errors (though differences are small)
2. 3D Application Accuracy: On complex 3D problems, error increases from 1.68% to 3.43%
3. VSN Energy Efficiency: Graded spikes are less energy-efficient than binary LIF neurons (but still superior to traditional neurons)
4. Neighborhood Point Requirements: SP method requires additional neighborhood points for computation, increasing sampling complexity
5. Training Algorithm: Still relies on surrogate gradients for parameter updates; future work could explore SNN-specific training methods
6. Hardware Verification Missing: Not tested on actual neuromorphic hardware (e.g., Loihi 2)

Future Directions

1. Improved Training Algorithms: Develop training methods completely eliminating surrogate gradients
2. Hardware Deployment: Implementation and testing on neuromorphic chips like Loihi 2
3. Extended Applications: Explore more complex PDE systems (e.g., Navier-Stokes equations)
4. Adaptive STS: Dynamically adjust spike timesteps to balance accuracy and efficiency
5. Theoretical Analysis: Provide theoretical guarantees for convergence and error bounds

In-Depth Evaluation

Strengths

1. Innovation

• First Native Integration: First work to natively integrate VSNs into PINNs
• Clever Dual Gradient Strategy: SP method ensures physical accuracy, surrogate gradients ensure trainability
• Solid Theoretical Foundation: SP method derives from upscaling theory with rigorous mathematical basis

2. Experimental Sufficiency

• Diverse Testing: Covers time-dependent, steady-state, regular and irregular domains
• Comprehensive Comparisons: Multiple baselines including ablation studies
• Energy Analysis: Detailed spike activity and energy estimates
• Practical Applications: 3D micromechanics case demonstrates practical value

3. Result Convincingness

• Consistent Advantages: Outperforms CPINN across all tests
• Robustness: Successful on irregular domains (where competitors fail)
• Quantified Energy Savings: Provides concrete energy ratio data

4. Writing Quality

• Clear Structure: Logical flow from methodology through experiments to analysis
• Mathematical Rigor: Detailed formula derivations and clear notation
• Good Visualization: Intuitive figures and tables presenting results

Shortcomings

1. Method Limitations

• Accuracy Trade-off: Sacrifices some accuracy for energy efficiency (though modest)
• Neighborhood Dependence: SP method requires additional neighborhood points, increasing sampling complexity
• VSN Selection: Insufficient justification for VSN choice over other novel spiking neurons

2. Experimental Defects

• Lack Hardware Verification: Energy savings are theoretical estimates, not tested on actual neuromorphic hardware
• Hyperparameter Sensitivity: No analysis of sensitivity to VSN threshold, leak factor, etc.
• Scalability: Not tested on larger-scale or higher-dimensional problems
• CPINN Tuning: Authors acknowledge CPINN might improve with better calibration

3. Insufficient Analysis

• Convergence Analysis: Lacks theoretical convergence guarantees
• Error Sources: Insufficient analysis of error increase causes
• SP Method Overhead: No quantification of SP method computational cost
• LIF Comparison: While mentioning LIF failure, no detailed results shown

4. Reproducibility

• No Open Source Code: Paper provides no code link
• Missing Implementation Details: Some training details (learning rate schedule, early stopping) not specified
• Randomness Control: Random seed settings not mentioned

Impact

1. Contribution to Field

• Pioneering Work: Opens new direction for energy-efficient scientific machine learning
• Methodological Contribution: SP method combined with spiking networks has general applicability
• Bridging Role: Connects neuromorphic computing and scientific computing domains

2. Practical Value

• Edge Computing: Suitable for mobile devices and IoT scenarios
• Real-time Applications: Low-latency event-driven computation
• Sustainable AI: Reduced energy consumption aligns with green AI trends

3. Potential Impact

• Inspire Follow-up Research: Likely to spawn more spiking PINN variants
• Hardware Motivation: Provides practical applications for neuromorphic hardware
• Interdisciplinary Fusion: Promotes cross-disciplinary collaboration between neuroscience and computational physics

Applicable Scenarios

Ideal Scenarios

1. Resource-Constrained Environments: Embedded systems, edge devices, mobile platforms
2. Real-time PDE Solving: Control systems requiring fast response
3. Large-scale Parallel Computing: Deployable on neuromorphic chip arrays
4. Moderate Accuracy Requirements: Applications tolerating 1-3% error

Unsuitable Scenarios

1. Extreme Accuracy Requirements: Critical aerospace computations
2. Abundant Traditional Hardware: When GPU/CPU resources are sufficient, traditional PINNs simpler
3. Ultra-large Scale Problems: Scalability not fully verified
4. Non-regression Tasks: Method designed specifically for PDE solving

Technical Highlights

1. Dual Gradient Separation: Separates physical gradient computation from parameter gradient computation, balancing accuracy and trainability
2. Graded Spike Mechanism: VSN non-binary output achieves balance between sparsity and expressiveness
3. Conversion-free Training: Avoids approximation errors and long spike sequence requirements of conversion strategies
4. Theoretical Support: SP method has solid stochastic differential equation theory foundation

Key References

1. Raissi et al., 2019: Physics-informed neural networks (original PINN paper)
2. Garg & Chakraborty, 2023: Variable spiking neurons for regression (VSN introduction)
3. Nowruzpour et al., 2019: Derivative-free upscaled theory (SP method theoretical foundation)
4. Navaneeth & Chakraborty, 2023: Stochastic projection for PINNs (SP-PINN)
5. Theilman et al., 2024: Spiking PINNs on Loihi 2 (CPINN)
6. Davidson & Furber, 2021: Energy comparison of ANNs and SNNs (energy analysis)

Overall Assessment

This is a highly innovative and rigorously experimental work that successfully introduces spiking neural networks into the physics-informed neural network domain. The core contribution lies in proposing a native training paradigm and dual gradient computation strategy, effectively resolving the contradiction between spiking neuron discontinuity and PDE solving smoothness requirements. Experimental results demonstrate the potential to achieve significant energy savings while maintaining reasonable accuracy.

Main strengths are methodological rigor and comprehensive experiments, particularly robustness on irregular domains. Main weaknesses are lack of actual hardware verification and theoretical convergence analysis. Overall, this is an important work likely to advance energy-efficient scientific computing development, providing strong demonstration of neuromorphic computing applications in scientific domains. Future work should prioritize hardware deployment and scalability verification.