Experimental Demonstration of an Optical Neural PDE Solver via On-Chip PINN Training

Basic Information

• Paper ID: 2501.00742
• Title: Experimental Demonstration of an Optical Neural PDE Solver via On-Chip PINN Training
• Authors: Yequan Zhao, Xian Xiao, Antoine Descos, Yuan Yuan, Xinling Yu, Geza Kurczveil, Marco Fiorentino, Zheng Zhang, Raymond G. Beausoleil
• Categories: cs.LG cs.AR physics.optics
• Publication Date: January 1, 2025
• Paper Link: https://arxiv.org/abs/2501.00742

Abstract

Partial differential equations (PDEs) are important mathematical tools in science and engineering. This paper experimentally demonstrates an optical neural PDE solver through backpropagation-free photonic chip training of physics-informed neural networks (PINNs).

Research Background and Motivation

1. Problem to be Addressed

PDE solving holds significant importance in science and engineering. Traditional discretization-based solution methods (such as finite difference and finite element methods) require substantial digital computational resources and runtime. Physics-informed neural networks (PINNs) provide a discretization-free unsupervised approach, but current PINN training typically requires running on powerful GPUs for hours to tens of hours.

2. Problem Significance

PDEs are widely applied in:

• Electromagnetic modeling and IC chip thermal analysis
• Medical imaging
• Autonomous system safety verification

These critical fields therefore require efficient real-time solution schemes.

3. Limitations of Existing Methods

• Traditional numerical methods: Large computational resource requirements, long runtime
• GPU-based PINN training: High energy consumption, unsuitable for edge device deployment
• Backpropagation training of optical neural networks: Requires additional memory and photonic hardware to implement backward computation graphs, necessitates fully differentiable neural network models

4. Research Motivation

Optical neural networks (ONN) offer high throughput, low energy consumption, low latency, and high parallelism solutions. However, training PINNs on photonic chips faces significant challenges, particularly in implementing backpropagation.

Core Contributions

1. First experimental demonstration of an optical neural PDE solver based on backpropagation-free photonic chip training
2. Proposed zeroth-order optimization method to address the non-differentiability of optical neural networks without requiring calibration
3. Implemented on-chip PINN training on a 1×4 microring resonator (MRR) weight group
4. Verified concept feasibility with ℓ2 error of 5E-3 on one-dimensional heat equation solving

Detailed Methodology

Task Definition

This paper investigates the task of training PINNs on photonic chips to solve partial differential equations. Specifically, using a one-dimensional heat equation as an example:

• PDE: ∂u/∂t = (1/π²)∂²u/∂x²
• Initial condition: u(x,0) = sin(πx)
• Boundary conditions: u(0,t) = 0; u(1,t) = 0
• Objective: Learn neural network parameters θ such that û(x,t) = f_θ(x,t) approximates the true solution u(x,t)

Model Architecture

PINN Architecture

• Network structure: Feedforward neural network with two hidden layers
• Weight matrix dimensions: 2×4, 4×4, 4×4, 4×1
• Loss function: L = L_r + L_0 + L_b
  • L_r: PDE residual loss
  • L_0: Initial condition loss
  • L_b: Boundary condition loss

Optical Implementation

• 1×4 MRR weight group: Utilizing wavelength division multiplexing (WDM) technology
• Time division multiplexing: Computing different weight matrices through different clock cycles
• Weight encoding: Controlling MRR weight values (range 0 to 1) through thermal tuning voltage

Technical Innovations

1. Zeroth-Order Optimization Replacing Backpropagation

Traditional backpropagation method:

• ∂L/∂θ = (∂L/∂f)(∂f/∂θ)
• Requires differentiable neural network models
• Requires additional backward computation hardware

Zeroth-order optimization method:

• Gradient estimation: ∂L/∂θ ≈ (L⁺ - L⁻)/(2μ)
• Requires only forward propagation
• Treats optical neural network as a black box

2. Zeroth-Order Method for Derivative Computation

Spatial derivatives: ∂²û/∂x² = (û₊ + û₋ - 2û)/(2μ) Temporal derivatives: ∂û/∂t = (û₊ - û₋)/(2μ)

Where û₊ and û₋ correspond to network outputs after adding and subtracting perturbations δ to the inputs, respectively.

3. Calibration-Free Training

• Directly optimizes tunable parameters (voltage values) of photonic devices
• Manufacturing errors and environmental noise can be automatically mitigated during training
• No pre-calibration process required

Experimental Setup

Hardware Platform

• Chip: 1×4 MRR weight group chip manufactured by AMF foundry
• Laser: Tunable laser
• Detector: Photodetector
• Control: Four voltage sources for thermal tuning

Network Configuration

• Input dimension: 2 (spatial coordinate x and temporal coordinate t)
• Hidden layers: 2 layers with 4 neurons each
• Output dimension: 1 (PDE solution û)
• Activation function: σ (sigmoid)

Evaluation Metrics

• ℓ2 error: Computed as the ℓ2 norm error between learned and true solutions on test point sets
• Visualization: Heat maps showing the value distribution of PDE solution u(x,t)

Training Configuration

• Optimization algorithm: Zeroth-order stochastic gradient descent (ZO-SGD)
• Training iterations: 1000
• Perturbation parameter: μ for gradient estimation

Experimental Results

Main Results

1. Numerical Simulation Results

Using experimentally measured voltage-to-weight lookup tables for simulation:

• 8-bit precision: Limited expressiveness prevents learning physical laws
• 10-bit and higher precision: Effectively learns PDE solutions

2. Hardware Experimental Results

• Final ℓ2 error: 5E-3
• Training convergence: ℓ2 error curves show gradual capture of physical laws described by PDEs during training
• Bit precision: Experimental chip training precision falls between 8-bit and 10-bit

Ablation Studies

The paper compares performance across different bit precisions:

• 8-bit: Poor performance, ineffective learning
• 10-bit: Significantly improved performance
• Hardware experiment: Falls between 8-bit and 10-bit

Experimental Findings

1. Robustness to manufacturing errors: Zeroth-order optimization method effectively trains despite manufacturing errors
2. Environmental noise adaptability: Training process adapts to environmental noise
3. Importance of bit precision: Limited bit precision in simulation constrains the accuracy of final learned solutions

Related Work

1. Physics-Informed Neural Networks (PINN)

• PINN framework proposed by Raissi et al. provides deep learning methods for solving forward and inverse problems
• Traditional PINN training relies on backpropagation with large computational overhead

2. Optical Neural Network Training

• Stochastic zeroth-order optimization: FLOPS method proposed by Gu et al.
• In-situ backpropagation: In-situ backpropagation of photonic neural networks implemented by Pai et al.
• Tensor compression methods: TT-PINN and TONN enable large-scale applications

3. Advantages of This Work

• First application of zeroth-order optimization to optical PINN training
• Experimental verification of concept feasibility
• No need for complex calibration procedures

Conclusions and Discussion

Main Conclusions

This paper successfully demonstrates an optical neural PDE solver based on zeroth-order optimization, achieving optical implementation of PINNs through backpropagation-free chip training, providing a new solution for real-time edge PDE solving.

Limitations

1. Scale limitations: Current experiments conducted only on 1×4 MRR weight groups, relatively small scale
2. Precision limitations: Limited bit precision in simulation affects final accuracy
3. PDE types: Verified only on one-dimensional heat equation, requires extension to more complex PDEs
4. Convergence speed: Zeroth-order optimization converges slower than first-order methods

Future Directions

1. Scale expansion: Implement thousand-neuron scale using tensor-decomposed PINN (TT-PINN) and tensorized optical neural networks (TONN)
2. Precision improvement: Enhance simulation precision
3. Application extension: Extend to more types of PDE problems

In-Depth Evaluation

Strengths

1. Strong innovation: First chip-level training of optical PINN, pioneering work
2. Clear technical approach: Zeroth-order optimization cleverly solves the non-differentiability problem of optical devices
3. Sufficient experimental validation: Complete verification chain from simulation to hardware experiments
4. High practical value: Provides new insights for edge computing PDE solving

Weaknesses

1. Limited scale: 1×4 weight group scale is too small, significant gap remains before practical deployment
2. Insufficient precision: 5E-3 error may be insufficiently accurate for certain applications
3. Missing complexity analysis: Lacks detailed computational complexity and energy consumption analysis
4. Insufficient generalization verification: Verified only on a simple PDE, generalization capability unknown

Impact

1. Academic value: Opens new direction combining optical computing with scientific computing
2. Technology advancement: Provides important reference for practical implementation of optical neural networks
3. Application prospects: Potential applications in edge computing, real-time simulation, and other fields

Applicable Scenarios

1. Edge computing environments: Resource-constrained scenarios requiring real-time PDE solving
2. Low-power applications: Energy-sensitive mobile devices or IoT devices
3. Concept verification: Technology verification platform for optical neural computing

References

This paper cites important works in PINN, optical neural networks, zeroth-order optimization and related fields, particularly:

• Pioneering work on PINN by Raissi et al.
• Related research on optical neural network training
• Recent advances in tensor-compressed neural networks

Overall Assessment: This is pioneering work at the intersection of optical computing and scientific computing. Although current scale and precision are limited, it establishes important foundations for future optical PDE solver development. The technical approach is sound, experimental validation is comprehensive, and it possesses significant academic value and application prospects.