Functional Tensor Train Neural Network for Solving High-Dimensional PDEs

Basic Information

• Paper ID: 2510.13386
• Title: Functional tensor train neural network for solving high-dimensional PDEs
• Authors: Yani Feng (The University of Hong Kong), Michael K. Ng (Hong Kong Baptist University), Kejun Tang (Great Bay University), Zhiwen Zhang (The University of Hong Kong)
• Classification: math.NA cs.NA
• Publication Date: October 15, 2025 (arXiv preprint)
• Paper Link: https://arxiv.org/abs/2510.13386

Abstract

Discrete tensor train decomposition is widely used to mitigate the curse of dimensionality in solving high-dimensional partial differential equations (PDEs) using traditional methods. However, direct application of tensor train methods typically requires uniform meshes on regular domains, which limits their applicability to non-uniform meshes or irregular domains. To address this limitation, this paper develops a Functional Tensor Train Neural Network (FTTNN) for solving high-dimensional PDEs that can represent PDE solutions on non-uniform meshes or irregular domains. The core of the method is to represent PDE solutions through functional tensor train format, where TT core functions are approximated by neural networks. To provide functional tensor train representation, the paper proposes and investigates the concept of functional tensor train rank and applies it to training with physics-informed loss functions. Due to the properties of tensor train representation, high-dimensional integrals in the loss function can be computed by solving one-dimensional integrals using Gaussian quadrature rules. Numerical experiments demonstrate that the proposed FTTNN outperforms Physics-Informed Neural Networks (PINNs) in solving high-dimensional PDEs on both regular and irregular domains.

Research Background and Motivation

Problem Background

High-dimensional partial differential equations are ubiquitous in scientific and engineering applications, such as Schrödinger eigenvalue problems, fluid mechanics, heat conduction, electromagnetism, and quantum mechanics. Traditional numerical methods (finite differences, finite elements, spectral methods) perform well on low-dimensional problems but face serious challenges from the curse of dimensionality (CoD) in high-dimensional problems.

Limitations of Existing Methods

1. Limitations of Tensor Methods:
   • Discrete tensor decomposition methods (CP decomposition, Tucker decomposition, tensor train decomposition) can reduce computational complexity but typically require uniform meshes on regular domains
   • Difficult to apply to non-uniform meshes or irregular domains
2. Limitations of Neural Network Methods:
   • PINN, DGM, DRM and other methods rely on sampling methods to estimate high-dimensional integrals
   • Sampling introduces statistical errors that may obscure the capabilities of neural networks

Research Motivation

Develop a method that can handle irregular domains while efficiently computing high-dimensional integrals, combining the advantages of low-rank structure in tensor decomposition and the function approximation capability of neural networks.

Core Contributions

1. Proposes Functional Tensor Train Neural Network (FTTNN): Extends discrete tensor train decomposition to continuous domains, enabling handling of high-dimensional PDEs on non-uniform meshes and irregular domains
2. Establishes Theoretical Foundation:
   • Defines functional tensor train rank (Functional Tensor Train Rank)
   • Provides mathematical representation of functional tensor train decomposition
   • Provides theoretical analysis of approximation properties of FTTNN
3. Efficient Integral Computation: Leverages tensor train structure to decompose high-dimensional integrals into one-dimensional integrals, computed exactly through Gaussian quadrature rules, avoiding sampling errors
4. Superior Numerical Performance: Validates FTTNN's accuracy advantages over PINN and other methods on multiple high-dimensional PDE problems

Methodology Details

Problem Formulation

Solve high-dimensional PDE boundary value problems of the form:

Lu(x) = f(x), ∀x ∈ Ω
Bu(x) = g(x), ∀x ∈ ∂Ω

where Ω ⊂ ℝᵈ is a bounded connected polygonal domain, L is a partial differential operator, and B is a boundary operator.

Core Theoretical Framework

1. Functional Tensor Train Rank Definition

Definition 2.4: Given a tensor function u: Ω ⊂ ℝᵈ → ℝ, the functional tensor train rank is defined as:

FTT-rank[u] := (r₀, r₁, ..., rₐ₋₁, rₐ)
rₖ = sup_{A∈S[u]} rank(Aₖ), k = 1, ..., d-1

where Su is the set of sampling tensors of u.

2. Functional Tensor Train Decomposition

Theorem 2.6: If u has FTT-rank (r₀, r₁, ..., rₐ), then there exist TT core functions uᵢ: Ωᵢ → ℝ^{rᵢ₋₁×rᵢ} such that:

u(x) = Σ_{α₀=1}^{r₀} ... Σ_{αₐ=1}^{rₐ} u₁^{(α₀,α₁)}(x₁) u₂^{(α₁,α₂)}(x₂) ... uₐ^{(αₐ₋₁,αₐ)}(xₐ)

Model Architecture

FTTNN Structure

Each TT core function is parameterized by a neural network:

u(x; θ) = u₁(x₁; θ₁)u₂(x₂; θ₂)...uₐ(xₐ; θₐ)

where θ = {θ₁, ..., θₐ} are trainable parameters of all sub-networks.

Loss Function

Physics-informed loss function is employed:

J(θ) = ∫_Ω (Lu(x; θ) - f(x))² dx + β ∫_{∂Ω} (Bu(x; θ) - g(x))² dx

Technical Innovations

1. Tensor Decomposition Computation of High-Dimensional Integrals

For second-order elliptic operators Lu(x; θ) = -c₁Δu(x; θ) + b(x)u(x; θ), the residual loss can be decomposed as:

∫_Ω (Lu(x; θ) - f(x))² dx = I₁ + I₂ + I₃ + I₄ + I₅

Each term can be expressed in tensor train format, computed through one-dimensional integrals:

∫_Ω y₁yₐ dx = ∏ᵢ₌₁ᵈ ∫_{Ωᵢ} (uᵢ-related terms) dxᵢ

2. Boundary Condition Treatment

Boundary conditions are directly embedded through network architecture:

uᵢ(xᵢ; θᵢ) := (xᵢ - aᵢ)(bᵢ - xᵢ)ûᵢ(xᵢ; θᵢ)

Experimental Setup

Test Problems

1. Function Approximation: High-dimensional functions with singularities
2. Irregular Domain Poisson Equation: 3D problem on L-shaped domain
3. High-Dimensional Poisson Equation: d=3,5,7 dimensional problems on regular domains
4. High-Dimensional Helmholtz Equation: Including high wavenumber cases
5. Schrödinger Eigenvalue Problem: d=5,10 dimensional problems

Evaluation Metrics

Relative error:

Relative error = ||u(x; θ) - u(x)||₂ / ||u(x)||₂

Comparison Methods

• Discrete Tensor Train Decomposition (TTD)
• Physics-Informed Neural Networks (PINN)
• Deep Ritz Method (DRM)

Implementation Details

• Network Architecture: Single hidden layer fully connected networks with sine activation
• Optimizer: Adam (5000 epochs) + L-BFGS (1000 epochs)
• Integral Computation: Gauss-Legendre quadrature rules
• Hardware: NVIDIA A100 GPU

Experimental Results

Main Results

1. Function Approximation (Test 1)

2. High-Dimensional Poisson Equation (Test 3)

3. High-Dimensional Helmholtz Equation (Test 4.1)

Key Findings

1. Dimension Sensitivity: FTTNN's advantages over PINN become more pronounced as dimension increases
2. Irregular Domain Adaptability: FTTNN successfully handles irregular geometries such as L-shaped domains
3. High Wavenumber Stability: Handles highly oscillatory problems with k=25π through progressive initialization strategy
4. Eigenvalue Accuracy: Outperforms DRM on Schrödinger eigenvalue problems

Related Work

Development of Tensor Methods

• Theoretical foundations of discrete tensor train decomposition by Oseledets et al.
• Continuous functional tensor train algorithms by Gorodetsky et al.
• Applications in uncertainty quantification and quantum dynamics

Neural Network PDE Solving

• PINN: Pioneering physics-informed neural networks
• DGM: Deep Galerkin method
• DRM: Deep Ritz method
• Ubiquitous sampling error issues

Innovations Compared to Existing Work

1. vs Traditional Tensor Methods: Extension to continuous domains and irregular geometries
2. vs Neural Network Methods: Avoids sampling errors with exact integral computation
3. vs Existing Hybrid Methods: Stronger expressiveness and theoretical guarantees

Conclusions and Discussion

Main Conclusions

1. FTTNN successfully extends discrete tensor train decomposition to continuous function spaces
2. Functional tensor train rank establishes theoretical connections between discrete and continuous representations
3. Tensor structure enables efficient and exact computation of high-dimensional integrals
4. Numerical experiments demonstrate superior performance on multiple classes of high-dimensional PDEs

Limitations

1. Tensor Rank Selection: Requires prior knowledge or adaptive strategies to determine appropriate FTT-rank
2. Function Form Restrictions: Source function f(x) and coefficients b(x) need tensor train representation
3. Initialization Sensitivity: High wavenumber problems require carefully designed initialization strategies
4. Theoretical Analysis: Tightness of approximation error bounds needs improvement

Future Directions

1. Fokker-Planck Equations: Extension to more complex stochastic PDEs
2. Adaptive Rank Selection: Develop algorithms for dynamically adjusting FTT-rank
3. More General Operators: Handle nonlinear and time-dependent PDEs
4. Parallel Computing: Exploit natural parallelism in tensor structure

In-Depth Evaluation

Strengths

1. Solid Theoretical Contributions:
   • Rigorous definition of functional tensor train rank
   • Complete theoretical framework for continuous tensor decomposition
   • Mathematical proofs of approximation properties
2. Significant Technical Innovations:
   • Tensor decomposition computation of high-dimensional integrals avoids sampling errors
   • Flexible capability to handle irregular domains
   • Maintains low-rank advantages of tensor methods
3. Comprehensive Experimental Validation:
   • Covers multiple types of high-dimensional PDEs
   • Systematic ablation studies and comparative analysis
   • Progressive validation from low to high dimensions
4. Outstanding Practical Value:
   • Resolves domain limitations of traditional tensor methods
   • Provides new tools for high-dimensional scientific computing

Shortcomings

1. Insufficient Computational Complexity Analysis:
   • Lacks theoretical complexity comparison with methods like PINN
   • Limited analysis of computational overhead in tensor decomposition
2. Limited Applicability Range:
   • Requires tensor decomposition representation of source functions
   • Extensibility to general nonlinear PDEs insufficiently explored
3. Lack of Parameter Tuning Guidance:
   • Systematic guidance for FTT-rank selection is lacking
   • Network architecture design is largely empirical
4. Convergence Analysis:
   • Convergence guarantees for training process are insufficient
   • Analysis of loss function landscape needs deeper investigation

Impact Assessment

1. Academic Impact:
   • Provides new paradigm for combining tensor decomposition and neural networks
   • May catalyze more research on functional tensor decomposition
2. Application Prospects:
   • Significant application value in high-dimensional problems in quantum chemistry and materials science
   • Provides new options for solving complex geometric PDEs in engineering
3. Method Generalizability:
   • Framework has good extensibility
   • Provides inspiration for other high-dimensional numerical problems

Applicable Scenarios

1. High-Dimensional Regular PDEs: Elliptic and parabolic problems with dimension d≥5
2. Irregular Domain Problems: PDEs with complex geometric shapes in medium-low dimensions
3. Eigenvalue Computation: Spectral problems of high-dimensional Schrödinger operators
4. Parametric PDEs: Parameter-dependent problems with low-rank structure

Overall Assessment: This is an important contribution to the field of numerical solution of high-dimensional PDEs. By cleverly combining the advantages of tensor decomposition and neural networks, the paper proposes the FTTNN method with both theoretical depth and practical value. Despite some limitations, its innovation and effectiveness make it a significant advance in the field, worthy of further development and application.