2025-11-17T15:28:12.912033

One Prompt Fits All: Universal Graph Adaptation for Pretrained Models

Huang, Zhao, He et al.
Graph Prompt Learning (GPL) has emerged as a promising paradigm that bridges graph pretraining models and downstream scenarios, mitigating label dependency and the misalignment between upstream pretraining and downstream tasks. Although existing GPL studies explore various prompt strategies, their effectiveness and underlying principles remain unclear. We identify two critical limitations: (1) Lack of consensus on underlying mechanisms: Despite current GPLs have advanced the field, there is no consensus on how prompts interact with pretrained models, as different strategies intervene at varying spaces within the model, i.e., input-level, layer-wise, and representation-level prompts. (2) Limited scenario adaptability: Most methods fail to generalize across diverse downstream scenarios, especially under data distribution shifts (e.g., homophilic-to-heterophilic graphs). To address these issues, we theoretically analyze existing GPL approaches and reveal that representation-level prompts essentially function as fine-tuning a simple downstream classifier, proposing that graph prompt learning should focus on unleashing the capability of pretrained models, and the classifier should adapt to downstream scenarios. Based on our findings, we propose UniPrompt, a novel GPL method that adapts any pretrained models, unleashing the capability of pretrained models while preserving the input graph. Extensive experiments demonstrate that our method can effectively integrate with various pretrained models and achieve strong performance across in-domain and cross-domain scenarios.
academic

One Prompt Fits All: Universal Graph Adaptation for Pretrained Models

Basic Information

  • Paper ID: 2509.22416
  • Title: One Prompt Fits All: Universal Graph Adaptation for Pretrained Models
  • Authors: Yongqi Huang, Jitao Zhao, Dongxiao He, Xiaobao Wang, Yawen Li, Yuxiao Huang, Di Jin, Zhiyong Feng
  • Category: cs.LG
  • Conference: NeurIPS 2025
  • Paper Link: https://arxiv.org/abs/2509.22416

Abstract

Graph Prompt Learning (GPL) serves as an effective paradigm connecting graph pretrained models to downstream tasks, alleviating label dependency and upstream-downstream task mismatch issues. Although existing GPL research explores various prompt strategies, their effectiveness and underlying mechanisms remain unclear. This paper identifies two critical limitations: (1) lack of consensus on underlying mechanisms—different strategies intervene in different model spaces (input-level, layer-level, representation-level); (2) limited scenario adaptability—most methods struggle to generalize when facing data distribution shifts. Through theoretical analysis, this paper reveals that representation-level prompting is essentially equivalent to fine-tuning a simple downstream classifier, proposing that graph prompt learning should focus on unleashing pretrained model capabilities while allowing classifiers to adapt to downstream scenarios. Based on this finding, the UniPrompt method is proposed, capable of adapting to any pretrained model and achieving superior performance in both in-domain and out-of-domain scenarios.

Research Background and Motivation

Core Problems

Graph prompt learning aims to address the mismatch between graph pretrained models and downstream tasks, but existing methods face two key challenges:

  1. Unclear Mechanism Understanding: Existing GPL methods encompass three different prompt strategies—input-level, layer-level, and representation-level—but lack a unified theoretical framework to explain how these different mechanisms work.
  2. Insufficient Adaptability: Most methods show significant performance degradation when facing data distribution shifts (e.g., from homophilic to heterophilic graphs), failing to generalize effectively to different downstream scenarios.

Research Significance

  • Theoretical Value: Provides a unified theoretical framework for graph prompt learning, clarifying the essence of different prompting mechanisms
  • Practical Value: Improves the effectiveness of pretrained graph models in few-shot learning and cross-domain transfer
  • Technical Advancement: Promotes the development of graph foundation models, enhancing model generality and adaptability

Limitations of Existing Methods

Through motivational experiments, the paper discovers that existing representation-level prompting methods (e.g., GPPT, GraphPrompt) exhibit unstable performance when switching pretrained models, sometimes even underperforming simple linear probing. This suggests existing methods may fall into a "pseudo-adaptation" trap.

Core Contributions

  1. Theoretical Contribution: First theoretically proves that representation-level prompting is essentially equivalent to linear classifier fine-tuning, proposing a new perspective that "prompts unleash pretrained capabilities while classifiers adapt to downstream scenarios"
  2. Methodological Innovation: Proposes UniPrompt, a universal input-level graph prompting method that learns topological prompt graphs while preserving original graph structure
  3. Experimental Validation: Conducts extensive experiments on homophilic and heterophilic graph datasets, demonstrating method effectiveness in both in-domain and out-of-domain scenarios
  4. Design Principles: Establishes design principles for graph prompt learning, providing guidance for future research

Method Details

Task Definition

Given a graph G=(V,E,X,Y)G = (V, E, X, Y), where VV is the node set, EE is the edge set, X∈RN×FX \in \mathbb{R}^{N \times F} is the feature matrix, and YY is the label set. The objective is to optimize the prediction function through learnable prompt parameters Ψ\Psi while freezing the pretrained encoder fθf_\theta:

max⁡Ψ1∣D∣∑(A,X,y)∈D∑i=1Nlog⁡P(yi∣PredictΨ(A,X,vi;fθ))\max_\Psi \frac{1}{|D|} \sum_{(A,X,y) \in D} \sum_{i=1}^N \log P(y_i | \text{Predict}_\Psi(A,X,v_i; f_\theta))

Theoretical Analysis

Representation-Level Prompting Equivalence Theorem

Theorem 4.1: Given a linear prompt function T(h)=WTh+bTT(h) = W_T h + b_T and classifier C(h)=WCThC(h) = W_C^T h, there exists an equivalent linear classifier C′(h)=WC′Th+bC′C'(h) = W_{C'}^T h + b_{C'} such that (C∘T)(h)=C′(h)(C \circ T)(h) = C'(h).

This theoretical result indicates that representation-level prompting is equivalent to linear probing in both function space and optimization objectives, suggesting focus should shift toward input-level prompting.

UniPrompt Architecture

1. Prompt Initialization

Initialize the prompt graph using kNN:

S_{ij}, & \text{if } S_{ij} \in \text{top-k}\{S_{i \cdot}\} \\ 0, & \text{otherwise} \end{cases}$$ where similarity is computed as: $S_{ij} = \frac{x_i x_j^T}{\|x_i\|_2 \|x_j\|_2}$ #### 2. Parameterization Mechanism Introduce learnable weights $w_{ij}$ for each edge using a gating mechanism: $$\tilde{A}_{ij} = \text{ELU}(w_{ij} \cdot \alpha - \alpha) + 1$$ #### 3. Bootstrap Ensemble Employ an iterative update strategy to prevent model collapse: $$\hat{A}^{(t)} = \tau \hat{A}^{(t-1)} + (1-\tau) \tilde{A}$$ where $\hat{A}^{(0)} = A$ and $\tau \in [0,1]$ controls the balance between original and prompt graphs. #### 4. Optimization Objective Jointly optimize prompt parameters and classifier: $$\min_{\phi, \Psi} \frac{1}{|V_L|} \sum_{v_i \in V_L} \ell_D(g_\phi(f_\theta(p_\Psi(A,X))_i), y_i)$$ ## Experimental Setup ### Datasets Nine node classification datasets are used: - **Homophilic Graphs**: Cora, CiteSeer, PubMed - **Heterophilic Graphs**: Cornell, Texas, Wisconsin, Chameleon, Actor, Squirrel ### Evaluation Metrics - **Accuracy**: Node classification accuracy - **Few-Shot Settings**: 1-shot, 3-shot, 5-shot learning ### Comparison Methods - **Baseline Methods**: Fine-tune, Linear-probe - **GPL Methods**: GPPT, GraphPrompt, All-in-one, GPF/GPF+, EdgePrompt/EdgePrompt+ - **Pretrained Models**: DGI, GRACE, GraphMAE ### Implementation Details - Use 2-layer GCN/GAT as backbone network - Train for 2000 epochs with early stopping patience of 20 - 5 random seeds × 20 repeated experiments ## Experimental Results ### Main Results #### 1-Shot In-Domain Node Classification Achieves significant improvements on heterophilic graphs: - Cornell: Improves from best baseline 34.56% to 51.13% on DGI - Texas: Improves from best baseline 37.50% to 48.21% - Wisconsin: Improves from best baseline 33.91% to 58.75% #### Cross-Domain Node Classification Under 1-shot cross-domain settings: - PubMed: Improves from 46.84% to 55.01% - Cornell: Improves from 40.77% to 51.58% ### Ablation Studies Validates key components through replacement experiments: - **Random_Topo**: Replacing kNN with random topology causes performance degradation - **Simple_Add**: Simple addition replacing bootstrap strategy leads to overfitting - **Discard_Topo**: Completely discarding original graph causes significant performance drop on homophilic graphs ### Hyperparameter Analysis - **τ Parameter**: Heterophilic graphs benefit from smaller τ values (0.999-0.9999), while homophilic graphs show stable performance at τ≥0.9999 - **k Parameter**: Sparse heterophilic graphs benefit most, while dense and homophilic graphs remain relatively stable ### Computational Overhead - Preprocessing time: approximately 1.3 seconds - Training time per epoch increases moderately - GPU memory usage remains acceptable ## Related Work ### Graph Pretraining - **Contrastive Learning Methods**: DGI, GRACE, GraphCL, etc., learn representations by maximizing mutual information - **Generative Methods**: GraphMAE and similar approaches learn representations through masked reconstruction ### Graph Prompt Learning - **Input-Level Prompting**: GPF series methods add prompt vectors in feature space - **Representation-Level Prompting**: GPPT, GraphPrompt, etc., add prompts at output layers - **Layer-Level Prompting**: Integrate prompt information across GNN layers ### Graph Foundation Models Recent developments in graph foundation models provide new application scenarios and challenges for GPL. ## Conclusions and Discussion ### Main Conclusions 1. **Theoretical Insight**: Representation-level prompting is equivalent to linear classifier fine-tuning; focus should be on input-level prompting 2. **Design Principles**: Prompts should unleash pretrained model capabilities while classifiers adapt to downstream tasks 3. **Practical Method**: UniPrompt achieves universal model adaptation through adaptive topological prompting ### Limitations 1. **LLM Integration Limitation**: Does not explore integration with large language models 2. **Hyperparameter Dependency**: τ and k parameters require tuning for different dataset types 3. **Limited Task Coverage**: Primarily evaluates node classification; other graph tasks require validation 4. **Noise Sensitivity**: Relatively sensitive to feature noise ### Future Directions 1. Extend to graph classification, link prediction, and other tasks 2. Combine with LLMs to build more powerful graph foundation models 3. Improve robustness to noise and distribution shifts 4. Explore automatic hyperparameter selection mechanisms ## In-Depth Evaluation ### Strengths 1. **Outstanding Theoretical Contribution**: First theoretically unifies understanding of different prompting mechanisms, providing important insights 2. **Clever Method Design**: Bootstrap ensemble strategy effectively prevents model collapse; kNN initialization reasonably leverages feature similarity 3. **Comprehensive Experiments**: Covers multiple pretrained models, dataset types, and evaluation settings 4. **High Practical Value**: Simple and effective method, easy to implement and deploy ### Weaknesses 1. **Limited Theoretical Analysis**: Primarily addresses linear cases; analysis of nonlinear prompting is insufficient 2. **Computational Overhead**: kNN construction and iterative updates increase computational cost 3. **Parameter Sensitivity**: Key hyperparameters require careful tuning with limited automation 4. **Noise Robustness**: Performance significantly degrades under feature noise ### Impact 1. **Academic Value**: Provides important theoretical foundation and design principles for graph prompt learning 2. **Practical Significance**: Improves adaptability and generalization of pretrained graph models 3. **Research Inspiration**: Guides subsequent research, particularly highlighting the importance of input-level prompting ### Applicable Scenarios 1. **Few-Shot Learning**: Graph learning tasks with scarce annotated data 2. **Cross-Domain Transfer**: Scenarios where pretraining and downstream tasks have different distributions 3. **Heterophilic Graph Processing**: Graph data where traditional homophily assumptions do not hold 4. **Rapid Deployment**: Applications requiring quick deployment of pretrained models ## References The paper cites 91 relevant references covering multiple domains including graph neural networks, graph self-supervised learning, and graph prompt learning, providing a solid theoretical foundation for the research. --- **Summary**: Through in-depth theoretical analysis and extensive experimental validation, this paper provides important theoretical insights and practical methods for the graph prompt learning field. The UniPrompt method is simple, effective, and demonstrates good generality and adaptability, making valuable contributions to the development of graph foundation models.