A Survey on Multi-Behavior Sequential Recommendation

Basic Information

• Paper ID: 2308.15701
• Title: A Survey on Multi-Behavior Sequential Recommendation
• Authors: Xiaoqing Chen, Zhitao Li, Weike Pan, Zhong Ming
• Category: cs.IR (Information Retrieval)
• Publication Date: August 30, 2023 (arXiv preprint)
• Paper Link: https://arxiv.org/abs/2308.15701
• Journal: IEEE Transactions on Knowledge and Data Engineering

Abstract

Recommendation systems aim to address the information overload problem in traditional information retrieval systems by focusing on recommending the most relevant content from massive information repositories to users. User interactions with systems typically exhibit sequential and heterogeneous characteristics, which has motivated the emergence of Multi-Behavior Sequential Recommendation (MBSR). MBSR represents a relatively novel and worthy research direction that can achieve state-of-the-art recommendation performance through appropriate modeling. This survey aims to clarify the MBSR problem, provide detailed coverage of its problem definition, application scenarios, and challenges, and categorize MBSR methods into neighborhood-based approaches, matrix factorization methods, and deep learning methods.

Research Background and Motivation

Problem Background

1. Information Overload Problem: With the explosive growth of internet information, users face challenges in rapidly obtaining desired content from massive information repositories
2. Limitations of Traditional Recommendation Systems:
   • Single-Behavior Recommendation (SBR) only considers a single type of user feedback, easily encountering data sparsity and cold-start problems
   • Static recommendation methods ignore the temporal nature and dynamic changes of user behavior

Research Motivation

1. Behavior Heterogeneity: In real-world scenarios, users interact with items through multiple types of behaviors (browsing, adding to cart, purchasing, etc.)
2. Sequential Dependency: User behavior exhibits inherent temporal characteristics, reflecting the dynamic changes between short-term and long-term preferences
3. Practical Application Needs: E-commerce, video recommendation, news recommendation and other domains urgently require recommendation algorithms that better align with real user behavior

Core Contributions

1. Systematic Survey: Provides the first comprehensive systematic investigation of the MBSR field, filling the gap of lacking relevant surveys in this emerging area
2. Multi-dimensional Classification Framework: Proposes a MBSR method classification framework based on three dimensions: technology, data, and modeling
3. In-depth Technical Analysis: Provides detailed analysis of the advantages and disadvantages of various methods, offering researchers a clear technical roadmap
4. Future Direction Guidance: Proposes five valuable future research directions to guide the development of this field

Method Details

Task Definition

The MBSR problem primarily focuses on next-item recommendation in heterogeneous feedback sequences. Given:

• User set U
• Item set I
• Behavior set F

The input consists of (user, heterogeneous behavior sequence) pairs: $(u, S_u)$, where: $S_u = \{(i_u^1, f_u^1), ..., (i_u^t, f_u^t), ..., (i_u^{|S_u|}, f_u^{|S_u|})\}$

The objective is to predict user u's preference value $\hat{r}_{u,t+1,j}$ for candidate item j at time step (t+1), and generate a Top-K recommendation list.

Classification Framework

1. Neighborhood-based Methods

• Core Idea: Utilize similarity between users or items for recommendation
• Representative Work: BIS (Bidirectional Item Similarity)
• Advantages: Easy to maintain, strong interpretability
• Disadvantages: Lack of transitivity, difficulty capturing complex user preferences

2. Matrix Factorization Methods

• Core Idea: Decompose user-item interaction matrices into inner products of low-rank matrices
• Representative Work: TransRec++
• Innovation: Introduces behavior transition vectors to capture sequential relationships and behavior dynamics
• Limitations: Model complexity increases dramatically with increasing behavior types

3. Deep Learning Methods (Primary Focus)

RNN Architecture

• RLBL: Combines RNN and log-bilinear models using behavior-specific transition matrices
• RIB: Utilizes GRU and attention mechanisms to model micro-behaviors
• BINN: Designs CLSTM and Bi-CLSTM to process multi-behavior information
• Advantages: Well-suited for sequence modeling, capable of storing short-term memory
• Disadvantages: Gradient vanishing/exploding problems, low computational efficiency

GNN Architecture

• MGNN-SPred: Constructs global graphs to capture behavior transition relationships
• DMBGN: Designed for coupon redemption rate prediction tasks
• GPG4HSR: Simultaneously considers global and personalized graphs
• BGNN: Distinguishes between homogeneous and heterogeneous behavior transition patterns
• Advantages: Capable of modeling complex relationships, strong data sparsity handling
• Disadvantages: Graph construction complexity increases with more behavior types

Transformer Architecture

• DMT: Employs MMoE for multi-task learning
• DFN: Handles implicit feedback noise through attention networks
• NextIP: Models the problem as dual-task learning for item prediction and purchase prediction
• MB-STR: Designs novel position encoding functions
• FLAG: Simultaneously models local preferences, global preferences, and local intents
• Advantages: Strong parallel computing capability, excellent long-term dependency modeling
• Disadvantages: Relatively weaker local information capture

Hybrid Architecture

• MKM-SR: Combines GNN and GRU to separately model item sequences and behavior sequences
• KHGT: Fuses Transformer and GNN, considering item relationship information
• Advantages: Synthesizes advantages of multiple techniques, significant performance improvements
• Disadvantages: High model complexity, difficult hyperparameter tuning

Experimental Setup

Application Scenarios

1. E-commerce Recommendation: Predicts user purchase behavior, considering auxiliary behaviors such as browsing, adding to cart, and favoriting
2. Video Recommendation: Uses sharing as target behavior and browsing as auxiliary behavior
3. News Recommendation: Combines explicit feedback (dislike) and implicit feedback (browsing)

Evaluation Metrics

• Accuracy Metrics: Hit Rate, NDCG, Recall, Precision
• Diversity Metrics: Coverage, Diversity
• Efficiency Metrics: Training time, inference time, memory consumption

Data Perspective Classification

1. (Item, Behavior) Pair Sequences: Most common input format
2. Behavior-specific Subsequences: Item sequences segmented by behavior type
3. Behavior-agnostic Item Sequences: Item sequences ignoring behavior types
4. Behavior Sequences: Pure behavior type sequences

Experimental Results

Main Findings

1. Deep Learning Methods Dominate: Due to the complexity of MBSR problems, most works adopt deep learning methods
2. Transformer Performs Excellently: Demonstrates superior performance compared to RNN and GNN across multiple tasks
3. Hybrid Methods Show Great Potential: Hybrid architectures combining multiple techniques achieve better recommendation performance
4. Importance of Attention Mechanisms: Nearly all excellent methods employ some form of attention mechanism

Technical Trends

1. From Single to Hybrid: Technical development evolves from single architectures toward hybrid architectures
2. From Local to Global: Modeling perspectives expand from local sequences to global graph structures
3. From Simple to Complex: Behavior modeling develops from simple embeddings to complex transition relationships

Related Work

Associated Research Areas

1. Sequential Recommendation: Foundation of MBSR, but considers only single behavior
2. Multi-behavior Recommendation: Considers heterogeneous behaviors but ignores sequentiality
3. Session-based Recommendation: Short-term interaction sequence recommendation
4. Federated Recommendation: Recommendation under privacy protection

Technical Evolution Path

Traditional Collaborative Filtering → Matrix Factorization → Deep Learning → Graph Neural Networks → Transformer → Hybrid Architecture

Conclusions and Discussion

Main Challenges

1. Heterogeneous Behavior Sequence Modeling: How to effectively model sequential relationships of different behavior types
2. Behavior Relationship Modeling: Capturing dependencies and transition relationships between behaviors
3. Long-term and Short-term Preference Fusion: Simultaneously modeling users' stable long-term preferences and dynamic short-term needs
4. Noise and Bias Handling: Addressing noise and selection bias in implicit feedback

Future Directions

1. Data Level: Address data sparsity, imbalance, periodicity, and noise issues
2. Technical Level: Improve computational efficiency, enhance model generalization, combine complementary techniques
3. Optimization Objectives: Transition from single-objective to multi-objective joint optimization
4. Large Language Models: Explore the application potential of LLMs in MBSR
5. Trustworthiness: Improve model interpretability and protect user privacy and security

In-depth Evaluation

Strengths

1. Comprehensiveness: Provides the first systematic survey of the MBSR field with broad coverage
2. Clear Structure: Multi-dimensional classification framework is well-organized and easy to understand and apply
3. Deep Analysis: Thorough analysis of advantages and disadvantages of each method category
4. Forward-looking: Proposed future directions provide strong guidance value
5. Practicality: Provides researchers and engineers with practical technical roadmaps

Limitations

1. Missing Experimental Comparisons: As a survey paper, lacks unified benchmark comparisons across methods
2. Insufficient Dataset Analysis: Limited analysis of characteristics and limitations of existing datasets
3. Limited Industrial Application Discussion: Insufficient discussion of practical deployment experiences in industry
4. Weak Theoretical Analysis: Lacks theoretical analysis of MBSR problem complexity and learnability

Impact

1. Academic Contribution: Establishes a systematic knowledge framework for the emerging MBSR field
2. Practical Value: Provides important reference for multi-behavior recommendation system design in industry
3. Research Guidance: Clarifies the technical development trajectory and future directions of the field
4. Standard Setting: Facilitates the establishment of evaluation standards and benchmarks for the MBSR field

Applicable Scenarios

1. Research Entry Point: Provides comprehensive field knowledge overview for beginners
2. Technology Selection: Guides engineers in selecting appropriate MBSR methods
3. Academic Research: Provides reference for researchers in determining research directions and innovation points
4. Industrial Application: Provides technical roadmaps for enterprises building multi-behavior recommendation systems

References

This paper references 107 related works covering important contributions from multiple domains including recommendation systems, deep learning, and graph neural networks, providing solid theoretical foundation and technical support for MBSR research.

Summary: This is a high-quality survey paper that systematically reviews the current state of research in the emerging field of multi-behavior sequential recommendation, laying important foundations for the field's development. The paper features clear structure and in-depth analysis, with significant value for advancing both theoretical development and practical applications of MBSR.