From Simulation to Strategy: Automating Personalized Interaction Planning for Conversational Agents

Basic Information

• Paper ID: 2510.08621
• Title: From Simulation to Strategy: Automating Personalized Interaction Planning for Conversational Agents
• Authors: Wen-Yu Chang, Tzu-Hung Huang, Chih-Ho Chen, Yun-Nung Chen (National Taiwan University)
• Classification: cs.CL (Computational Linguistics)
• Publication Date/Venue: arXiv Preprint (October 8, 2025)
• Paper Link: https://arxiv.org/abs/2510.08621

Abstract

With the rapid emergence of agent-based conversational models, realistic user simulation research is crucial for optimizing effective dialogue strategies. This study investigates a sales-oriented agent that adapts its conversational strategy based on user profiles (including age, gender, and occupation). While age and gender influence overall performance, occupation produces the most significant differentiation in dialogue intents. Based on this insight, the authors introduce a lightweight, occupation-conditioned strategy that guides the agent to prioritize intents aligned with user preferences, thereby achieving shorter and more successful conversations. The results highlight the importance of enriching simulator profiles and demonstrate how simple persona-based strategies can enhance the effectiveness of sales-oriented dialogue systems.

Research Background and Motivation

Core Problem

The core problem addressed by this research is how to implement personalized interaction strategies in sales-oriented dialogue systems to improve dialogue success rates and efficiency. Specifically, the study explores how users' demographic characteristics (age, gender, occupation) influence dialogue dynamics and outcomes.

Problem Significance

1. Practical Application Needs: With the widespread application of AI agents in commercial scenarios, personalized dialogue strategies are crucial for enhancing user experience and business conversion rates
2. User Behavior Variations: Users from different backgrounds exhibit different preferences and behavioral patterns in conversations, requiring targeted strategies
3. Efficiency Enhancement: By understanding user characteristics, dialogue turns can be reduced and success rates improved

Limitations of Existing Methods

1. Lack of Personalization: Existing sales dialogue systems typically employ "one-size-fits-all" strategies without considering individual user differences
2. Insufficient User Modeling: The utilization of user profiles remains superficial, lacking in-depth behavioral analysis
3. Poor Strategy Adaptability: Existing systems struggle to dynamically adjust dialogue strategies based on user characteristics

Research Motivation

Based on the SALESAGENT sales-oriented dialogue system, the authors aim to understand the impact of different user profiles on dialogue effectiveness through large-scale user simulation experiments, and design lightweight personalized strategies based on discovered patterns.

Core Contributions

1. Comprehensive User Profile Analysis: First systematically analyzes the impact of three dimensions—gender, age, and occupation—on sales-oriented dialogue systems, identifying occupation as the most discriminative factor
2. Occupation-Oriented Strategy Framework: Proposes a lightweight dialogue strategy based on occupation that prioritizes recommending relevant intents according to user occupation, significantly improving dialogue success rates
3. Cross-Model Generalization Verification: Validates the generalizability and robustness of the proposed strategy by using different LLMs for strategy derivation and testing
4. Modular Architecture Design: Demonstrates how to seamlessly integrate strategy enhancements into existing LLMs without requiring model retraining

Methodology Details

Task Definition

Input: User profile (gender, age, occupation, MBTI personality) and dialogue history Output: Sales agent response strategy and specific replies Objective: Maximize dialogue success rate (user explicitly expresses purchase intent) and minimize dialogue turns

Experimental Architecture

User Profile Definition

• Gender: Male/Female
• Age: Adolescent (15-19), Adult (20-45), Middle-aged (45-65), Elderly (65+)
• Occupation: Six representative industries selected based on International Standard Industrial Classification (ISIC): Agriculture (Agr), Information Technology (Info), Finance (Fin), Education (Edu), Healthcare (Heal), Arts (Arts)
• Personality: Eight dimensions based on MBTI

Simulation Experiment Setup

1. User Simulator: Uses LLaMA-3.1-8B-Instruct to generate user responses conforming to specific profiles
2. Sales Agent: SALESAGENT system with chain-of-thought reasoning capabilities
3. Dialogue Termination Conditions:
   • Reaches 20-turn limit
   • Agent actively ends conversation ("bye")
   • User explicitly expresses purchase intent

Evaluation Metrics

• Success Rate: Proportion of dialogues where users explicitly express purchase intent
• Average Turns: Average number of turns in successful dialogues
• Continuation Guidance Ratio: Metric measuring agent proactivity

Occupation-Oriented Strategy Design

Based on occupation-intent association patterns discovered in experiments, a lightweight strategy is designed:

Based on occupational statistics, users are likely interested in the following intents: {intents}
Rationale: {rationale}

Examples:

• Agricultural Workers: FindRestaurants, FindAttraction (prioritize leisure and relaxation)
• IT Professionals: SearchHotel, FindRestaurants (business travel needs)
• Finance Professionals: SearchHotel, FindRestaurants (prefer premium services)

Experimental Setup

Dataset

• User Profile Data: 20 unique profiles generated for each fixed attribute condition
• Dialogue Data: 15 dialogues per profile, totaling 9,000 dialogues with up to 180,000 dialogue turns
• Cross-Model Validation: Strategy derivation uses LLaMA-3.1-8B, testing uses Qwen3-8B and Mistral-7B

Evaluation Metrics

1. Success Rate: Proportion of dialogues successfully identifying user intent
2. Average Turns: Average number of turns in successful dialogues (lower is better)
3. Continuation Guidance Ratio: Proportion of agent continuation after topic transitions
4. Intent Distribution: Frequency distribution of intents appearing in dialogues

Comparison Methods

• Baseline Method: Standard SALESAGENT without personalization strategy
• Enhanced Method: SALESAGENT integrated with occupation-oriented strategy

Experimental Results

Main Results

User Profile Impact Analysis

Age Dimension:

• Adults (20-45 years) achieve highest success rate (0.61), adolescents lowest (0.46)
• ANOVA analysis shows statistical significance (p=0.02)
• Possible reason: Adults have stronger purchasing power and decision-making authority

Gender Dimension:

• Male success rate (0.57) slightly higher than female (0.50)
• Difference lacks statistical significance (p=0.15)
• Intent distribution patterns highly similar

Occupation Dimension:

• Agricultural workers achieve highest success rate (0.57)
• Significant differences in intent preferences across occupations (p<0.01)
• Intent frequency positively correlates with success rate

Occupation-Oriented Strategy Effects

Key Findings

1. Occupation as Most Important Personalization Factor: Compared to age and gender, occupation has the most significant impact on dialogue intents and success rates
2. Significant Strategy Enhancement Effects: Success rate improvements exceed 40% for education, healthcare, and arts industries
3. Balance Between Efficiency and Proactivity: Strategy improves success rates while increasing agent proactivity
4. Cross-Model Generalization Capability: Strategy effectiveness validated across different LLMs

Related Work

Personalized Dialogue Systems

• Zhang et al. (2018): First introduces user profiles into dialogue systems for personalized response generation
• Cho et al. (2022): Implicitly detects user personality traits through conditional variational inference
• Wang et al. (2025): Proposes USP framework for modeling implicit user characteristics from dialogue data

LLM Role-Playing Research

• Park et al. (2023): Generative Agents with dynamic memory and reflective reasoning
• Shao et al. (2023): Character-LLM utilizing carefully curated profiles for consistent personality modeling

SALESAGENT System

• Sales-oriented dialogue agent built on SalesBot dataset
• Employs chain-of-thought reasoning with four core strategies
• Enables smooth transitions from chitchat to task-oriented dialogue

Conclusions and Discussion

Main Conclusions

1. Critical Role of Occupation Dimension: Among three user profile dimensions, occupation provides the greatest guidance value for dialogue strategy
2. Effectiveness of Lightweight Strategy: Significant dialogue improvements achieved through prompt engineering alone without model retraining
3. Necessity of Personalized Strategy: Customized strategies for different user groups significantly improve dialogue success rates and efficiency

Limitations

1. Simulation vs. Real Users: Experiments based on LLM-simulated users may not fully capture the complexity of real human behavior
2. Coarse Strategy Granularity: Current strategy operates only at occupational level, without considering individual personality differences
3. Increased Agent Proactivity: Strategy improvements in success rate accompanied by increased agent proactivity, potentially affecting user experience
4. Static Occupation Detection: Requires prior knowledge of user occupation, lacking dynamic detection mechanisms

Future Directions

1. Dynamic Profile Detection: Develop mechanisms to infer user characteristics during dialogue
2. Multi-Dimensional Strategy Fusion: Design finer-grained strategies combining occupation, personality, and other dimensions
3. Reinforcement Learning Optimization: Use RL methods to balance success rates and user experience
4. Real User Validation: Verify strategy effectiveness in real commercial scenarios

In-Depth Evaluation

Strengths

1. Systematic Analysis: First systematic analysis of multiple user profile dimensions' impact on sales dialogue, discovering the critical role of occupation
2. Strong Practicality: Proposed lightweight strategy is easy to deploy without model retraining
3. Rigorous Experimental Design: Cross-model validation ensures result reliability and generalizability
4. Large-Scale Data: 9,000 dialogues provide substantial statistical support

Weaknesses

1. Simulation Limitations: Complete reliance on LLM simulation may introduce biases, lacking real user validation
2. Strategy Simplicity: Current strategy relatively simple, underutilizing rich user profile information
3. Limited Evaluation Metrics: Primarily focuses on success rate and turns, lacking subjective metrics like user satisfaction
4. Domain Specificity: Research focuses on sales scenarios; applicability in other domains remains to be verified

Impact

1. Theoretical Contribution: Provides important empirical evidence and design guidance for personalized dialogue systems
2. Practical Value: Lightweight strategy framework has strong industrial application potential
3. Methodological Innovation: Demonstrates integration of user behavior analysis with dialogue strategy design

Applicable Scenarios

1. E-commerce Customer Service: Recommend relevant products and services based on user occupation
2. Financial Consultation: Provide customized investment advice based on client background
3. Education and Training: Adjust course recommendation strategies based on learner occupation
4. Tourism Services: Recommend suitable tourism products based on user occupational characteristics

References

This paper cites 24 relevant references covering personalized dialogue systems, LLM role-playing, sales dialogue systems, and other research directions, providing solid theoretical foundation.

Overall Assessment: This is a high-quality research paper that systematically analyzes the impact of user profiles on sales dialogue through large-scale simulation experiments and proposes a practical personalized strategy framework. Despite certain limitations, its discovery of occupation's critical role and the effectiveness of lightweight strategies provide important insights for developing personalized dialogue systems.