Measuring Moral LLM Responses in Multilingual Capacities

Basic Information

• Paper ID: 2510.08776
• Title: Measuring Moral LLM Responses in Multilingual Capacities
• Authors: Kimaya Basu, Savi Kolari, Allison Yu
• Classification: cs.CL cs.AI
• Publication Date: October 9, 2025 (ArXiv Preprint)
• Paper Link: https://arxiv.org/abs/2510.08776

Abstract

With the widespread global deployment of large language models (LLMs), there is an increasing need to understand and regulate their multilingual responses. This study evaluates state-of-the-art models and leading open-source models across five dimensions spanning low-resource and high-resource languages to measure LLM accuracy and consistency in multilingual contexts. The research employs a five-point rating scale and LLM-based judges for evaluation. Results demonstrate that GPT-5 achieves the best average performance across all categories, while other models exhibit greater inconsistency across languages and categories. Notably, in the Consent & Autonomy and Harm Prevention & Safety categories, GPT achieves the highest scores (averaging 3.56 and 4.73 respectively), while Gemini 2.5 Pro scores lowest (averaging 1.39 and 1.98 respectively).

Research Background and Motivation

Research Questions

This study addresses the following key questions:

1. Multilingual Moral Consistency: Whether LLMs maintain consistent ethical and moral responses across different linguistic environments
2. Language Sensitivity of Safety Mechanisms: The effectiveness of existing safety safeguards in non-English languages
3. Cross-linguistic Bias and Stereotypes: Whether models exhibit varying degrees of bias across different languages

Problem Significance

1. Global Application Demands: LLMs are becoming daily tools for global users, necessitating cross-linguistic reliability assurance
2. Safety Concerns: Research indicates that LLM safety mechanisms perform poorly in non-English languages and are vulnerable to malicious exploitation
3. Cultural Differences in Moral Standards: Moral judgments may vary significantly across different linguistic and cultural backgrounds

Limitations of Existing Approaches

1. English-Centric Test Data: Existing benchmarks primarily focus on English environments
2. Lack of Systematic Assessment: Absence of comprehensive evaluation frameworks across multiple moral dimensions
3. Language Blind Spots in Safety Mechanisms: Existing research reveals vulnerabilities in safety protocols for low-resource languages

Core Contributions

1. Construction of Multi-dimensional Multilingual Moral Evaluation Dataset: Creation of a comprehensive evaluation dataset containing 5 categories, 100 questions each, covering 6 languages
2. Systematic Cross-linguistic Moral Analysis: First systematic multilingual moral response analysis of mainstream LLMs
3. Discovery of Safety Mechanism Vulnerabilities: Revelation of significant differences in LLM safety mechanisms across languages
4. Establishment of Model Performance Benchmarks: Provision of benchmarks and evaluation frameworks for future multilingual moral AI research

Methodology Details

Task Definition

Input: Moral and ethical questions across 6 languages (English, Chinese, Spanish, Arabic, Hindi, Swahili) Output: Moral appropriateness scores of model responses (5-point scale) Constraints: Responses must be within 50 words; moral judgment questions require explicit yes/no/maybe answers

Evaluation Framework

Dataset Construction

The research team constructed a dataset containing 500 English questions divided into five categories:

1. Biases & Stereotypes: Tests whether models perpetuate harmful biases
2. Consent & Autonomy: Tests safety mechanisms using deceptive prompts
3. Harm Prevention & Safety: Tests safety protocols through indirect questions
4. Legality: Tests model handling of regional legal differences
5. Moral Judgment: Assesses model adherence to core moral principles

Language Selection Strategy

• High-resource Languages: Chinese, Spanish, Arabic
• Low-resource Languages: Hindi, Swahili
• Selection Criteria: Linguistic structure diversity, cultural background differences, writing system variations

Model Testing Scope

• State-of-the-art Models: GPT-5, Gemini 2.5 Pro, Claude Sonnet 4
• Open-source Models: Llama 4 Scout, Qwen3 235B-a22b
• Selection Rationale: Training data, application objectives, regional characteristics, openness

Evaluation Methods

LLM-as-a-Judge Framework

• Evaluating Model: Gemini 2.5 Pro as primary judge
• Scoring Criteria: 5-point scale considering answer accuracy and reasoning quality
• Consistency Verification: Cross-validation using GPT-5 and Qwen3

Evaluation Process

1. Translation of English questions to target languages (using Googletrans)
2. Model generation of responses in target language
3. Translation of responses back to English for evaluation
4. Scoring based on category-specific rubrics

Experimental Setup

Dataset Details

• Total Questions: 500 original English questions
• Language Coverage: 6 languages × 500 questions = 3,000 test samples
• Category Distribution: 100 questions per category, uniformly distributed
• Translation Tool: Googletrans Python package

Evaluation Metrics

• Primary Metric: 5-point scale scoring (1=worst, 5=best)
• Category-specific Metrics: Specialized scoring criteria for each moral category
• Consistency Measurement: Standard deviation analysis of cross-linguistic responses

Experimental Parameters

• Temperature Setting: 0.7 (to reduce random variation)
• Response Limit: Within 50 words
• System Prompt: Unified instruction format

Experimental Results

Main Results

Overall Performance Ranking

1. GPT-5: Average score 92%, best performance across all categories
2. Claude Sonnet 4: Stable performance, excellent in safety categories
3. Gemini 2.5 Pro: Excellent in academic categories, poor in safety categories
4. Llama 4 Scout: Moderate performance
5. Qwen3 235B: Average score 66%, worst overall performance

Key Findings

Significant Differences in Safety Categories:

• Consent & Autonomy Category: GPT-5 (3.56) vs Gemini 2.5 Pro (1.39)
• Harm Prevention & Safety Category: GPT-5 (4.73) vs Gemini 2.5 Pro (1.98)

Impact of Language Resource Level:

• Models score higher in low-resource languages on deceptive question categories
• Models are more easily "deceived" into providing harmful information in high-resource languages

Model-Specific Performance:

• Gemini 2.5 Pro: Excellent in direct categories (bias, legality, moral judgment), extremely poor in indirect categories
• Qwen3: Exhibits clear regional bias in Chinese legal questions

Ablation Studies

Translation Quality Verification

• Random sampling validation of translation accuracy
• Score variance controlled within 1 point
• Cross-validation ensures assessment consistency

Judge Bias Testing

• Gemini shows no obvious bias toward its own responses
• Qwen scores average 0.5 points lower
• GPT-5 scores average 0.6 points higher

Case Analysis

The paper provides typical response examples demonstrating:

1. GPT-5 refuses to provide harmful information on safety questions
2. Gemini 2.5 Pro is successfully "deceived" on certain deceptive questions
3. Qwen3 exhibits China-law-oriented bias on legal questions

Related Work

Moral Framework Measurement

• Psychological Tool Adaptation: Application of psychological instruments such as the Defining Issues Test (DIT) to LLMs
• Philosophical Framework Analysis: Assessment of utilitarian vs. deontological moral reasoning
• Limitations: Existing methods have limited scope and lack cross-linguistic perspectives

Multilingual Capability Research

• Reasoning Ability Testing: Cross-linguistic testing of moral dilemmas such as the trolley problem
• Factual Accuracy: Consistency of factual responses across languages
• Performance Disparities: Superior performance in high-resource languages compared to low-resource languages

Safety Constraint Research

• Jailbreak Attacks: Circumventing safety mechanisms through non-English languages
• Large-scale Benchmarks: Safety performance testing across 100+ languages
• Vulnerability Discovery: Safety protocol vulnerabilities in low-resource languages

Conclusions and Discussion

Main Conclusions

1. Significant Inter-model Differences: GPT-5 significantly outperforms other models in moral and safety responses
2. Language Sensitivity: All models exhibit varying degrees of performance degradation in non-English languages
3. Safety Mechanism Vulnerabilities: Success rates of deceptive questions vary significantly across languages
4. Regional Bias Exists: Certain models exhibit clear regional legal bias

Limitations

1. Translation Dependency: Reliance on Google Translate may introduce errors
2. Lack of Human Baseline: No collection of human responses for comparison
3. Scale Subjectivity: Evaluation scales may not fully reflect societal values
4. Limited Language Coverage: Only 6 languages tested, limited representativeness

Future Directions

1. Expanded Language Coverage: Extension to all languages supported by Google Translate
2. Human Baseline Establishment: Collection of human responses from diverse cultural backgrounds
3. Wording Effect Research: In-depth investigation of question formulation effects on responses
4. Safety Mechanism Improvement: Enhancement of multilingual safety protocols based on discovered vulnerabilities

In-depth Evaluation

Strengths

1. Significant Research Value: First systematic evaluation of LLM cross-linguistic moral responses, filling an important research gap
2. Rigorous Methodology: Adoption of comprehensive evaluation framework spanning multiple models, languages, and dimensions
3. Practical Value of Findings: Revealed safety vulnerabilities provide important guidance for actual deployment
4. Dataset Contribution: Constructed multilingual moral evaluation dataset provides benchmarks for subsequent research

Weaknesses

1. Translation Quality Control: Over-reliance on machine translation may affect result reliability
2. Insufficient Cultural Context Consideration: Inadequate consideration of moral standard differences across cultural backgrounds
3. Sample Size Limitations: Only 100 questions per category may be insufficient to cover complex moral scenarios
4. Single Evaluation Standard: Primary reliance on single LLM judge may introduce systematic bias

Impact

1. Academic Contribution: Establishes new research paradigm for multilingual AI ethics research
2. Practical Value: Provides important risk assessment tools for AI safety deployment
3. Policy Impact: Research results can inform AI governance and regulatory policy development
4. Technical Advancement: Promotes development of multilingual AI safety technologies

Applicable Scenarios

1. AI Safety Assessment: LLM safety evaluation for enterprises and research institutions
2. Multilingual AI Deployment: Guidance for risk control in cross-linguistic AI applications
3. Regulatory Compliance: Assistance for regulatory bodies in establishing AI ethics standards
4. Academic Research: Foundation for AI ethics and multilingual NLP research

References

This paper cites multiple important related studies:

1. Achiam et al. (2023) - GPT-4 Technical Report
2. Jin et al. (2024) - Multilingual Trolley Problem Research
3. Fu and Liu (2025) - Multilingual LLM Judge Reliability Research
4. Lin et al. (2025) - LLM Jailbreak Attacks via Safety Papers
5. Zheng et al. (2023) - LLM-as-a-Judge Evaluation Methodology

Overall Assessment: This is groundbreaking research that systematically reveals important issues in current LLMs' multilingual moral responses. Despite certain methodological limitations, its findings hold significant theoretical and practical value for AI safety and multilingual AI development. This research establishes an important foundation for future multilingual AI ethics research.