Interoperability From OpenTelemetry to Kieker: Demonstrated as Export from the Astronomy Shop

Basic Information

• Paper ID: 2510.11179
• Title: Interoperability From OpenTelemetry to Kieker: Demonstrated as Export from the Astronomy Shop
• Authors: David Georg Reichelt (Lancaster University Leipzig), Shinhyung Yang (Kiel University), Wilhelm Hasselbring (Kiel University)
• Classification: cs.SE (Software Engineering), astro-ph.IM (Instrumentation and Methods for Astrophysics)
• Publication Date: October 13, 2025
• Paper Link: https://arxiv.org/abs/2510.11179

Abstract

This paper addresses the interoperability issue between the observability framework Kieker and the OpenTelemetry standard. While Kieker offers rich analytical capabilities, it supports only a limited set of programming languages (Java, C, Fortran, Python), whereas OpenTelemetry provides reference implementations for most programming languages, including C# and JavaScript which Kieker does not support. The paper describes how to convert OpenTelemetry trace data into Kieker framework format, enabling the generation of analytical results such as call trees based on OpenTelemetry instrumentation. The feasibility of the approach is validated through trace data visualization of the Astronomy Shop, an OpenTelemetry demonstration application.

Research Background and Motivation

Problem Definition

1. Language Support Limitations: Although the Kieker framework provides powerful analytical capabilities and low-overhead characteristics, it supports only limited languages such as Java, C, Fortran, and Python
2. Standardization Requirements: OpenTelemetry, as a de facto standard, provides agent implementations for multiple programming languages but cannot directly leverage Kieker's analytical capabilities
3. Missing Interoperability: The absence of effective data conversion mechanisms between the two frameworks limits the combined utilization of their respective advantages

Research Significance

• Modern microservice architectures typically employ polyglot technology stacks, requiring unified observability solutions
• Kieker's low-overhead advantages and OpenTelemetry's broad language support are complementary
• Achieving interoperability can fully leverage the respective strengths of both frameworks

Limitations of Existing Approaches

• Lack of data conversion schemes from OpenTelemetry to Kieker
• Fundamental differences between asynchronous and synchronous tracing concepts create compatibility challenges
• Existing tools cannot utilize OpenTelemetry's multilingual agent ecosystem while maintaining Kieker's analytical capabilities

Core Contributions

1. Implemented data format conversion from OpenTelemetry to Kieker, enabling the use of OpenTelemetry's rich agent ecosystem within the Kieker analysis framework
2. Resolved conceptual differences between asynchronous and synchronous tracing through the introduction of asynchronous marking mechanisms to handle incompatible control flow representations
3. Provided comprehensive field mapping schemes establishing correspondence relationships between the two data formats
4. Validated method feasibility through the Astronomy Shop case study, demonstrating trace data analysis capabilities for multilingual microservice applications

Methodology Details

Task Definition

Convert trace data in OpenTelemetry format (received via gRPC) into Kieker monitoring log format, enabling processing by Kieker's analysis pipeline to generate analytical results such as call trees and component diagrams.

Data Format Analysis

Kieker Data Format

• Uses Instrumentation Recording Language (IRL) to define data formats
• Record structures defined via Xtext
• Supports synchronous tracing with only one call execution at a time
• Uses execution order index (eoi) and execution stack size (ess) to represent control flow

OpenTelemetry Data Format

• Standard format based on protobuf serialization
• Supports three observability pillars: logs, metrics, and traces
• Traces composed of spans, each containing name, timestamp, attributes, etc.
• Supports asynchronous tracing through parent-child relationships representing control flow

Conversion Methodology

Basic Field Mapping

Established direct field correspondence relationships:

• startEpochNanos → tin
• endEpochNanos → tout
• name → signature
• Hostname generated through attribute combinations in OpenTelemetry semantic conventions

Control Flow Conversion Strategy

Facing fundamental differences between asynchronous and synchronous tracing, four solution approaches were proposed:

1. Trace Linearization: Arranges calls by caller but significantly increases resource usage
2. Direct Conversion: Bypasses monitoring logs for direct ExecutionTrace conversion but violates Kieker's architectural separation
3. New Asynchronous Record Types: Requires rewriting substantial portions of analysis pipelines
4. Asynchronous Marking Scheme: Adds asynchronous markers to traces for special handling during analysis

Solution 4 was ultimately selected, processing asynchronous traces through the --asynchronousTrace flag in kieker-trace-analysis.

Technical Innovations

1. Asynchronous Compatibility Handling: Innovatively resolved compatibility issues between two different tracing models through a marking mechanism
2. Semantic Mapping Strategy: Established intelligent mapping relationships between OpenTelemetry semantic conventions and Kieker fields
3. Architecture Preservation: Achieved interoperability without compromising Kieker's original architecture

Experimental Setup

Test Application

Astronomy Shop was used as the demonstration application:

• Default demonstration application for OpenTelemetry
• Contains 14 services using 11 different programming languages
• Includes .NET and TypeScript services that Kieker cannot directly support

Experimental Environment

• Astronomy Shop with OpenTelemetry instrumentation enabled
• Kieker-otel-transformer activated for data conversion
• Kieker trace analysis tools employed for visualization

Evaluation Methodology

• Verified conversion correctness through call tree generation
• Analyzed visualization effectiveness of inter-service call relationships
• Validated completeness of cross-language trace data

Experimental Results

Primary Results

Successfully implemented data conversion from OpenTelemetry to Kieker and generated complete call tree visualizations. Figure 3 demonstrates call relationships between product and recommendation services, proving the effectiveness of the conversion method.

Case Analysis

Through actual execution of Astronomy Shop:

• Successfully captured call relationships among multilingual services
• Generated call trees clearly displayed service dependencies
• Validated the practical utility of the asynchronous tracing marking mechanism

Experimental Findings

1. Structural Difference Impact: OpenTelemetry's asynchronous tracing model and Kieker's synchronous model exhibit fundamental differences
2. Marking Mechanism Effectiveness: The asynchronous marking scheme effectively handles complex calling patterns in microservice applications
3. Cross-Language Support: Successfully extended Kieker's language support range

Related Work

Primary Research Directions

1. OpenTelemetry Data Processing: Weber et al. investigated OpenTelemetry and Palladio interoperability for performance prediction
2. Trace Data Compression: TraceZip proposed a compression storage scheme for OpenTelemetry data, reducing memory requirements by 33.8%
3. Model Transformation: Groner et al. studied developer perspectives on data model transformation

Advantages of This Work

• First work implementing OpenTelemetry to Kieker conversion
• Capable of executing complete Kieker analysis workflows
• Maintains Kieker's low-overhead advantages

Conclusions and Discussion

Main Conclusions

1. Successfully implemented conversion of OpenTelemetry trace data to Kieker format
2. Resolved compatibility issues between two tracing models through asynchronous marking mechanisms
3. Extended Kieker's language support capabilities, enabling analysis of multilingual microservice applications

Limitations

1. Asynchronous Processing Complexity: Manual specification of asynchronous markers increases usage complexity
2. Conceptual Differences: Fundamental differences between tracing models limit complete compatibility
3. Performance Considerations: The paper lacks in-depth analysis of conversion process performance overhead

Future Directions

1. Native Support: Direct support for OpenTelemetry data format in Kieker
2. Hybrid Tracing: Combining Kieker's low overhead with OpenTelemetry's broad applicability
3. Performance Optimization: Investigating performance characteristics of different conversion implementations

In-Depth Evaluation

Strengths

1. High Practical Value: Addresses interoperability between two important observability frameworks
2. Methodological Innovation: The asynchronous marking mechanism innovatively resolves tracing model differences
3. Sufficient Validation: Method feasibility verified through actual multilingual applications
4. Architecture-Friendly: Achieves extension without compromising original architecture

Weaknesses

1. Insufficient Theoretical Analysis: Lacks theoretical guarantees for conversion correctness and completeness
2. Missing Performance Evaluation: No performance overhead analysis of the conversion process provided
3. Limited Edge Case Handling: Limited capability in handling complex asynchronous scenarios
4. User Experience: Manual marking requirements increase usage complexity

Impact

1. Technical Contribution: Provides important reference for observability tool interoperability
2. Practical Value: Directly applicable to existing microservice monitoring scenarios
3. Ecosystem Significance: Promotes collaboration between different observability tools

Applicable Scenarios

• Performance monitoring in multilingual microservice architectures
• Scenarios requiring combined OpenTelemetry broad support and Kieker analytical capabilities
• Situations where existing Kieker users wish to extend language support
• Academic research on observability tool interoperability

References

The paper cites 9 relevant references covering important works in observability frameworks, microservice applications, model transformation, and related domains, providing a solid theoretical foundation for the research.

Overall Assessment: This is a highly practical systems software paper that addresses interoperability between two important observability frameworks. While it has limitations in theoretical analysis and performance evaluation, the proposed solution possesses significant practical value and provides valuable tools and methods for the microservice monitoring domain.