Montsalvat: Partitioning Java Applications to Minimize the TCB in Intel SGX

Basic Information

• Paper ID: 2305.00766
• Title: Montsalvat: Partitioning Java Applications to Minimize the TCB in Intel SGX
• Authors: Peterson Yuhala, Jämes Ménétrey, Pascal Felber, Valerio Schiavoni, Alain Tchana, Gaël Thomas, Hugo Guiroux, Jean-Pierre Lozi
• Classification: cs.CR (Cryptography and Security)
• Conference: 22nd International Middleware Conference (Middleware 2021)
• Paper Link: https://arxiv.org/abs/2305.00766

Abstract

The popularity of the Java programming language has led to its widespread adoption in cloud computing infrastructure. However, Java applications running in untrusted cloud environments are vulnerable to various privileged attacks. The emergence of Trusted Execution Environments (TEEs) such as Intel SGX mitigates this issue. TEEs protect code and data within secure enclaves from access by untrusted software, including kernels and hypervisors. To efficiently utilize TEEs, developers must manually partition applications into trusted and untrusted components to reduce the size of the Trusted Computing Base (TCB) and minimize security vulnerability risks. This paper presents Montsalvat, a practical and intuitive annotation-based partitioning approach for Java applications targeting secure enclaves. Montsalvat provides RMI-like mechanisms to ensure inter-object communication and consistent garbage collection across partitioned components.

Research Background and Motivation

Problem Definition

1. Security Threats: Java applications in untrusted cloud environments face threats from privileged attacks, including kernel-level and hypervisor-level attacks
2. TCB Minimization Requirements: When using TEEs such as Intel SGX, it is necessary to minimize the Trusted Computing Base to reduce the attack surface
3. Partitioning Complexity: Manual partitioning of Java applications requires handling complex issues such as inter-object communication and garbage collection consistency

Limitations of Existing Solutions

1. Complete Application Approaches: SCONE, Graphene-SGX, and others place entire applications (including the JVM) within enclaves, resulting in large TCBs (millions of lines of code)
2. Framework-Specific Solutions: VC3, SecureKeeper, and others target only specific frameworks, lacking generality
3. Existing Java Partitioning Approaches:
   • Civet requires a complete LibOS, resulting in a still-large TCB
   • Uranus requires third-party tools to infer trusted partitions, requiring significant developer intervention

Research Motivation

Develop a general-purpose, automated Java application partitioning tool capable of:

• Providing an intuitive class-level annotation approach
• Implementing efficient cross-enclave object communication
• Ensuring distributed garbage collection consistency
• Significantly reducing TCB size

Core Contributions

1. Annotation-Based Partitioning Method: Proposes a practical and intuitive class-level annotation approach (@Trusted, @Untrusted, @Neutral) for partitioning Java applications
2. RMI Mechanism: Designs an RMI-like cross-enclave object communication mechanism that implements transparent remote method invocation through proxy objects
3. Distributed Garbage Collection: Implements a GC extension based on weak references to ensure object destruction consistency between trusted and untrusted heaps
4. GraalVM Integration: Deep integration with GraalVM native-image, leveraging AOT compilation for performance optimization
5. Performance Improvements: Achieves significant performance gains in real applications (PalDB 6.6×, GraphChi 2.2×)

Methodology Details

Task Definition

Input: Java application source code with security annotations Output: Partitioned trusted and untrusted native images deployable to Intel SGX environments Constraints: Preserve original application semantics, ensure object communication and GC consistency

System Architecture

Montsalvat's workflow comprises four main stages:

1. Code Annotation Stage

Provides three annotation types:

• @Trusted: Classes and their instances are always created and operated within the enclave
• @Untrusted: Classes and their instances are always operated outside the enclave
• @Neutral: Utility classes that can exist independently on both sides with multiple copies

2. Bytecode Transformation Stage

Uses the Javassist framework for bytecode transformation:

Proxy Class Generation:

• Generates proxy classes for each trusted class in the untrusted runtime
• Generates proxy classes for each untrusted class in the trusted runtime
• Proxy classes retain original method signatures but replace method bodies with cross-enclave calls

Relay Method Injection:

// Original method transformed into relay method
@CEntryPoint
public static void relayAccount(Isolate ctx, int hash, 
                               CCharPointer buf, int b) {
    String s = deserialize(buf);
    Account mirror = new Account(s, b);
    mirrorProxyRegistry.add(hash, mirror);
}

Object Mapping Mechanism:

• Each proxy object is assigned a unique hash value
• Maintains proxy-mirror object registries
• Implements cross-enclave object references through hash values

3. Native Image Partitioning

Leverages GraalVM's static analysis:

• Trusted image: Contains concrete implementations of trusted classes + proxies of untrusted classes
• Untrusted image: Contains concrete implementations of untrusted classes + proxies of trusted classes
• Points-to analysis automatically prunes unreachable code

4. SGX Application Generation

• Shim Library: Lightweight library that intercepts unsupported libc calls and relays them to the untrusted runtime
• EDL Generation: Automatically generates enclave definition language files
• Final Linking: Generates executable SGX applications

Technical Innovations

1. Class-Level Partitioning Strategy

Compared to method-level or data-level partitioning:

• More intuitive for object-oriented programming
• Avoids complex data flow analysis
• Natural encapsulation boundaries

2. Transparent Proxy Mechanism

• Preserves original programming models
• Automatically handles parameter serialization/deserialization
• Supports cross-enclave transmission of complex object graphs

3. Distributed GC Design

// GC synchronization based on weak references
WeakReference<ProxyObject> weakRef = new WeakReference<>(proxy);
proxyWeakRefs.add(new ProxyWeakRefEntry(weakRef, proxy.getHash()));

// GC Helper periodically checks
if (weakRef.get() == null) {
    // Proxy object has been garbage collected, notify other side to destroy mirror object
    notifyMirrorDestruction(hash);
}

Experimental Setup

Experimental Environment

• Hardware: Intel Xeon E3-1270 CPU (3.80 GHz), 64GB RAM
• SGX Configuration: 128MB EPC (93.5MB available), maximum heap 4GB, stack 8MB
• Software: Ubuntu 18.04, SGX SDK v2.11, GraalVM CE v21.0.0
• Baseline Comparison: SCONE (JVM running within enclave)

Evaluation Metrics

• Performance Metrics: Application execution time, operation latency
• Overhead Analysis: Proxy object creation overhead, RMI call overhead, GC performance
• Speedup: Performance improvements relative to unpartitioned and SCONE approaches

Test Applications

1. Synthetic Benchmarks: CPU-intensive and I/O-intensive operations
2. PalDB: LinkedIn's embedded key-value store
3. GraphChi: Large-scale graph processing framework (PageRank algorithm)
4. SPECjvm2008: Standard Java micro-benchmark tests

Experimental Results

Main Performance Results

1. Real Application Performance Improvements

• PalDB: 6.6× improvement over SCONE (RTWU scheme), 2.8× (RUTW scheme)
• GraphChi: 2.2× improvement over SCONE
• SPECjvm2008: Significant improvements in 5/6 benchmarks (1.38×-2.66×)

2. Proxy Object Overhead Analysis

• Proxy object creation is 3-4 orders of magnitude slower than concrete objects
• RMI calls are 3-4 orders of magnitude slower than local calls
• Serialized parameters add 10× overhead (within enclave) to 3× overhead (outside enclave)

3. GC Performance Impact

• GC within enclave is 1 order of magnitude slower than outside enclave
• GC consistency verification: Mirror objects are correctly cleaned up when proxy objects are destroyed

Ablation Studies

Partitioning Strategy Comparison

Through synthetic benchmarks testing different proportions of trusted classes:

• CPU-Intensive: Significant performance improvements when moving more classes outside enclave
• I/O-Intensive: More pronounced performance improvements when moving I/O operations outside enclave

Unpartitioned vs. Partitioned vs. SCONE

Performance ranking across all tests:

1. Native execution (no SGX) - Fastest but unsafe
2. Partitioned native image - Balanced performance and security
3. Unpartitioned native image - Moderate performance
4. SCONE+JVM - Slowest but best compatibility

Case Studies

PalDB Partitioning Strategy

@Trusted class DBReader   // Protect sensitive read operations
@Untrusted class DBWriter // Move I/O write operations outside enclave

RTWU scheme performs better because it avoids expensive write operation enclave transitions.

GraphChi Partitioning Strategy

@Trusted class GraphChiEngine    // Protect core computation logic
@Untrusted class FastSharder     // Move graph sharding I/O operations

Performance approaches native execution after moving sharding operations outside enclave.

Related Work

Complete Application Enclave Solutions

• SCONE, Graphene-SGX, SGX-LKL: Provide good compatibility but large TCB
• Haven: Early complete application solution

Framework-Specific Partitioning

• VC3: Hadoop data analytics
• SecureKeeper: ZooKeeper extension
• Opaque: Spark SQL platform
• Limitations: Applicable only to specific frameworks

General-Purpose Partitioning Solutions

• Glamdring: Automatic C/C++ partitioning, not applicable to Java
• Panoply: Micro-container approach
• Civet: Java partitioning but uses complete LibOS
• Uranus: Method-level annotations, requires third-party tools

Conclusions and Discussion

Main Conclusions

1. Feasibility Verification: Annotation-based class-level partitioning approach is feasible and intuitive
2. Performance Advantages: Partitioned applications show significant performance improvements over complete enclave approaches
3. TCB Reduction: Significantly reduces trusted code base through shim library rather than LibOS
4. Developer-Friendly: Annotation approach is more intuitive for developers

Limitations

1. Encapsulation Requirements: Assumes all annotated classes are properly encapsulated (private fields)
2. RMI Overhead: Cross-enclave calls still incur significant overhead, unsuitable for frequent interaction scenarios
3. GraalVM Dependency: Limited to GraalVM ecosystem
4. Static Analysis Limitations: Limited support for certain dynamic features

Future Directions

1. Zero-Copy Calls: Research techniques to reduce expensive RMI overhead
2. Multi-Isolate Support: Extend to multiple isolate environments
3. Automatic Partitioning: Automatic partitioning recommendations based on static analysis
4. Other TEE Support: Extend to AMD SME, ARM TrustZone

In-Depth Evaluation

Strengths

1. Strong Innovation: First class-level Java application partitioning approach with intuitive and reasonable design
2. Complete Engineering: Full toolchain from annotations to final SGX applications
3. Excellent Performance: Demonstrates significant performance improvements in real applications
4. High Practical Value: Solves practical problems of secure Java application deployment in the cloud

Weaknesses

1. Significant Overhead: RMI call overhead remains 3-4 orders of magnitude, limiting applicable scenarios
2. Strict Assumptions: Requirements for class encapsulation may limit applicability to existing code
3. Incomplete Evaluation: Lacks evaluation of large-scale, long-running applications
4. Insufficient Security Analysis: Limited analysis of protection against advanced threats such as side-channel attacks

Impact

1. Academic Contribution: Provides new perspectives for managed language application partitioning in TEE environments
2. Practical Value: Can be directly applied to security hardening of Java cloud applications
3. Reproducibility: Authors commit to open-sourcing, facilitating research reproduction and extension

Applicable Scenarios

1. Cloud Java Services: Cloud applications requiring protection of sensitive business logic
2. Data Analytics Platforms: Analytics frameworks processing sensitive data
3. Financial Technology Applications: Financial services requiring high security guarantees
4. IoT Edge Computing: Secure computing in resource-constrained environments

References

The paper cites 60 related references covering multiple domains including SGX technology, TEE applications, Java security, and compilation optimization, providing a solid theoretical foundation for this research.

Overall Assessment: This is a high-quality systems research paper that addresses practical deployment challenges of Java applications in TEE environments. The methodology is well-designed, the implementation is complete, and the experiments are comprehensive, demonstrating good academic value and practical significance. Although there remains room for improvement in RMI overhead and applicability, it provides important reference for related research.