"We just did not have that on the embedded system": Insights and Challenges for Securing Microcontroller Systems from the Embedded CTF Competitions

Basic Information

• Paper ID: 2503.08053
• Title: "We just did not have that on the embedded system": Insights and Challenges for Securing Microcontroller Systems from the Embedded CTF Competitions
• Authors: Zheyuan Ma, Gaoxiang Liu, Alex Eastman, Kai Kaufman, Md Armanuzzaman, Xi Tan, Katherine Jesse, Robert J. Walls, Ziming Zhao
• Classification: cs.CR (Cryptography and Security)
• Publication Venue/Conference: ACM SIGSAC Conference on Computer and Communications Security (CCS '25)
• Paper Link: https://arxiv.org/abs/2503.08053

Abstract

Microcontroller systems are indispensable in everyday life, powering critical applications such as vehicles, medical devices, and industrial control systems. This research examines the complete lifecycle of secure microcontroller system development from both technical and cognitive perspectives through analysis of team submissions and post-competition interviews from the 2023 and 2024 MITRE Embedded CTF (eCTF) competitions. The study identifies two major categories of challenges: conceptual challenges, including difficulties in transitioning from microprocessor to microcontroller systems and insufficient awareness of microcontroller-specific attacks; and practical challenges, including inadequate support for memory-safe languages like Rust on microcontrollers and weak entropy sources that compromise cryptographic security and key generation. The research provides targeted recommendations for researchers, developers, vendors, and educators.

Research Background and Motivation

1. Research Questions

Microcontroller (MCU) systems are widely deployed in critical infrastructure, yet their secure development faces unique challenges. Existing research primarily focuses on firmware vulnerability analysis, lacking deep understanding of vulnerability root causes, particularly from developer cognition and practice perspectives.

2. Problem Significance

• Widespread Application: Microcontrollers power vehicles, medical devices, industrial control systems, and other critical infrastructure
• Security Vulnerabilities: Lack standard security features such as MMU; common C/assembly programming is prone to memory errors
• Physical Accessibility: More susceptible to hardware attacks such as side-channel attacks and fault injection compared to general-purpose computers

3. Limitations of Existing Approaches

• Closed-Source Barriers: Real firmware is difficult to obtain and analyze
• Single Perspective: Technical analysis only, overlooking developer cognition and decision-making processes
• Lack of Full Lifecycle View: Unable to trace vulnerability evolution from design to implementation

4. Research Motivation

Through the unique lens of eCTF competitions, the research team can:

• Access complete source code, documentation, and build tools
• Combine technical analysis with developer interviews
• Observe security practices of early-career researchers, providing evidence for educational improvements
• Identify systematic and empirical security challenges

Core Contributions

1. Methodological Innovation: Proposes a method for studying microcontroller system security challenges through CTF competitions, combining technical analysis with cognitive perspectives to examine the complete development lifecycle
2. Dual Challenge Classification Framework: Systematically identifies and categorizes conceptual challenges (knowledge gaps) and practical challenges (tool/resource limitations)
3. Empirical Findings:
   • Conceptual challenges: Insufficient application of fundamental security mechanisms such as privilege separation, memory erasure, and stack canaries; platform adaptation difficulties; weak awareness of hardware attack defenses
   • Practical challenges: Inadequate support for memory-safe languages like Rust; lack of high-quality entropy sources
4. Actionable Recommendations: Provides nine specific recommendations for five stakeholder groups (researchers, vendors, educators, developers, compiler maintainers)
5. Data Resources: Analysis of 47 team submissions (20 from 2023, 27 from 2024) with 22 in-depth interviews

Methodology Details

Task Definition

The research objective is to identify and understand challenges in secure microcontroller system development, specifically including:

• Input: eCTF team submissions (source code, documentation, build tools) + participant interview data
• Output: Security challenge classification, root cause analysis, improvement recommendations
• Constraints: Focus on security-prioritized competition environment with early-career developer participants

Research Architecture

1. Submission Analysis

Data Sources:

• 2023: 20 teams using TI TM4C123GXL development board (ARM Cortex-M4F)
• 2024: 27 teams using Analog Devices MAX78000FTHR development board (ARM Cortex-M4 + RISC-V)

Analysis Dimensions:

• Build Tools: Programming languages, compilers, optimization levels, security compilation flags, linker script properties
• Source Code: Track modifications using git diff, inspect inline assembly, memory operations, random number generation, timing-related code
• Disassembly: Verify compiler optimization impacts on security features
• Runtime Analysis: Use debugging tools to verify uncertainties in static analysis

Key Checkpoints:

• Privilege separation (MPU configuration)
• Memory erasure implementation (memset optimization issues)
• Stack canary enablement
• Non-executable stack configuration
• Hardware attack defenses (side-channel, fault injection, physical tampering)
• Entropy source quality

2. Participant Interviews

Sample Characteristics (n=22):

• Educational background: 12 undergraduates, 6 master's students, 4 PhD students
• Security course experience: 8 with no security course background
• Embedded experience: 14 with embedded development experience

Interview Design:

• Semi-structured interviews, duration 42-107 minutes (average 69 minutes)
• Questions derived from recurring issues in submission analysis
• Explore cognition (knowledge, misconceptions) and decision-making (priorities, trade-offs)

Data Analysis:

• Transcribed using Otter AI with manual verification
• Independent open coding by three researchers
• Iterative codebook development: 8 main themes, 40 sub-themes, 278 codes
• Collaborative resolution of coding conflicts without formal reliability testing

Technical Innovation Points

1. Dual-Track Methodology: First to combine large-scale code analysis with in-depth interviews, revealing both "what" and "why"
2. Full Lifecycle Perspective: Trace vulnerability evolution from design documents → source code → binary → developer cognition
3. Ecosystem Analysis Framework: Identify issues not only attributable to developers but also involving compilers, vendors, education, and other stakeholders
4. Reproducibility: Public release of interview materials and codebook (https://github.com/CactiLab/eCTF-User-Study-Material)

Experimental Setup

Dataset

Competition Data:

• 2023 eCTF: Remote keyless entry system (vehicle + key fob firmware)
• 2024 eCTF: Insulin pump system (controller + blood glucose monitor + pump actuator)
• Reference Design: Written in C, meets functional requirements but lacks security features

Threat Model:

• Physical access to devices and communication channels
• Access to source code (excluding keys/flags)
• Software, network, and hardware attacks

Evaluation Metrics

Quantitative Metrics:

• Security mechanism implementation rate (privilege separation, stack canaries, memory erasure, non-executable stack)
• Hardware attack defense rate (side-channel, fault injection, asynchronous tampering)
• Entropy source usage distribution

Qualitative Metrics:

• Interview theme saturation
• Types of cognitive misconceptions
• Decision priority patterns

Comparison Methods

Comparison with existing research:

• Firmware Analysis Research (FirmXRay, Nino et al., Tan et al.): Technical analysis only; this work adds cognitive dimension
• BIBIFI Research: Focuses on microprocessor systems; this work addresses microcontroller-specific challenges
• Rust Adoption Research (Fulton et al., Sharma et al.): This work combines embedded-specific constraints

Implementation Details

• Collaboration of three PhD-level embedded security researchers
• Author team excluded from case study analysis
• IRB exemption approval
• Participant compensation of $50 per person

Experimental Results

Main Results

Conceptual Challenge Statistics

1. Security Mechanism Implementation Rate (Figure 1):

2. Hardware Attack Defense Rate (Figure 2):

• Any defense: 17/47 (36.17%)
• Side-channel defense: 13/17 (76.47%)
• Fault injection defense: 9/17 (52.94%)
• Asynchronous tampering defense: 7/17 (41.18%)

3. Entropy Source Usage (Figure 3):

• 2023: 25% no entropy, 25% hardcoded/defective, 45% single entropy source, 5% mixed entropy sources
• 2024: 22.2% no entropy, 14.8% hardcoded/defective, 55.6% single entropy source, 7.4% mixed entropy sources

Practical Challenge Statistics

Rust Adoption Decline:

• 2023: 5/20 (25%) teams used Rust
• 2024: 2/27 (7.4%) teams used Rust
• Primary reason: 2024 SDK size large; Rust+C mixed compilation exceeded flash memory limits

Ablation Studies

Memory Erasure Compiler Optimization Case

Case T12 (Listing 1):

• Used memset 10 times to erase sensitive data
• Compiler optimization eliminated 5 calls (including AES key erasure)
• Developer interview revealed: completely unaware compiler would optimize

Effective Implementation Cases:

• T20/T15: Used Monocypher library's crypto_wipe (volatile keyword)
• T14/T02: Used Rust zeroize library
• T18: Custom inline function to prevent optimization

Stack Canary Configuration Issues

• Only 2/47 teams enabled compiler flags
• No teams initialized random canary values (defaulted to fixed constants)
• Consistent with real firmware: <0.2% enablement rate (Xi et al. research)

Case Studies

Case 1: Non-Executable Stack Misconception (T18, T13)

Incorrect Implementation:

// T18's linker script
MEMORY {
    FLASH (rx) : ORIGIN = 0x00008000, LENGTH = 0x00038000
    SRAM (rw) : ORIGIN = 0x20000000, LENGTH = 0x00008000  // Only marked rw
}

Issues:

• Only modified ELF header attributes, failed to configure MPU during firmware initialization
• Interview revealed: 21/22 participants believed compiler flags were sufficient

Correct Implementation (4 teams):

1. Enable MPU
2. Configure stack memory region as XN (eXecute Never)
3. Enable the region

Case 2: Rust Unsafe Block Abuse (T11)

Problem: Widespread use of unsafe blocks to call C SDK functions Reason: Incremental development model allowing gradual code migration to Rust Contrast: C18-T08 limited unsafe to low-level hardware interaction layer

Case 3: Entropy Source Combination (T14)

Strategy: Mixed four entropy sources

• RTC and CPU clock drift
• Device-specific seed
• Internal temperature ADC
• Uninitialized SRAM (actually ineffective)

Effect: Even if one source fails, combined seed remains unpredictable

Experimental Findings

Observation 1: Compiler optimization can compromise security state beyond language specifications (e.g., memory erasure)

Observation 2: Knowledge gaps regarding embedded-specific attacks are the primary barrier to defense implementation

Observation 3: Rust adoption factors: familiarity, vendor support, library support, learning curve

Observation 4: Rust users face challenges: no_std compilation, HAL implementation, unsafe management

Observation 5: Automated hardware descriptor transformation to Rust structures can accelerate HAL development, but fidelity and security require further research

Observation 6: Despite limited microcontroller entropy sources, combining multiple available sources effectively enhances randomness robustness

Related Work

CTF Research

• Education-Oriented: Vigna et al. (iCTF framework), Vykopal et al. (CTF as teaching tool)
• Challenge Analysis: Crispin et al. (Defcon CTF experience), Chung et al. (organization pitfalls)
• This Work's Distinction: First to combine submission analysis with interviews, focusing on security development challenges rather than educational effectiveness

Secure Software Development and User Research

• BIBIFI Research (Parker et al., Ruef et al., Votipka et al.): Analyze microprocessor system development, finding most vulnerabilities stem from misconceptions rather than mistakes
• Rust Adoption Research:
  • Fulton et al.: High-level developer perspective, identifying learning curve and library support issues
  • Sharma et al.: Analyze 6000+ embedded Rust projects, revealing insufficient ecosystem support
• This Work's Contribution: Focus on microcontroller-specific constraints, combining technical and cognitive perspectives

Microcontroller System Security

• Defense Techniques: Privilege separation (Kage, ACES, EPOXY), CFI (μRAI, Silhouette, SHERLOC), randomization (fASLR, HARM)
• Firmware Analysis: FirmXRay, Nino et al., Tan et al. (static analysis of real firmware)
• This Work's Uniqueness: First to study developer cognitive challenges rather than solely technical solutions

Conclusions and Discussion

Main Conclusions

1. Ecosystem Responsibility: Secure implementation is a shared responsibility of developers, educators, researchers, and vendors
2. Unique MCU Development Requirements:
   • Deep understanding of platform characteristics (hardware, compiler, toolchain)
   • Addressing compiler opacity and counterintuitive behavior
   • Defending against unique threats from physical access
3. Educational Gaps: Current curricula insufficiently prepare students for embedded-specific challenges
4. Three Major Conceptual Challenges:
   • Insufficient application of fundamental security mechanisms
   • Platform adaptation difficulties
   • Weak awareness of hardware attack defenses
5. Two Major Practical Challenges:
   • Inadequate support for memory-safe languages
   • Lack of high-quality entropy sources

Limitations

1. External Validity:
   • eCTF is a competition environment; gamification elements may influence behavior
   • Participants predominantly students/early-career developers; generalization to experienced industrial settings is limited
   • Authors believe findings represent conservative lower bound of real vulnerabilities
2. Research Scope:
   • Does not cover team collaboration dynamics and competition structure
   • Conceptual/practical classification may have overlaps
3. Data Limitations:
   • Self-reported data may suffer from social desirability bias
   • Relatively small sample size (n=22)
   • Lack of detailed educational background data; educational recommendations are preliminary
4. Threat Model:
   • Competition threat model may not fully reflect all real-world scenarios

Future Directions

1. Educational Research: Systematically review existing embedded security curricula, identify curriculum gaps
2. Experience Comparison: Survey experienced professionals to determine if they face similar challenges
3. Tool Development:
   • Automate privilege separation tools
   • Compiler security optimization verification tools
   • Simplify Rust HAL development tools
4. Vendor Support:
   • Complete Rust SDKs or Rust-C bindings
   • Entropy source transparency and API standardization

In-Depth Evaluation

Strengths

1. Methodological Innovation:
   • First to combine code analysis with in-depth interviews, revealing both "what" and "why"
   • Full lifecycle perspective tracing vulnerability evolution
   • Strong reproducibility (public data and codebook)
2. Empirical Rigor:
   • Complete analysis of 47 team submissions
   • 22 in-depth interviews (average 69 minutes)
   • Triangulation (code, documentation, interviews, disassembly)
   • Qualitative analysis follows mature methods (Saldaña, Clarke & Braun)
3. Practical Value:
   • Nine specific recommendations for five stakeholder groups
   • Identifies systemic barriers (not merely individual mistakes)
   • Findings consistent with real firmware vulnerability rates, validating representativeness
4. Insight Depth:
   • Reveals compiler optimization compromising security (Observation 1)
   • Identifies knowledge gaps as primary barrier to defense implementation (Observation 2)
   • Discovers multi-dimensional Rust adoption challenges (Observations 3-5)
5. Writing Clarity:
   • Structured classification (conceptual vs. practical)
   • Rich cases and code examples
   • Clear figures (Figures 1-3)

Weaknesses

1. Generalization Limitations:
   • Competition environment differs from industrial practice
   • Participant experience level relatively junior
   • Data spans only two years (2023-2024)
2. Causal Inference:
   • Cannot fully separate competition pressure from knowledge gaps
   • Lacks systematic comparison between participants with/without security course backgrounds
3. Quantitative Analysis Depth:
   • Lacks statistical significance testing
   • Does not quantify relative importance of different factors
   • Relatively small interview sample (n=22)
4. Tool Evaluation:
   • Does not empirically test effectiveness of proposed recommendations
   • Lacks intervention experiments validating improvement measures
5. Coverage Scope:
   • Focuses only on ARM Cortex-M platforms
   • Does not address RTOS environments (bare-metal only)
   • Limited exploration of team collaboration factors

Impact

1. Academic Contribution:
   • Establishes new research paradigm for embedded security user studies
   • Provides empirical foundation for educational reform
   • Identifies compiler and toolchain improvement directions
2. Practical Value:
   • Vendors can improve SDKs and documentation
   • Educators can adjust curriculum
   • Developers can avoid common pitfalls
3. Policy Significance:
   • Supports incorporating security into embedded development standards
   • Provides vulnerability root cause analysis for regulators
4. Reproducibility:
   • Public materials facilitate verification and extension
   • Methodology applicable to other CTFs or development competitions

Applicable Scenarios

1. Education:
   • Embedded systems security curriculum design
   • CTF competition organization and evaluation
   • Developer training materials
2. Industry:
   • IoT device security audits
   • Secure Development Lifecycle (SDL) improvements
   • Toolchain selection and configuration
3. Research:
   • Compiler security optimization
   • Embedded adaptation of memory-safe languages
   • Hardware attack defense automation
4. Standards Development:
   • Embedded security best practice guidelines
   • Vendor SDK security requirements

Nine Core Recommendations Summary

Selected References

1. Privilege Separation:
   • 16 Kage (Du et al., 2022)
   • 10 ACES (Clements et al., 2018)
   • 11 EPOXY (Clements et al., 2017)
2. Memory Safety:
   • 18 Rust Adoption Research (Fulton et al., 2021)
   • 52 Embedded Rust Challenges (Sharma et al., 2024)
3. Firmware Analysis:
   • 65 FirmXRay (Wen et al., 2020)
   • 42 IoT Firmware Security (Nino et al., 2024)
   • 56 Cortex-M Systems Survey (Tan et al., 2024)
4. User Research:
   • 46 BIBIFI (Ruef et al., 2016)
   • 62 Developer Security Misconceptions (Votipka et al., 2020)

Overall Assessment: This is a high-quality embedded security user research paper that reveals deep challenges in microcontroller system secure development through an innovative dual-track methodology. Its greatest value lies in combining technical analysis with developer cognition, providing actionable paths for improving education, tools, and practice. Despite generalization limitations, consistency between findings and real firmware vulnerability rates enhances conclusion credibility. This research establishes a new research paradigm for the embedded security community, warranting further verification and extension by subsequent work.