Authors:
João Sousa, PhD Student at UMINHO
Tiago Gomes, Assistant Professor at UMINHO
In many IoT systems, various devices coexist, presenting an ongoing challenge to ensure they all provide the essential security to establish a security baseline across the whole system. As technology advances, system complexity increases, and the integration of functionalities with different criticality levels within the same device becomes more common, leading to the rise of Mixed-Criticality Systems (MCSs) [1].
To securely integrate MCSs across a spectrum of devices, from low-end microcontrollers (MCUs) with limited resources to high-end processors (APUs) equipped with advanced security primitives, security through separation has become a fundamental principle [2,3]. This approach is widely adopted by microkernels [4], hypervisors [5], and Trusted Execution Environments (TEEs) [6,2]. Among these, TEEs have become a cornerstone in modern security architectures by isolating secure workloads from untrusted software and protecting billions of devices across mobile [7], IoT [8], and industrial applications [9]. However, TEE implementations vary significantly in terms of underlying hardware (e.g., Arm TrustZone [2], Intel SGX [10], and RISC-V PMP [11]), architectures, security models, and design choices. These differences have led to the TEE fragmentation challenges where different vendors implement their own TEEs (e.g., Qualcomm QSEE [12], Huawei TrustedCore [13], and OP-TEE [14]) across distinct architectures (e.g., Arm, Intel, and RISC-V). Additionally, although being developed as a technology to protect secure data and execution from untrusted environments, researchers (such as studies conducted at CROSSCON) have identified multiple types of vulnerabilities in TEE implementations, including those in commercial products [15, 16]. This leads us to two main challenges of the CROSSCON: TEE interoperability and isolation challenges on heterogeneous devices.
Security Challenges in Low-End Devices (MCUs)
On low-end devices, its heterogeneity and limited computational resources pose significant challenges in integrating MCSs and establishing TEEs. IoT system developers often struggle due to the lack of fundamental security mechanisms required to isolate critical applications [17,18]. Fortunately, as mentioned in a previous CROSSCON Blog Post [19], efforts have been made by the IoT community, particularly with the integration of TrustZone-M on Arm Cortex-M devices, which enables software-based TEE solutions like TrustedFirmware-M [20], Kinibi-M [21] and mTower [22]. However, as partially demonstrated by either CROSSCON [23] and by some previous researchers [3], the lessons gathered from the past based on vulnerabilities in TZ-A TEE environments appear to be falling short, as these new systems are trapping into the same pitfalls of the past.
Security Challenges in High-End Devices (APUs)
On high-end devices, CROSSCON [23] and other previous researchers [15] have already revealed several security issues on TEE implementations assisted by Arm TrustZone on Cortex-A processors. These studies reveal continuous issues with commercial implementations, including Qualcomm, Trustonic, Huawei, NVIDIA, and Linaro vendors. These findings highlight the urgent need for the industry to integrate more robust security techniques and mechanisms, such as those proposed in academic research, to ensure the trustworthiness of commercial TEEs. Additionally, TEE software development suffers from severe fragmentation, leading to interoperability and compatibility challenges. This fragmentation presents barriers for integrators due to:
Proprietary TEE implementations (e.g., Qualcomm QSEE, Huawei TrustedCore) that rely on independent designs;
Tightly coupled TEE implementations to specific hardware architectures, which limits TEE portability and flexibility;
Existing solutions that attempt to mitigate TEE heterogeneity lack support for multiple TEE programming models;
CROSSCON Approach: Virtualization as a Solution
To address interoperability challenges while enhancing isolation and security guarantees, CROSSCON leverages (i) widely available hardware virtualization primitives across heterogeneous platforms, including Arm VE [24] and the RISC-V Hypervisor extension [25] for high-end (APU-based) devices and (ii) the dynamic reconfiguration capabilities of TrustZone-M controllers for low-end (MCU-based) devices. At the core of this approach is the CROSSCON Hypervisor, a unifying software layer designed to ensure isolation, interoperability, and other critical security features, all demonstrated on Figure 1 and listed bellow.
Figure 1: Unified security stack model with CROSSCON Hypervisor managing three VMs. VM1 runs a general-purpose OS, while VM2 and VM0 operate as Trusted Execution Environments (TEEs) based on different security models: Intel SGX and Arm TrustZone, respectively.
Key Features of the CROSSCON Hypervisor
MCU-Based Capabilities:
- Operates at the highest privilege level (secure privileged mode) on MCUs and relies on TrustZone-M hardware security primitives;
- Enables the creation of multiple domains in the Untrusted World, supporting not only standard applications but also TEE-based kernel components;
- Extends dual-world TrustZone-M implementations to provide multiple secure execution environments and enhanced TEE capabilities;
- Prevents privilege escalation and lateral movements across virtualized environments.
APU-Based Capabilities:
- Operates at dedicated Hypervisor level on APUs and relies on virtualization primitives of different architectures (e.g., RISC-V Hypervisor Extension and Arm VE);
- Provides per-VM TEE features, enabling the association of multiple Trusted OSes with a single application or OS within a VM. This shifts from a single Trusted OS in the Trusted World to multiple Trusted OS VMs running in the Untrusted World, allowing, for example, two TEE-VMs (each running a Trusted OS) to be linked to a single VM hosting an OS.
- Provides TEE interoperability: The hypervisor can simultaneously run multiple TEE models and host unmodified TEE software binaries (e.g., running SGX enclaves on an Arm platform).
- Provides TEE portability: Secure software stacks can be ported across architectures, including devices lacking native TEE support, e.g., running an Arm TZ-based TEE such as OP-TEE on a RISC-V device.
- Provides TEE configurability: The CROSSCON hypervisor’s flexible architecture enables custom TEE adaptations, including future TEE models with new security primitives.
Conclusion
In an era where IoT systems are increasingly complex and heterogeneous, ensuring security across diverse hardware architectures remains a critical challenge. TEEs have emerged as a fundamental security mechanism, but their fragmentation, interoperability limitations, and recurring implementation flaws highlight the need for a more unified approach. CROSSCON addresses these challenges by leveraging hardware virtualization to provide a flexible, portable, and secure TEE framework across both low-end and high-end devices. By integrating the CROSSCON Hypervisor, the project contributes a standardized, robust, and adaptable security architecture, fostering a future where secure execution environments can seamlessly coexist across heterogeneous platforms. The CROSSCON Hypervisor is available on GitHub, and those interested in exploring its capabilities on different platforms can contact the CROSSCON project team anytime for further details and support.
Bibliography
[1] S. Pinto, H. Araujo, D. Oliveira, J. Martins, and A.Tavares, ‘‘Virtualization on TrustZone-enabled microcontrollers? Voilà!’’ in Proc. IEEE Real-Time Embedded Technol. Appl. Symp. (RTAS), 2019
[2] S. Pinto and N. Santos, ‘‘Demystifying arm TrustZone: A comprehensive survey’’, ACM Comput. Surv, 2019.
[3] D. Oliveira, T. Gomes and S. Pinto, "uTango: An Open-Source TEE for IoT Devices," in IEEE Access, 2022
[4] G. Klein, M. Norrish, T. Sewell, H. Tuch, S. Winwood, K. Elphinstone, G. Heiser, J. Andronick, D. Cock, P. Derrin, D. Elkaduwe, K. Engelhardt, and R. Kolanski, ‘‘SeL4: Formal verification of an OS kernel’’, in Proc. ACM SIGOPS 22nd Symp. Operating Syst. Princ. (SOSP), 2009
[5] J. Martins and S. Pinto. "Shedding Light on Static Partitioning Hypervisors for Arm-based Mixed-Criticality Systems", in Proc RTAS 2023, 2023
[6] W. Li, Y. Xia, L. Lu, H. Chen, and B. Zang, ‘‘TEEv: Virtualizing trusted execution environments on mobile platforms,’’ in Proc. ACM SIGPLAN/SIGOPS Int. Conf. Virtual Execution Environ. (VEE), 2019
[7] M. Busch, P. Mao, and M. Payer. "GlobalConfusion: TrustZone trusted application 0-days by design". In Proc. of USENIX Security, 2024.
[8] T. Ensergueix. "Cortex-M23 and Cortex-M33 - Security foundation for billions of devices", 2016. [online] Available at: https://community.arm.com/arm-community-blogs/b/architectures-and-processors-blog/posts/cortex-m23-and-cortex-m33---security-foundation-for-billions-of-devices.
[9] S. Gupta. "Retracted: An edge-computing based Industrial Gateway for Industry 4.0 using ARM TrustZone technology". In Journal of Industrial Information Integration, 2023.
[10] V. Costan and S. Devadas. “Intel SGX Explained”. IACR Cryptology ePrint Archive, 2016
[11] D. Lee, D. Kohlbrenner, S. Shinde, Krste Asanović, and D. Song. “Keystone: An Open Framework for Architecting Trusted Execution Environments”, In
Proc. of EuroSys, 2020.
[12] F. Khalid and A. Masood. "Vulnerability analysis of Qualcomm Secure Execution Environment (QSEE)", In Computers & Security, 2022.
[13] M. Busch, J. Westphal, and T. Mueller. "Unearthing the TrustedCore: A Critical Review on Huawei’s Trusted Execution Environment". In Proc. of USENIX WOOT, 2020.
[14] TrustedFirmware. "OP-TEE". [online] Available at: https://www.trustedfirmware.org/projects/op-tee/.
[15] D. Cerdeira, N. Santos, P. Fonseca, and S. Pinto. "SoK: Understanding the Prevailing Security Vulnerabilities in TrustZone-assisted TEE Systems.” In Proc. of S&P, 2020.
[16] X. Tan, Z. Ma, S. Pinto, L. Guan, N. Zhang, J. Xu, Z. Lin, H. Hu, and Z. Zhao. "SoK: Where’s the “up”?! A Comprehensive (bottom-up) Study on the Security of Arm Cortex-M Systems”, In Proc. of USENIX WOOT, 2024.
[17] A. Sadeghi, C. Wachsmann, and M. Waidner, ‘‘Security and privacy challenges in industrial Internet of Things,’’ in Proc. Annu. Design Autom. Conf., 2015
[18] D. Oliveira, M. Costa, S. Pinto, and T. Gomes, ‘‘The future of low-end motes in the Internet of Things: A prospective paper,’’ Electronics, 2020.
[19] J. Sousa, “TEE Vulnerabilities? Are you still there?”. [Online] Available at: https://crosscon.eu/blog/tee-vulnerabilities-are-you-still-there
[20] TrustedFirmware. "Trusted Firmware-M Documentation". [online] Available at: https://www.trustedfirmware.org/projects/tf-m/.
[21] Trustonic. "Kinibi-M". [online] Available at: https://www.trustonic.com/opinion/not-just-droning-rise-kinibi-m/
[22] T. Drozdovskyi and O. Moliavko. "mTower: Trusted Execution Environment for MCU-based devices". In Journal of Open Source Software, 2019.
[23] S. Pinto, “D3.1 CROSSCON Open Security Stack Documentation” ‐ Draft. Zenodo, 2023 https://doi.org/10.5281/zenodo.15025181
[24] ARM Architecture Group, “Virtualization Extensions Architecture Specification”, ARM Limited, 2010.
[25] B. Sá, J. Martins,S. Pinto, S. “A first look at RISC-V virtualization from an embedded systems perspective.” IEEE Transactions on Computers, 2021