Computer hardware's role in securing operating systems and hypervisors in trusted computing applications
Many software applications run on “least privilege,” meaning that the software only receives minimal access to the hardware, other applications, and other system resources. A security context separation between an application and other resources like operating systems and hypervisors ensures that less-secure applications and software can’t access critical data from more-secure and critical trusted computing applications.
Highly sensitive data must be protected to ensure that only the code that needs to operate on that data has access to it.
The responsibility of maintaining this type of secure application separation belongs to the operating system and the hypervisor, if one exists on the system. Think of an application that sits on top of a software stack; each lower layer of that stack must do its part to maintain the application layer’s security.
At the bottom of the stack is the hardware, which must be able to enforce the access controls. On top of the hardware is an operating system or a Type-1 hypervisor that manages access controls. Type-2 hypervisors, instead of running on top of the hardware, run on top of an operating system.
For optimal protection, systems designers should configure the system to ensure that the operating system or Type-1 hypervisor exploits all available hardware security to manage scheduling, resources, processes, and security from the next layer. It is very difficult to build a secure application if the foundation is missing those essential security building blocks.
Related: Trusted computing hardware features for maintaining cyber security during operation
First let’s consider the operating system, and how it plays an integral role in ensuring system security on any moderately powerful processor.
For many years now, operating systems have maintained separation between kernel processes and user space applications. In fact, the entire function of an operating system is to ensure the consistent operation of several applications running on one piece of hardware. As part of their responsibility, operating systems have evolved to prevent or limit a malicious actor in one application to influence other concurrently running applications.
As processors have become faster and more efficient, they also have become more complicated, which in turn has made the responsibilities of operating systems even more complex. It's complicated, for example, for an operating system to handle cache management between tasks as processes move in and out of memory; the operating system must flush or invalidate any cached memory, which can be difficult and error-prone.
Add to that challenge factors such as direct memory access (DMA), side effects, and security issues like the row hammer, meltdown, and spectre attacks, and the security responsibilities of the operating system become immense.
Hardware security capabilities
Most processing hardware today includes security capabilities for operating system or hypervisor. That’s because many of these hardware capabilities perform operations in supervisor mode. In processor-based trusted boot resources like Intel SGX or ARM TrustZone, the operating system must create and manage process and resource access to security domains.
Software engineers must design the operating systems and hypervisor to use the hardware's built-in security features. The operating system and hypervisor typically run in privileged mode; they are the only entities able to use all the features of the processing architecture.
Using an old operating system on new hardware can negate the hardware's security capabilities, and updating operating systems in military and aerospace systems can be costly. Systems designers must consider he potential tradeoffs between risk and program costs to determine when to insert new versions of operating systems during system refresh.
The operating system also must manage resource access securely -- in addition to maintaining security boundaries between processes and using hardware security to its full potential; it’s not enough just to maintain process separation. It compromises trusted computing if a process can read another’s data from a peripheral device as the data flows in and out.
Software engineers must design operating system software drivers that access peripherals with security in mind. They must understand potential tradeoffs between increasing the access and availability of I/O resources, as well as maintaining and controlling access separation.
Some processors provide enhanced capabilities for managing I/O, such as an input–output memory management unit (IOMMU), which enables the operating system and software drivers to enhance security while making the most of I/O resources.
Security aspects of hypervisors
A hypervisor manages virtualizing resources to enable several operating systems to operate on the same hardware at the same time. The operating systems and the hypervisor must work together when running in a virtualized environment to maintain a secure and trusted environment.
A set of guest operating systems work in the system stack above the hardware and the Type-1 hypervisor. Each of these guest operating systems works similarly to a single operating system when a hypervisor is not present. The Type-1 hypervisor virtualizes all hardware resources and manages access to all the operating systems running above it in the stack. VmWare ESX and Xen are examples of Type 1 hypervisors.
In contrast, Type-2 hypervisors run on top of another operating system, using that parent operating system to access the hardware resources. The Type-2 hypervisor virtualizes those resources to the guest operating systems that reside above. VmWare Workstation and Oracle VirtualBox are examples of Type 2 hypervisors.
Linux KVM is a hybrid hypervisor, and executes in Linux kernel mode directly on the hardware, yet uses the Linux operating system architecture to manage virtualized resources.
The type of hypervisor used dictates where the base security responsibilities for the guest operating systems reside. The Type-1 hypervisor must use all security features available in the hardware to prevent a compromise in which one guest operating system leaks information or access across another guest operating system.
It's imperative for software engineers to write host or parent operating systems to use available hardware security capabilities where Type-2 hypervisors are concerned; the Type-2 hypervisor uses the host operating system to manage and control access to the virtualized resources.
The operating system -- or set of guest operating systems -- separates user space processes from supervisory processes. Each process that helps operate the system has a defined interface that helps it communicate and interoperate with other processes. The operating system also ensures that processes operate within their defined roles and use interfaces to communicate. For example, it catches and prevents invalid operations and interface access, which prevents one failed process from bringing the entire system to a halt.
Operating system security
Systems designers should give the most stringent security requirements to the operating system or hypervisor when considering the layers of software running on the stack.
The most stringent security must reside at the lowest layer because of the risk of a security failure at that level. Designers should analyze the operating system and hypervisor to prevent bugs that could allow for inappropriate access. If the lowest-level operating system or hypervisor has a bug, failure could compromise all trusted systems that use that operating system or hypervisor. It also could allow for escalation of privilege, which could leverage an application to compromise all other processes running in the operating system or hypervisor.
The risk of failure at the operating system or hypervisor level normally is much more constrained, yet systems designers still must review applications for security. A failure at the application level can compromise one application or a guest operating system, yet the next level below normally should prevent any such compromise from propagating further in the system.
Related: Trusted computing hardware: what you need to know
It’s of great value to talk with the hardware vendor as early as possible in the design cycle to best understand system security requirements. A clear understanding will help designers give adequate weight to security capabilities when evaluating operating systems or hypervisors.
Selecting the most recent operating system or hypervisor with built-in security usually will be the best choice. Even better is keeping potential exploits and security vulnerabilities to a minimum when paring down the chosen operating system or hypervisor’s feature set.
Embedded systems often offer the opportunity to configure the operating system to remove unnecessary features, processes, and libraries. Tailoring the operating system to minimize any potential exploits also is a good security practice.
More information on Curtiss-Wright’s Trusted COTS (TCOTS) program for protecting critical technologies and data in deployed embedded computing systems is online at www.curtisswrightds.com/technologies/trusted-computing.
David Sheets is senior principal security architect at the Curtiss-Wright Corp. Defense Solutions division in Ashburn, Va.
Ready to make a purchase? Search the Military & Aerospace Electronics Buyer's Guide for companies, new products, press releases, and videos
