Real-time software in embedded computing, like real-time operating systems (RTOS) in military and aviation applications, must work quickly and with no errors. On top of reliability, experts in real-time technology say it must have robust information security for critical and classified data.
“The intrinsic value of an RTOS is to provide guaranteed processing performance,” explains Richard Jaenicke, director of marketing for Green Hills Software in Santa Barbara, Calif. “For sensor signal processing, that can be used to make sure the data is processed in real time so that none are dropped. To achieve that level of determinism, an RTOS generally is smaller and faster than a basic OS, consuming fewer resources. Consuming fewer resources is a particular advantage when running on processor cores embedded in an FPGA [Field Programmable Gate Array]. Finally, many RTOS are safety-certifiable. The safest RTOS are based on a separation kernel, which isolates applications from each other and from the kernel so that a fault in one cannot affect the others. When a particular function requires the absolute lowest latency, it can be included in the kernel and still retain the benefits of isolation from the other applications.”
Abaco Systems in Huntsville, Ala., which produces rugged embedded computing solutions for defense, aerospace, and industry, use real-time software to carry out signal processing.
“In terms of the underlying hardware, FPGA-based systems deliver a perfect solution for rapid signal processing,” says Abaco software product manager Francesco Fiaschi. “No less important than speed in sophisticated signal processing applications is determinism — and the challenge is often how to deliver information in a deterministic manner from the point of acquisition in the FPGA to the destination application. This is normally a function of the operating system. Choosing the appropriate real time operating system — such as VxWorks, Integrity, LynxOS or any Linux real time extension — will provide predictability in the way data from the source can be delivered to the user in the most expeditious and reliable manner. When an application is mission critical, an RTOS can deliver both predictability and reliability.”
Ray Petty, vice president of global aerospace and defense at Wind River Systems in Alameda, Calif., says that embedded systems control technologies are used in our daily lives from phones to airplanes.
“Embedded systems typically comprise both hardware and software,” Petty says. “The hardware consists of tiny components, like microcontrollers or microprocessors. The software is usually a real-time operating system, such as VxWorks, that performs dedicated tasks and is designed to control time-dependent applications and components in a consistent and predictable manner. RTOS form the foundation of an embedded system.”
Petty says that “an RTOS must respond in a fully predictable manner to unpredictable external occurrences. If it does not, in many cases it can impart severe and irreversible repercussions. Timing errors, for example, in a car’s anti-lock braking system, or in an airplane’s autopilot system, can threaten or even result in the loss of life.”
Petty continues, “RTOS are created to be tough and rigid but, at the same time, very flexible. A highly complex genre of software, an RTOS is based on highly intricate inter-task communications and multitasking technologies. An RTOS’s effectiveness is decided not only by the correctness of a job’s results, but also by the time in which the results are produced. Real-time computing occurs when the system acts in a predictable manner, but also within an exact delivery time. RTOS are designed to handle the execution of applications within extremely rigid response times—sometimes involving microseconds and nanoseconds. This timing factor, of course, separates RTOS from general-purpose operating systems. RTOS absolutely cannot perform sluggishly. They must meet system-timing constraints to ensure predictability and accuracy. They must react in a timely, fully predictable way to unpredictable external conditions as they arrive. They must eliminate risk from extreme load conditions.”
Open standards
For years, the U.S. Department of Defense (DOD) and its military branches have been driving adoption of open standards to make systems compatible and
upgradable.
“It’s also worth noting that rapid signal processing on CPU-based systems is facilitated by optimized signal processing and math libraries that take advantage of specific features of the target CPU architecture,” Abaco’s Fiaschi says. “Well-optimized, open-source offerings that support embedded real-time operating systems are not commonplace, neither are they tailored to the needs of mil-aero applications. It is important to ensure that suitable libraries meeting the needs of the signal processing application are available for the RTOS of choice.”
Green Hills’s Jaenicke concurs and says the move to open standards “has been happening for a while but has accelerated due to the tri-service mandate to use a modular open systems approach (MOSA). For RTOS in military applications, the most important open standard is the Future Airborne Compute Environment (FACE) technical standard. One of the many unique qualities of the FACE approach is that it has a domain-specific data modeling mechanism that expedites the integration of multiple open standards for the same system. That is important because, as the saying goes, the wonderful thing about standards is that there are so many to choose from.”
Virtualization
Real-time experts note that hypervisors — a system made of software or hardware that runs virtual machines — are making a mark in this sector. Ian Ferguson, vice president of marketing and strategic alliances at Lynx Software Technologies in San Jose, Calif., says that hypervisors are seeing increased usage in mixed-criticality systems.
“Separating out resources that are doing video processes from other resources that are doing time sensitive stuff around GPS networks,” Ferguson says. “Increased use of hypervisors into those elements — that helps partition parts of the software that you have to take through certification and prove that you can isolate that from the other pieces of the system that is running on Linux typically.”
Abaco’s Fiaschi says that major RTOS vendors are now offering virtualized hypervisors in a single platform.
“This very much reflects the need in the mil-aero market to deploy encapsulated applications that might, at some point, also be part of a certifiable software solution. Products like the open source KVM, Helix from Wind River, INTEGRITY-178 from Green Hills Software and MOSA.ic from Lynx are well suited for this purpose, because they embed a bare metal hypervisor to abstract the hardware horizontally, while also offering guest operating systems as a vertical solution in separate containers. This architecture serves multiple goals. First, it uses most of the hardware capabilities in the single board computer to use and efficiently share hardware resources. The multicore capabilities that today’s SBCs are able to offer adapt well to this hypervisor abstraction,” says Fiaschi. “Beyond this, dedicated hardware functionality at the CPU and at the device level, such as SR-IOV, allow hardware design to function efficiently in a virtualized context.”
Fiaschi notes that hypervisors allow the simple migration of existing software applications.
“A hypervisor — sitting on top of the hardware — provides the possibility of encapsulating the application and operating system in a single container,” Fiaschi says. “This means that porting the application from one operating system environment to another can be achieved much more easily.”
Finally, Fiaschi says that the architecture allows those applications to coexist with other previously-certified software components without compromising system integrity and responsiveness.
As mentioned by Fiaschi, Green Hills Software’s INTEGRITY-178 RTOS is suitable for use as a hypervisor.
“The INTEGRITY-178 tuMP RTOS from Green Hills Software supports ARM processors on FPGAs, such as the Zynq UltraScale+ MPSoC, and heterogeneous ARM processor combinations, such as the Cortex-A72 and Cortex-A53 core in the NXP i.MX 8QuadMax,” says Green Hills’s Jaenicke. “The combined solutions are suitable for high-assurance sensor and image processing applications, such as synthetic vision for degraded visual environments. All INTEGRITY-178 tuMP deployments are capable of simultaneously meeting the safety requirements of DO-178C to DAL A and the security requirements in support of NSA ‘high robustness’ and Common Criteria EAL6+.”
Jaenicke explains that in the avionics market, customers are deploying synthetic vision systems that fuse data from radar and infrared imagers to enable landing in a degraded visual environment (DVE).
“Future systems will use an array of multi-spectral, multi-function sensors to enable DVE terrain flight as well as landing,” Jaenicke says. “Both of those systems can require flight safety certification up to the highest design assurance level (DAL A). The INTEGRITY-178 tuMP RTOS provides the architecture and tools to run multitasking fusion algorithms across multiple cores while achieving DAL A.”
‘Soft’ real-time
“Another trend we are seeing in embedded signal processing applications is a transition from the traditional real time operating systems widely used on the PowerPC architecture to Linux running on the Intel architecture,” Abaco’s Fiaschi says. “Linux on Intel, if well-tuned, can provide ‘soft real-time’ performance that is sufficient for many signal processing applications. For applications where hard real-time performance is required, more specialized commercial versions of Linux are being utilized on Intel platforms. These have modified micro-kernels with specific features to provide the real-time determinism needed for the application.
Lynx’s Ferguson says that increases in Linux usage stems from an abundance of code and the need to certify systems.
“One of the things that we are seeing is, rather than having everything on a real-time operating system, we’re seeing the use of mixed-criticality systems,” Ferguson says. “Increased use of Linux because there’s networking stacks that exist. So rather than migrating everything onto an RTOS and having a single point of failure there, the first bit is partitioning the system so that you got only the real certification stuff that you absolutely need on the RTOS — offload other bits onto other operating systems and run them there.”
Optimizing processing
“Our customers request middleware to help them optimize signal processing applications running on real-time systems,” Abaco’s Fiaschi says. “They also need
middleware to move data efficiently between processing entities within multicore systems, as well as tools to help characterize and tune performance. Abaco is meeting these requests via our AXIS software environment as well as additional toolkits that provide optimized middleware for signal and image processing, system health monitoring, data movement and graphics. This is complemented by graphical tools to enhance productivity to minimize time to deployment.”
Fiaschi continues, “To meet the significant demands of system responsiveness and application partitioning, customers need to look even deeper than the software ecosystem and into how data streaming is controlled by the underlying hardware components. For example, the SR-IOV feature in an Intel architecture single board computer opens up the benefits of a fully virtualized platform — but it’s not a feature of all Intel-based platforms. The 3U VPX SBC3511 single board computer from Abaco not only guarantees SR-IOV capabilities in the data plane because it leverages the XL710 chipset, but also provides support for hypervisors such as those featured by KVM, Helix and MOSA.ic.”
Vector instructions
High-intensity data processing can be aided by using vectors to store a large number of variables. Lynx’s Ferguson explains that mil-aero experts are now taking full advantage of vector processing with custom instructions.
“If you look at these newer Intel Xeons AVX512 their accelerator extensions,” says Ferguson. “There are custom instructions on ARM; there’s custom instructions on PowerPC. We certainly see people taking more advantage over those instructions to get the performance up without scaling up frequency, staying within power budgets and correlated to that really is increases to FPGAs. We see people doing more encryption and crypto and bit manipulation inside the FPGAs, also video encoding subsystems.”
Ferguson continues, “Silicon partners have actually created Operating Systems and software specs that take full advantage of that. Typically, open source running on Linux coming into the aerospace and defense area and those people really segment and isolate as minimal amount of code onto the RTOS as possible.”
Green Hills’s Jaenicke says that the industry is seeing an “explosion” of demand for RTOS on ARM-based processors, which he says is driven in part by the ARM cores embedded in FPGAs.
“Whereas many military signal processing systems, such as radar signal processing, traditionally divide the processing between FPGAs in the front end and Intel or Power Architecture processors in the back end, the dividing line is shifting to do more in the FPGAs,” Jaenicke says. “Even if all of the signal processing is done in the FPGA fabric, the processor cores are still used for the control functions.
Security and certification
Wind River’s Petty notes that customers are asking for support in new development, security, and operations (DevSecOps) requirements.
“Legacy software acquisition and development practices in A&D do not provide the agility to deploy new software ‘at the speed of operations,’” Petty says. “In addition, security is often an afterthought, not built in from the beginning of the life cycle of the application and underlying infrastructure. DevSecOps is the industry best practice for rapid, secure software development ... The main characteristic of DevSecOps is to automate, monitor, and apply security at all phases of the software life cycle: plan, develop, build, test, release, deliver, deploy, operate, and monitor. In DevSecOps, testing and security are shifted to the left through automated unit, functional, integration, and security testing — this is a key DevSecOps differentiator since security and functional capabilities are tested and built simultaneously.”
Petty says that benefits from adopting DevSecOps include reduced mean-time to production; increased development frequency; fully automated risk characterization; and software patching at “the speed of operations.”
Green Hills’s Jaenicke also says that industry is seeing an increase in the need for security in RTOS in the mil-aero sector.
“With our Wind River Simics offering, teams can simulate anything, chip to system, giving them a path to DevSecOps and Continuous Integration and Deployment,” Petty says. “Also, a number of Wind River offerings are FACE conformant, such as VxWorks 653, Wind River Helix Virtualization Platform, and Wind River Linux. In fact, Wind River is the first and only to achieve FACE Conformance for Linux.”
“(This is) driven by the rise in identified vulnerabilities over the last few years,” Jaenicke explains. “Security has finally become a serious enough concern that programs are requiring security instead of giving waivers when the system integrator tells them how much it will cost.”
Lynx Software Technologies’ Ferguson says that certifying multi-core processors for cockpit avionics is difficult because the systems weren’t designed with that task in mind.
“They’re designed for servers, they’re designed for base stations, they’re designed for whatever other workloads in video technology isn’t designed with Lockheed as their primary customer focus,” Ferguson says of multi-core processors. “Certification and how it works around the current flavor around multi-core products are still a big challenge. How do you guarantee determinism on certain things? What happens when you have memory systems that have unpredictable access times and those pieces? There are people that have claimed to have solved multi-core processors for avionics, we are in the camp that thinks it isn’t solved yet. There are things you can do to mitigate it but I think there’s going to need to be more work done into the underlying hardware to get to a place where software can help partner up with hardware to deliver...where the FAA can truly feel comfortable that a multicore system can be certified for all eventualities.”
Jamie Whitney
Jamie Whitney joined the staff of Military & Aerospace Electronics and Intelligent Aerospace. He brings seven years of print newspaper experience to the aerospace and defense electronics industry.
Whitney oversees editorial content for the Intelligent Aerospace Website, as well as produce news and features for Military & Aerospace Electronics, attend industry events, produce Webcasts, oversee print production of Military & Aerospace Electronics, and expand the Intelligent Aerospace and Military & Aerospace Electronics franchises with new and innovative content.
