Simulation and mission rehearsal relies on state-of-the-art computing

Simulation has been an integral part of military training since the first humans banded together, brandishing rocks and clubs, to defend against or attack other groups, probably between 12,000 and 15,000 years ago. Smaller, lighter sticks helped young warriors-in-training learn the basics before they tried using real weapons. Metal swords saw the introduction of wooden swords, just as wooden rifles helped recruits become familiar with firearms centuries later.

The 20th century’s great technological advances in weaponry, like airplanes and tanks, led to the introduction of more advanced, technology-based simulation, along with mission rehearsal exercises using combat radios and other devices, real and simulated, giving warfighters the opportunity to use those technologies, often for the first time, in a combat scenario.

Rapid advances throughout the last century, especially in computing speed and capabilities, improved real-time sensor data, significantly improved and individualized communications, long-range intelligence, and surveillance and reconnaissance (ISR) were matched by equally advanced combat simulation and mission rehearsal. Before deployment to Southwest Asia, U.S. troops spent weeks at simulation-supported training bases in California, North Carolina, Louisiana, and other states.

For many, simulation reached its zenith with the six-degree of freedom, full-motion dome flight simulator, which gives new and veteran pilots the feel, visuals and real hardware of a combat aircraft in flight. When linked to others — from desktop simulators to actual aircraft flying over a training range — it offered the most intense and comprehensive ground-based training and mission-rehearsal environments.

There are drawbacks, however. The domes are huge, meaning they have to be housed permanently at specific bases. They also are expensive, complex, and require a high level of maintenance oversight and work — sometimes more than the aircraft they simulate.

In its first two decades, the 21st century already has seen more technological advances — and glimpses of even greater leaps still in development — than any other period in history. Imperatives for rapid development and deployment is moving those from lab to battlefield seemingly overnight. Training has required cutting edge technologies to ensure warfighters not only receive training on new weapons and technologies in the continental U.S., but also training that is mobile enough to accompany them on deployment.

“The first headsets came out eight years ago, but in the past year have reached a level of resolution where they can seriously replace the visual system on many trainers,” says Phil Perey, head of technology at CAE Defence & Security in Saint-Laurent, Quebec. “That makes them much more compact and enables training in the field you couldn’t do with a big trainer on six motion jacks. Another advance has been data analytics and machine learning, which are revolutionizing much deeper insights into how students improve their training performance. By analyzing hundreds of maneuvers, it can help instructors identify underlying weaknesses that may not have been identified before.

Cloud computing

an environment for commanders and a ‘what if’ scenario, giving them access to make the best and most informed decisions.”

Fortunately, as many older officers and senior enlisted military personnel have noted, today’s young warfighters grew up surrounded by constantly changing and advancing technologies, making them far more adaptable to and acceptable of new systems than might otherwise be the case. Nonetheless, they still must be trained with simulators and exercise their new skills with cutting-edge mission rehearsal.

Even for a tech-savvy generation, however, rapid advances in simulation and mission rehearsal can overwhelm student warfighters, leading to another set of requirements and cutting-edge developments, part high-tech, part advanced medical understanding of the human brain and sensory organs.

The next five years will see major advances in simulation and mission rehearsal, just short of “plugging a cord right into your brain,” says Peter Fedak, contractor of flying training and support site manager for KF Aerospace in Kelowna, British Columbia.

“The visuals and processing power of the equipment is reaching the point where the human eye and brain can’t absorb a lot more,” Fedak says. “How do you process terabytes of data being generated by each student and compare that to the global database, then model the DNA? The training tools are more than adequate to do that, but now it is about adapting that, aligning it to make sure each student is getting back the information they need when they need it,” he says.

“Whether it is 4k or 8k visuals, the eye can’t pick it all up, anyway, so the real question is how you work with the tools you have now,” Fedak continues. “We’re looking at eye-tracking and emotional tracking, where the cameras watch your face and determine when you are stressed and use analytics to determine what is causing that stress. It’s more looking at the human aspect, how the human interfaces with all the technology.”

Advanced weapons, especially semi- or fully autonomous, are likely to have two major influences on humans in uniform — fewer people will be necessary, especially in the battlespace, and those who remain will have to receive intense training unprecedented in military history. As the military becomes more and more reliant on AI-driven simulations, with increasingly realistic sounds and visuals, systems designers must take care to ensure that these devices do not give trainees a false impression of reality.

Simulation across the military

Simulation and mission rehearsal are not limited to weapons and platforms, but cover the entire spectrum of military activities — communications, logistics, tactical data links, and battlefield medicine — all of which must be part of any realistic training environment. Those at the heart of bringing emerging technologies out of the lab and onto the battlefield often disagree.

“AI is not ready for this field yet; it still has a long way to go before it becomes intelligent. There’s no room for guesswork when talking about tactical data links,” says Brian Bass, director of operational support at the Curtiss-Wright Corp. Tactical Communications Group segment in Tewksbury, Mass. Nonetheless, he says he sees a need and a future for AI as the technology matures. “In many of the scenarios in which we participate, our adversaries are cuffed — held to what we know about them. But they could do something different in reality.

“In simulation and mission rehearsal, it would be better to train our troops to face the unexpected,” Bass continues. “Advanced systems could come up with new possibilities in a shorter period of time — and hopefully at lower cost. AI will multiply the speed at which those things happen, creating a lot of scenarios we might not think of so quickly.”

Military radios most likely will play a big role in future simulation and mission rehearsal. “The primary goal of tactical data links is giving anybody with a tactical radio the ability to receive the information they need, in the format they expect, in near-real time,” Bass says. “We would schedule advances in simulations in closer proximity instead of spacing them out. If we need to stress a system or operator, we can transmit a future event at a different time. We currently do not have any AI tool to assist the operator in making those decisions.”

Growing importance

Cutting-edge simulation and mission rehearsal are of growing importance to all of the nation’s military services, including its joint commands, such as the Special Operations Command (SOCOM). SOCOM representatives highlighted the application of emerging technologies to unique SOF mission training sets at the annual Interservice/Industry Training, Simulation and Education Conference (I/ITSEC) in Orlando, Fla., in December 2019.

“SOCOM is focused on addressing defense and security threats and challenges from emerging great power competitors as well as terrorists and violent extremist organizations, as has been identified in the 2018 National Defense Strategy,” Army Maj. Gen. Robert Karmazin, Director, J-7/9, Directorate of Training, Doctrine and Capability Development, told a presentation of the SOF Simulation Technologies Capability Assessment Event.

“To help enable our objectives, SOCOM needs to expand the use of transformative technologies. For mission preparedness, virtual reality, augmented reality, mixed reality, AI machine learning capabilities are absolutely paramount.”

Randy Jackson, chief of mission preparation, J3 Training and Education, explained the extent of emerging technologies on SOCOM’s overall training, mission rehearsal and operational future.

“The use of advanced technologies should help prepare SOF for what lies ahead. They will increase cognition, optimize human materiel performance, reduce operational risk and better enable SOF adaptation for a variety of situations,” he said. “We must harness these ideas, leverage capability, share information, capture the good, smell the bad, navigate data, transfer knowledge, fuse networks, bolster cyber security and increase interoperability — again to reduce risk, save time, increase cognitive learning and truly change the way we do business.”

Throughout most of the 20th century, cutting-edge and transformative technologies were driven by the government, which had specific requirements needed to keep the United States in the global technology lead, a position it assumed at the end of World War II. But increasingly higher research costs to meet increasingly small, military-specific needs made it more and more difficult for industry to recover costs from the Pentagon alone.

Commercial technology

Commercial developers brought in more academic researchers to provide new ideas and speed the development process, leading to new generations of technology evolving far faster than traditional U.S. Department of Defense (DOD) acquisition processes could follow. By 2020, cutting-edge and emerging technologies aimed at simulation and mission rehearsal had even larger customer bases in the gaming, automotive, communications, and other civilian areas than DOD could muster.

graphics specialist Vricon in McLean, Va. “The government side has some of the very large service models looking at how training and mission rehearsal are conducted, focusing on leveraging commercial technologies to provide timelines that allow soldiers to train more frequently and connect the dots between synthetic environments and simulation to train like they fight. That has traditionally been cost prohibitive or the tech just wasn’t there.

“With advances in commercial technology and remote sensing in the geospatial community, you can achieve that mission today. You have a confluence of advances with what is coming out of the computing world — large scale processing, cloud computing, gaming world 3D rendering, on top of the image generators,” Zaworski says. “That combination is driving a massive amount of innovation in a very short time. On the academic side, you have organizations such as the University of Southern California bridging the gap between government and industry on technologies being developed for mission rehearsal, for example. Those require real-world accuracy unique to that community.”

Vricon is working with the Army on elements of the Synthetic Training Environment (STE), a 3D training and mission-rehearsal tool combining live, virtual, constructive, and gaming environments for soldier and unit readiness. Vricon’s focus is on STE’s One World Terrain (OWT) component, an accessible 3D representation of the global operating environment.

In its winning proposal, Vricon officials said rather than trying to bring together incomplete and misaligned data sets from different sources, their company would create a foundation built from high-resolution 3D data already collected and establish a single data standard as the interface for OWT. The goal, Zaworski says, is to make it through a successful operational test at STE initial operating capability, scheduled for September 2021.

Tough definitions

Many experts have a hard time defining the current state of the art in simulation and mission rehearsal or forecasting what will be emerging from research labs in the near-term, much less in 10 or 20 years. They point to programs such as the Army’s Integrated Visual Augmentation System (IVAS) as driving the bleeding edge of commercial augmented reality and the military’s synthetic training environment into military modeling and simulation problem sets.

Vricon’s Zaworski. “When it comes to simulation and mission rehearsal, it’s all about being able to bring, in a very agile way, these different segments together at program levels to implement capabilities that can be scaled largely across very large, service-level problem sets.

“Even five years ago, the mission-rehearsal world was largely relegated to the very tip of the spear, with tightly integrated, often heavily proprietary systems supporting that use case,” Zaworski continues. “Now there is a recognition that to scale these technologies, government needs to partner with industry for structure and implementation.”

Advancing technologies have changed the nature of simulation for training and mission rehearsal significantly, moving from large platforms and projection screens to virtual reality and augmented reality headsets. The key, however, is creating ever-more realistic visuals — without the nausea and disorientation of earlier efforts. Gone forever are displays where green cones represented trees, although Star Trek-like holograms, while perhaps the ultimate desirable goal, remain unlikely.

Advanced image generators

A high-fidelity, immersive simulated environment happens by using advanced image generator systems and high-density databases to create the appearance of a live-action picture in the trainee’s helmet-mounted display. Adding 3D sound, haptic feedback, and full-body sensation via combat training suits make the virtual world more realistic, but emerging technologies are expected to advance training and mission rehearsal through the next decade well beyond the state of the art of the last decade.

“Users continue to look for better image quality and more realism,” says Andrew Fernie, senior technical fellow at CAE Defence & Security. “Each person evaluating a visual system makes a comparison, consciously or unconsciously, against what they see with their own eyes in the real world. And no matter how far industry has come, we still have work to do to satisfy that test of making the synthetic environment more like the real world.”

“Improvements in resolution, density of the content and the fidelity of special effects will contribute to meeting expectations — and will require work on all aspects of the visual system: database content, image generator and display system. At the same time, however, we are expected to reduce the life cycle costs associated with the visual system,” Fernie says.

Data collection has grown dramatically in the past five years, from military unmanned aerial vehicles (UAVs) to civilian self-driving cars equipped with cameras and other sensors. In the next five years, a lot of effort will go into developing ways to link that raw data, in near-real time, into a reference frame of the 3D world in which we live, organizing it to power the simulation component of simulation training and mission rehearsal.

“Simulation requires a whole lot of knowledge about the digital world, very early constructions of transportation systems, the interiors of buildings and how we as people interact with those structures,” says Vricon’s Zaworski. “How do you use modern computer technology and machine learning to take satellite and drone imagery and rapidly and accurately correlate all that in near-real time? In the next five years, we will see a greater ability to build out the interior and underground elements.

“There also will be massive progress on the end user hardware side — integrating visual systems to allow people to interact with this data in ways we’ve never been able to before,” Zaworski continues. “You will see a host of other applications start to emerge, where we take the incredible investment in the commercial market into virtual reality and mixed reality headsets and impact the modeling and simulation world to leverage the latest and greatest of hardware tech with the core data underneath that is a living, breathing version of the planet, at human scale, to support mission rehearsal.”

Cyber security

As new and more advanced commercial technologies adapt to military use — especially making deployed simulation and mission rehearsal more aligned to the real world in near-real time — equally significant advances in hacking and code-breaking technologies raise cyber security issues to new highs. AI and quantum computing for simulation training and mission rehearsal close to or within a battlespace would provide an enemy with invaluable up-to-date information of U.S. capabilities, and logistics.

“The closer you push for mission rehearsal and high fidelity, the higher the need for protection. To practice team training, you need to share classified information over a network, so it is important the construction, the cyber security access, is very high so you don’t compromise the technology or the system,” says Lenny Genna, president of the military training sector at L-3 Harris Technologies in Arlington, Texas.

With the quality and affordability of virtual, augmented and mixed reality technologies essential to military readiness, Lockheed Martin Corp. has identified three current and evolving technologies for simulation training and mission rehearsal. According to the company:

Evolving technologies

1 — augmented reality and on-demand training

Warfighters need mission rehearsal tech at their fingertips. Simulation combined with augmented reality glasses provide a new tool to conduct maintenance on critical warfighting platforms at the point of need.

2 — virtual reality and multi-domain training

In the multi-domain battlespace, systems are connected and integrated. F-35 Lightning II simulators network with other systems to provide a train-as-you-fight virtual environment. The vision is to integrate simulators on the ground with aircraft flying and virtual friendly or foe forces to make the training environment realistic to the max.

3 — mixed reality and turn-key training

Military training is about people, not just tech. Traditional approaches focus on delivering training tools and devices, but today’s militaries are taking a more innovative approach to provide the shortest path to learning. Turn-key training programs provide every element needed to prepare military personnel in one integrated system, including simulators and aircraft, resulting in increased skill sets, shorter training times and lower training costs.

Simulation training and deployable mission rehearsal are becoming increasingly important and valuable parts of military readiness, enabling effective training on difficult and complex scenarios, especially when using new and advanced equipment.

“The flexibility of procedural training systems allows students to become proficient in a wider range of emergency and hazardous scenarios that would be impractical or dangerous to train under real conditions. We believe the use of simulation training will become more and more central to the training of military personnel and, as technology continues to evolve, the gap in fidelity between the simulation and reality will continue to close,” predicts Ian Cox, head of training and simulation at Systems Engineering & Assessment Ltd. (SEA) in Beckington, England.