The increasing role of COTS in high-fidelity simulation

The best commercial off-the-shelf (COTS) computer, display, and networking technology that industry has to offer is playing a pivotal role in today’s cutting-edge simulation, training, and mission-rehearsal systems.

By J.R. Wilson

Each new generation of warfighter grows up with the latest commercial technology, which the military increasingly adopts to enhance mission readiness, capabilities, and even lethality.

The first military personnel to use mechanical, non-fieldable devices for training were aircraft pilots. From crude, stationary and often not especially realistic small boxes in which pilot trainees learned the location and function of primary controls, dials, and gauges to the full-motion, high-fidelity flight trainers of the 1990s took about half a century, and had to overcome contentions of veteran pilots that the only way to learn to fly aircraft was to fly the actual aircraft.

As fighters, bombers, helicopters, and other military aircraft became more complex, however, the aviation elements of the armed services came to rely ever-more on huge computer- and power-hungry flight simulators. But they were too large, complex and expensive to send forward for mission rehearsal, which still relied on limited numbers of expensive real aircraft flying in controlled airspace that rarely bore much resemblance to where the mission would be flown.

In the 21st Century, civilian gaming industry advances in virtual and augmented reality (VR and AR), increasingly smaller and more powerful computers, highly advanced displays and graphics (including helmet-mounted displays and goggles) are making it possible to conduct very realistic mission rehearsal and training aboard ships or at military bases anywhere in the world. The same is becoming true for operating and maintaining battle tanks, surface warships and other military platforms.

Generational resistance also has dropped sharply, as today’s senior officers and noncoms were the first generation of tech-savvy junior officers and enlisted personnel in the 1990s and early 2000s.

Simulation, training, and mission rehearsal “all are all threads of the same thing — building proficiency in people. My team provides ready forces for the fleet, men and women who can do the jobs. We utilize various simulation technologies to create environments for training that are a collection of activities that allow them to gain skills,” says Capt. Erik Etz, commanding officer of the U.S. Naval Air Warfare Center Training Systems Division (NAWCTSD) in Orlando, Fla.

Regimen of instruction

“It’s usually not just one device that satisfies that need, but a regimen from classroom instruction to full mission simulators. Most of our Navy and Marine aviators need far more complex skills to operate our aircraft and weapons systems than they did just a few years ago. We can’t fly more to train on those, based on funding or life of the airframe, so the game-changer is to provide more capability in the simulator to rehearse those skills over and over, so when you do get flight time, you aren’t fumbling with the basics but dealing at the high end.”

That also is true for the U.S. Army, notes Matt Clarke, Director of the Army Simulation & Training Technology Center (STTC) in Orlando, Fla.

“Internally, we have worked on the Augmented Reality Sandtable (AReS) ... [which] is more mission command, the ability to project. We have interactive terrain projected onto sand, which students can shape based on digital inputs from the terrain program for height, vegetation, etc. They can manipulate it to represent a lake or river. You can emulate forces or tactical units via symbols or actual model tanks, in a 2D or 3D rendering,” Clarke says.

“We are integrating the AR part of that and looking at future capabilities to get the 3D representation and the immersion piece and make it even more distributed than it is today. 3D is critical to visually represent autonomous vehicles — ground or air — and their field-of-view or give a 3D representation of what an artillery fire solution might be, along with the electromagnetic [EM] spectrum.”

Among the military contractors working to advance the technologies, capabilities and size, weight and power (SWaP) aspects of current and future simulators are CAE Defense & Security in Tampa, Fla., and L-3 Link Training and Simulation in Arlington, Texas.

An unidentified L3 Link customer trains using an F-16 in a high-fidelity tactical environment.

CAE is a global training systems integrator whose programs include the Medium Support Helicopter Aircrew Training Facility (MSHATF) in the United Kingdom; the Brunei Multi-Purpose Training Centre (MPTC); the Operational Training Systems Provider (OTSP) for Canada’s fleets of CC-130J and CH-147 aircraft; a simulation-based Naval Warfare Training System for the Swedish navy; Helicopter Flight Training Services for Germany’s NH90 helicopter program; synthetic training for AgustaWestland helicopters, including the AW109, AW139, AW169, AW189 and naval variant of the NH90; a Tampa-based training center for U.S. Coast Guard and Reserve (and 20 international militaries) C-130H pilots, flight engineers, load masters, and maintenance technicians; Royal Australian Air Force (RAAF) C-130J aircrew training and the world’s largest civil pilot training network, with more than 65 centers around the globe training more than 120,000 crew members annually.

“Our programs are almost all integrated, with elements of live, virtual, and constructive training ... that leverage simulation-based technologies to be more effective and efficient,” says CAE Group President Gene Colabatistto. “There are two sides to this — provision of the systems and actually running the training, which is what is becoming what the company is all about.

Enabling technologies

“There are three classes of enabling technologies involved, the first of which is a surprise we would not have predicted just five years ago. Defense organizations around the world want to invest more in the immersive nature of the simulator itself, with more and more visual acuity, which is both a hardware issue, with screens delivering higher and higher resolution, and to make sure the visual databases have both natural and manmade features at increasing levels of fidelity. Our customers continue to demand higher levels of fidelity in the visual part of the system – they want it to feel immersive. The second is end-things, like cockpit geometry and the functioning of the simulator itself. Five years ago, there was no switch, knob or control in the simulator that did not do what it actually does in the aircraft. Today, that has become a requirement,” Calabatistto says.

“What’s new is in this environment military aircraft do not fly alone, so when we build simulation systems, realistic training demands we have a lot of training entities in that sim, so you see other planes out the window. Those may be a pilot flying a real plane generated on the simulator screen or another simulator or a computer-generated aircraft. That is a requirement — more than 98 percent of all our simulators around the world are on networks and sharing data with other simulators. Distributed mission operations are no longer nice to have, but very much part of the training baseline.”

The CAE 700MR series flight training device is designed for military helicopter flight and mission training, and features an extreme field-of-view visual system to provide a realistic and immersive training environment.

Networking, in turn, has increased cyber security requirements, which are now key on all new systems.

“The third thing is we still train military pilots much as we have for many years, with a student pilot receiving instruction from a more experienced pilot, which will differ from student to student. One of the challenges is to enforce standards so the instructor pilot has tools to help the student,” Colabatistto continues.

The integration of big data analytics, using artificial intelligence (AI), into the training domain will take all the data coming off the simulator or live aircraft and give the instructor better information to evaluate the student pilot. The Air Force is talking about its new training paradigm that will result in a much more tailored training system that should move students through more quickly and reduce attrition.

With its acquisition of Link Simulation & Training and Thales Training & Simulation’s civil aircraft simulation and training business, both in 2012, and Raytheon Training Devices and Training Services in 2000 (among numerous other acquisitions since the turn of the century), L-3 Link can trace its role in military simulators to before World War II. Today its systems include the F-22 Full Mission Trainer, B-2 Aircrew Training System, F/A-18 Tactical Operational Flight Trainers, F-16 Simulators, the U.S. Army’s Aviation Combined Arms Tactical Trainer (AVCATT) and Predator Mission Aircrew Training System (PMATS) ground control station simulation units.

“The desire is to be able to train anywhere, anytime,” says Lenny Genna, President of L-3 Technologies Defense Training Solutions. “When our warfighters go into the field, they want to do training and they need the equipment to allow that. They may not be able to take a full system, but we can provide a subset that comes close to the full system. Using the technologies we have today, we can do training in the field with other components in a secure fashion When you really want to mission rehearse, you’re doing that with a large package; live virtual constructive — using real aircraft and VR simulators — allows you to do that.

Lt. Col. Jason Turner, 80th Operations Support commander, looks up during a maneuver in a mixed reality flight simulator at Sheppard Air Force Base, Texas

High-fidelity displays

One of the biggest improvements in recent years involves high-fidelity, realistic displays. “Before, the computational power did not exist, the visual systems were very archaic, where today it is very high fidelity, so the realism and capability of the models is significantly higher,” Genna says. “Another change is from what I call safety aflight, which is a valuable piece, but today only a small percentage of the training, which predominantly is how to use the system most efficiently and incorporate changes reported from the field so those going in next will be much safer and more accurate in what they do.”

The enabling technologies, for the current state of the art and next-generation systems, have seen tremendous advancement in this century — especially the last decade — with even greater changes anticipated in the decade to come.

“AI is certainly an enabling technology for visuals, which also can give you virtual humans and intelligent tutors. Cloud-based architecture supporting the point-of-need is critical, as are haptics and greater computer power,” Genna says. “We’re not brand specific, but want to swap out and integrate quickly. We need to reduce lags and latency, to give entity representation — not just individuals and where they are located, but also the number of entities required for a realistic training event. Today’s first-person games are very closed scenarios, where the military training scenario can involve thousands of entities.”

High-performance computers are essential for rendering convincing imagery in simulation and training devices. “You need the computing power to make them look and act the way they should,” Genna continues. “Basically, what we need is a warehouse of computer power packed into a cigarette-case sized pack, which goes to SWaP. Each device has a different power requirement you can measure by the mission scenario. An average mission event of say 48 hours requires that much uninterrupted power during the mission, but you also have to do the follow-on piece and regenerate the next 48 without a significant lag or drop in performance.

Two U.S. Army Rotary-wing aviators conduct mission training in an L3 UH-60M.

Deployable simulation

“Weight is critical,” Genna points out. “The average soldier today has potentially 400 pieces of kit that can be configured for him to carry, depending on the mission. So every time you add something, you are adding the proverbial last straw. Our systems could be helmet-mounted. The key to it all is balance. If you have weight without proper balance, you have a problem. More power, less size and less weight is where you want to go, but packaging is critical to get optimum performance — and it also has integrated to the soldier so it is usable over a long period of time.”

Today’s state-of-the-art flight simulators can send pilots into combat fully trained in and ready to respond to a wide range of mission scenarios. In reality, however, once deployed they tend to fly a limited number of mission types over and over for weeks or months at a time, during which their skills on those missions they are not flying — but may be called upon to begin at a moment’s notice — deteriorate.

“Training occurs wherever you are, including operational environments, sometimes utilizing the operational platforms. But we need to train in a simulated environment, even in forward deployments, to improve skills,” says NAWCTSD’s Etz. “Once deployed, skills in a range of mission proficiencies may decrease without the ability to execute those using deployable trainers with low-cost tech — from computer-based, multiple screen platforms to advanced VR or AR.

COTS information technology

A wide variety of commercial off-the-shelf (COTS) computers and other information technology are available for today’s leading-edge simulation and training systems. “The current state of the art for flight training utilizes the best-of-breed of commercially available IT systems, allowing us replicate aircraft functions in a virtual environment, including accurate representation of aerodynamic models and mission systems capabilities and the environments in which those aircraft operate,” Etz says. “That has been one of the biggest improvements in the past 10 or 15 years. Visual and sensor representations are far more capable today, with pilots not just seeing high-def versions of geography and clouds, sunlight, darkness, etc., but the EM spectrum that goes along with that to represent radar environments, for example, which are far more robust and more accurately represent the real world.”

The ability to do highly realistic mission rehearsal in the field, altering aspects of the simulation in real time as new information becomes available, is considered critical to the U.S. military maintaining the technology and operational edge it has enjoyed for more than three decades — an edge that is now facing serious challenges from potential peer and near-peer adversaries.

“We’re learning more and more that powerful simulation benefits training and MR,” says the Army Simulation & Training Technology Center’s Clarke. “Cost alone is one reason. The operating cost of driving a real tank a mile down the road is significant; you can get that kind of training in a simulator at a fraction of the cost. And you can recycle sims over and over again. High quality, AR-immersive simulation allows a commander to subject a soldier or a team to a scenario where the risk is much higher than they would ever do with live fire. You can simulate casualty care, noise and confusion, and visual impacts. You also can alter content, change the scenario.”

One of the most high-level counter-terrorism operations in decades is a stark example. “Look at the raid on Osama bin Laden,” Clarke says. “We knew very little about the interior of that building and they had a short time to train — and had to do it in the dark. Simulation would allow you to change the wall structure, add additional guards, so they know what to do under differing circumstances and broaden the training scenario in ways you couldn’t do live. The virtual or AR environment will become more mainstream in our training — not just mission rehearsal, but a continuum from the institutional base to home station to deployed. Eventually, we will have intelligent tutors or avatars to train the soldier in the field on something they have not seen before in real-time.”

Intelligent tutors

Intelligent AI tutors will be a key element of next-generation systems, observing a wider range of student actions and reactions than a human instructor can handle in real time and reducing the assessment of a student’s performance from a lengthy after-action analysis and report to real-time feedback while the training is still underway.

“Currently, when evaluating individual soldier or group performance in training, you have a team of people who spend a huge amount of time to understand if the trainee is meeting certain objectives,” says Army STTC Deputy Director Ivan M. Martinez. “So there is a lot of effort on how to automate those so it can be done in real-time by the training system.”

Those will be integrated into future systems that will replace the Army’s current Close Combat Tactical Trainer (CCTT) and Aviation Combined Arms Tactical Trainer (AVCATT).

“CCTT and AVCATT are expensive, and maintenance and upgrade are costly,” Martinez explains. “So the Army is moving away from heavy infrastructure, large simulation facilities, to a more streamlined, software-based capability where simulation is provided over the network to where the soldier is so he doesn’t have to go to a fixed facility. And it is a more immersive, virtual system with some haptic components. We’re currently in the transition period of moving into new technology to enable that vision, which includes the future of ground, air and dismounted soldier simulations and the integration of them all into a new training capability.”

Psychological aspects

What is now considered a vital piece of training — and especially mission rehearsal — is stress. If the pilot knows nothing bad can happen in the simulator, the stress inherent is real-world combat will not be there to influence his or her reaction speeds and thought processes. That also applies to other platform and weapons training, from tanks and mortars to ships and submarines.

“There is a team of psychologists here at the Center looking at those issues. When we take a new technology out, we have a number of metrics we collect while doing that experiment — including how the stress affects the soldiers, from heart rate to pupil size, even saliva production,” Martinez explains. “That group is trying to figure out when and how to put cues into the training to bring those responses from the soldier to ensure the training outcomes are met and the system is optimized to trigger the types of responses we want when we need it to meet the training goals.”

Thus creating immersive environments where those who fly or work on or maintain aircraft, ships, tanks, etc., can rehearse and optimize their skills, wherever they may be, is a major goal of next-generation simulation, training and mission rehearsal.

“We want to create a scenario that replicates the pressures of mission operations,” he says. “That may include motion-based simulators with hi-fidelity visuals and accurate representations of cockpits. In others, we may not have full motion, but bladders that inflate or deflate to replicate G-forces. In others, wrap-around VR screens can create a sense of motion. And accurately creating a simulated X X X X helps bring up the stress level,” Etz concludes.

“It is always a balance between technical solutions and the requirements of the humans in the loop, those being trained and the instructors. There is usually not just one answer to solve a training requirement and at each point in a spectrum of solutions there are people involved in the loop. We provide solutions that improve human performance through simulation, but where we really want to meet the needs of the fleet is to provide trained sailors to meet the requirements of their mission.”