Military researchers and industry experts are adapting different EO/IR sensors to work together such that they can become greater than the sum of their parts, and help provide greater situational awareness than ever before.
By J.R. Wilson
Situational awareness has been the centerpiece of combat throughout the history of warfare. In the 21st Century's increasingly networked digital, data-heavy battlespace, it has become more crucial than ever to warfighter security and success. This is where electro-optical and infrared (EO/IR) sensors come into play. These sensors and their data processors often serve as the eyes of deployed military forces.
The continuing challenge for the military is maintaining a technological edge in EO/IR sensor systems. Adversaries are becoming more sophisticated in their tactics and technologies, and the U.S. military's efforts to maintain the technological advantage have grown in importance and difficulty.
"If you see the gap between potential in performance and where we are in delivering performance, that gap is just starting to open," Chief of Naval Operations Adm. John Richardson told the U.S. Naval Institute's annual Defense Forum in December. "It's not a huge gap right now; it's probably small enough that on our best days we can convince ourselves we're doing OK. But you can see, going forward, that gap is only going to widen at ever-increasing rates."
Often the real value of EO/IR sensor develop- ment only becomes apparent during demonstrations in actual military environments. "The real magic happens when you put [technology] with the developers and they meet the sailors out there and they watch their idea come to life," Richardson says. "The amount of creativity out there in the fleet is tremendous."
Keith Lannan, technical warrant holder for surface EO/IR sensing systems at Naval Sea Systems Command (NAVSEA) in Washington, says the Navy fleet recognizes the value of EO/IR sensors, so the demand is strong and growing stronger.
"The current naval missions driving work in NAVSEA primarily cross different missions and functions, such as navigation, force protection, surface warfare, anti-air, and other support areas," Lannan says. "The key driver is motion imagery from EO/IR sensors that provide day-night, long-range eyes on target, which improves the user's ability to identify targets, perform threat assessment, assess intent in accordance with the rules of engagement, and support weapons engagement through automatic tracking and fire control solutions through line-of-sight. The sensors also support assessment of engagement effectiveness."
Future capabilities
For future capabilities, naval experts are concentrating on how new EO/IR technologies can make existing sensors even better. "The situational awareness driver currently is focused on augmenting the current sensors, which were primarily driven in the past by requirements for specialized weapons sensors, with adjunct sensors for situational awareness that will enhance that for the ship," Lannan says. "There are some differences, but also similarities, across ship classes. We are performing analyses on requirements for standardizing sensors to improve capabilities and reduce total ownership cost."
One way sensors are evolving is dual-purpose use for weapons and situational awareness. Navy leaders want targeting and surveillance sensors to be operationally available for intended missions, so NAVSEA is augmenting them for situational awareness to alleviate the wear and tear on special-purpose sensors.
"Today our sensors are intended to be a point-solution for various weapon systems, so one thing that is evolving is looking hard at how to integrate sensors into an integral part of the ship's combat system and streaming video dissemination, for example, for overall improved interoperability. Standards will help facilitate that in an open-systems approach and will enable us to have better integration with other combat system elements," Lannan says. "If we can standardize sensors and use common modules for situational awareness and engagement support, that is a major benefit. So a lot of the emphasis is on things we can do, such as standardizing interfaces at the system and module levels and other technology refresh cycle and maintenance modules so they can be used across multiple platforms and systems. When standardized, it will help achieve the Navy's open architecture goals and maintain competition throughout the life cycle."
Maintaining the technological edge depends not just on improving individual sensor technologies, but also on merging two or more sensors into a single device, fusing raw data, on-platform processing of data into useable information, and real-time sharing of that information across all assets, from theater commanders to individual warfighters.
"A general trend, toward which we're already progressing, is full situational awareness distributed across the battle network," says Andrew Saxton, marketing director for surveillance at EO/IR specialist FLIR Systems Inc. in Wilsonville, Ore. "In the next five years, you will see marked progress in the ability of every asset in the field to share what is being seen to one central command but also sending that out to the rest of the battlespace I think the uses [of related sensors] will remain relatively stable, but different assets will be called upon to conduct them."
Shrinking size
One pervasive trend in EO/IR sensors is shrinking size, weight, and power consumption (SWaP). "As you get better performance out of smaller sys- tems, you can send a small UAV [unmanned aerial vehicle] rather than a large manned aircraft or light tactical vehicle instead of a large platform. As more of them become man portable, more tasks can be accomplished from the ground rather that the air. Distance and operating conditions will see new systems providing commanders with greater flexibility in achieving their objectives."
A leader in that evolution is the merger of EO/IR sensors into new applications, ranging from Navy ships to special operators. "EO and IR are more or less permanently bonded now," says Peter Zinsli, a sensors expert with the U.S. Army's Intelligence and Electronic Warfare & Sensors program executive office at Fort Belvoir, Va. "The EO provides day capability, IR day and night, but EO has better range capability; it is more interpretable. Long-term, we're looking at better sensors and more sensor fusion and AITR [aided target recognition]. We will see a continuation of wanting more resolution, more stand-off, which will require larger, better FPAs [focal plane arrays]. I'm skeptical of truly novel optical enablers; the big thing will be putting a larger aperture on these sensors."
Sensor fusion is a big trend in military EO/IR sensor technology. "A fusing of multiple sensors, of all types, and building software that provides products as opposed to data, is going to be key in the long-term. As expensive as the sensors are, it also is expensive to have more eyeballs looking at each sensor. With more data to parse, unless you put more and more people on it, you need to shorten the timeline between raw data and actual usable information."
The vast variety of platforms built by several different prime contractors incorporate a lot of FPAs, EO/IR sensors, different frame rates, and number of channels that spew out an ever-growing volume of data. Companies such as Curtiss-Wright Defense Solutions in Ashburn, Va., build digital signal processing, high-performance embedded computing modules for subsystems to capture that data, often pre-digitized, some in high bandwidth formats. Some primes buy individual modules, some preconfigured subsystems.
Sensor processing
"There is not typically one type of processing technology that can do the whole application," says Marc Couture, senior product manager of digital signaling at Curtiss-Wright Defense Solutions. "The most popular mix is FPGAs [field-programmable gate arrays], the first to capture the fiber-optic stream, that does some preprocessing, then hands off to GPGPUs [general-purpose graphics processing units] - teraflop floating-point-per-second, very high throughput processing and Intel devices, such as the new Xeon D that is a bit more cognitive, can analyze data and change modes, kind of a director/conductor that also does processing."
It takes a special blend of different processing technologies to handle fast streams of high-resolution imagery from EO/IR sensors. "That trio is the mix du jour, (but) every application uses a different combination of FPGAs, GPUs, etc., and different primes are more partial to different combinations," Couture says. "The form factor most popular in defense is VPX; there are smaller form factor implementations when needed. We often take industrial versions of those chips and ruggedize them for extreme temps, different vibration profiles, and sealing against foreign particulates."
The Army is pursuing similar efforts to enhance battlefield situational awareness. "The Army operating concept of winning in complex environments with complete situational understanding creates requirements for EO/IR sensors doing a targeting mission. You also need wider situational awareness, which you can cover with picture-in-picture and Step-Stare, which essentially is mosaicing, where you move your sensor around and take multiple images, then stitch those together," Zinsli says. "As we move forward, we hope to have better integrated payloads. You may have sensors covering a lot of different bands, but with some sort of AITR with a software component that turns all the incoming data into information or at least cues for the operator."
Technology insertion
For the near- and mid-term, the military is looking at upgrading legacy platforms and systems. One difficulty is the research budget, which has led the Army, for example, to look at partner labs to come up with good ideas, and work on related technologies they eventually can transition.
"We have moved away from the quick reaction capability mode of operating, which leads to longer timeframes to develop technology. QRCs were the product of urgent needs in the battlespace and, during the Bush years, having the money to solve the problem of people dying," Zinsli says.
As with more and more military technology, cutting-edge research is coming from the commercial world, modified to meet the needs of warfighters but advancing what they have far more quickly than would be possible with specialized military hardware. Hand-in-hand with that is a heavy focus on making new systems easy to use and intuitive for the warfighter in the field.
"There is always pressure to provide more resolution, more standoff, and more imaging capability. One of the new groundswell trends has been on the intel those systems can provide in addition to a high-definition (HD) image. Other than just seeing a pretty picture, the technology has reached the point where the field equipment must make other decisions," says FLIR's Saxton. "All our airborne, man-portable, and vehicle systems provide precise target locations to the operator, which they can share to the regional command."
It was the commercial world that first developed HD imaging affordable enough for widespread deployment. "HD imaging was available to the consumer electronics base before it was widely adopted throughout the military, but HD thermal imaging of targets is a much harder and more difficult technology to stand up," Saxton says. "Last year we debuted the HISS-XLR, an HD thermal imager mountable to a rifle, a capability that previously only existed at high-end airborne and vehicle levels that now is available to the soldier."
NAVSEA experts have been examining the potential future capability of wide field-of-view (FOV) distributed-aperture sensors or persistent 360-degree surveillance. That includes the transition of large-format, small-pixel FPA technology, stitching sensor images together, and integrating wide FOV sensors with more conventional sensors for long-range target identification and classification.
Increasing fields of view
"The intent with wide FOV is to add an auto-detection and multi-target tracking capability to achieve wider situational awareness and for weapons systems and engagement support. There is a big focus on adding wide FOV and interoperability between those and narrow FOV sensors, with the latter taking a closer look for identification, threat assessment, determining false alarms, etc.," says NAVSEA's Lannan. "Another thing we're looking at is shortwave IR sensors for improved performance over visible band sensors in the maritime environment to help with degraded visual environments and for in-band laser imaging," Lannan continues. "The imagery will relieve some of the workload and improve situational awareness in cluttered environments. With all that imagery, we have to be smart about how it is managed and displayed to avoid information overload."
As evidenced by attacks on vessels operating along coastlines and in harbors, situational awareness afloat is not limited to radar scans for hostile aircraft or sonar for submarines. That is especially true for the next generation of Air Cushion Vehicles, designed to conduct day/night off-shore, near-shore, and on-shore missions. ACV operators must have the highest level of situational awareness in all those mission environments. That includes not only time-of-day visibility, but sea spray, high humidity, haze, smoke, even sand and dust. The current trio of radar, night-vision devices, and visual watches - relatively unchanged since World War II - on the Navy's Landing Craft, Air Cushion (LCAC), is no longer adequate, prompting the Navy to seek a new durable, low-cost EO/IR imaging device for such vessels.
"An EO/IR system (or similar hyper-spectral imaging sensor) should greatly increase operator situational awareness by providing the ability to discern land terrain, sea and waterway features and contacts, such as small boats and patrol craft in the near-shore environment where surface radar performance is limited," according to a 2015 Navy solicitation. "A hyper-spectral imagery system with thermal contrast, such as an EO/IR sensor, will provide capability to detect and identify surface contacts and beach-borne and land-based threats that have very low radar signatures. The unique capabilities of a combination of EO imagery, IR imaging, NV systems, and laser and range-finding tools will enhance ACV combat operations."
The solicitation notes such a system also could have dual-use applicability for commercial hovercraft, ferries, pilot boats, or any other watercraft operating in the near- or on-shore environment, as well as Army riverine vessels and military vehicles operating on-shore.
Reclaiming the night
Meanwhile, the Army Communications-Electronics Research, Development and Engineering Center's Night Vision & Electronic Sensors Directorate (CERDEC NVESD) at Fort Belvoir, Va., is working to move beyond the night-vision systems that enabled U.S. warfighters to "own the night" in the 1990s and 2000s when night-vision goggles became readily available to everyone. While still key parts of the warfighter's kit, reclaiming the night will require new and more advanced technologies.
"Today's average soldier out in the field has a pair of image-intensified goggles," NVESD director Dr. Donald Reago said at the 2015 SPIE Defense Security and Sensing Conference. "He or she has a night-vision goggle that works in the near infrared. The technology has been around for 40 years; it works phenomenally well in terms of amplifying starlight and ambient light and producing it within your eye. There's no analog, there's no digital; light comes in, light comes out."
NVGs always have had operational limitations: no thermal capability, no seamless transition from day to night, and no close-up to long distance viewing. NVESD's Enhanced Night Vision Goggle (ENVG) incorporates an image intensified channel, a thermal channel, and an optical overlay that can be combined with the camera's output and the image intensified on a small display. The current ENVG still does not have full digital capability.
We would like to go to "something which is more of a digital sensor, the helmet-mounted display or helmet-mounted imager, and then digital sensors beaming that display," NVESD's Reago added. His goal is to move to a system with a camera in the back, a sensor, light display, storage and memory capabilities, and which runs software applications. "This is what our soldiers are growing up with and this is the future."
A return to near-peer/peer adversaries is driving the development of EO/IR sensors and other enhanced situational awareness technologies.
"Across the Pacific, we have adversaries far more sophisticated and agile than we've faced the past 15 years," says Curtiss-Wright's Couture. "Because things have sped up, I think we will be more agile, as well, but you can never assume we will always be king of the hill. You have to work for that. As we got into the Gulf wars, we got a lot of requests for protection against fine particulates, heat rather than cold."
