Probing what the human eye cannot see

Military electro-optical technologies are evolving quickly for intelligence, surveillance, and reconnaissance missions from small platforms like unmanned aircraft.

By J.R. Wilson

Electro-optical (EO) sensors, which passively sense energy emitted by or reflected from a target, have been in use by the U.S. military since the Vietnam War, beginning with forward looking infrared (FLIR) systems that provided gunship crews with night vision and later the stabilized TV telescope.

Advances in range, size, power requirements, and ruggedization have mirrored the evolution of computing and other technologies through the 1990s and into the 21st century. As those components evolved more quickly with each passing year, the application of electro-optical sensors across a wide range of manned and unmanned aircraft have greatly enhanced intelligence, surveillance and reconnaissance (ISR) missions.

An Air Force officer adjusts his night vision goggles during a night training mission.

“Surging demand for high-resolution and high-throughput sensors for gaining superior situational awareness, countermeasures, and reconnaissance is the key driver for the military EO/IR systems sector globally. To tackle conventional as well as newly evolved threats, militaries worldwide are focusing on installation of advanced sensor-based systems, which can provide intelligence in real time for necessary actions,” according to the report Global Military Electro-Optical/Infrared Systems Market 2018-2028.

“The increasing complexities in modern-day war techniques have resulted in a heightened requirement for ISR activities, which has resulted in several key defense spenders to invest heavily in the research and development of equipment such as thermal sights, infrared vision systems and laser target designators,” the report says. “The rising demand for new sighting devices as well as wearable night vision and thermal imaging systems is therefore contributing significantly to the growth of the global military EO/IR systems market.”

The state-of-the-art in turreted electro-optical infrared systems is a multi-spectral, multi-sensor system with a steerable and stabilized line-of-sight, say industry experts.

Modern systems include sensors that image effectively under all illumination conditions, from daylight to full darkness. A combination of small pixel size and long optical focal lengths also enables color daylight cameras to provide extended range under full illumination conditions.

The Air Force AgilePod is a multi-intelligence, open architecture, reconfigurable prototype pod for the ISR and Air Force Special Operations communities.

Low-light imaging

As light levels start to drop, at dusk and in overcast conditions, low-light cameras are necessary. These cameras generally have a sensitivity band covering visible and near-infrared light. Under completely dark conditions, a thermal imager can image using radiated heat.

Midwave infrared, which provides better sensitivity and resolution than longwave infrared, is considered preferable to longwave for most military scenarios. In surveillance missions, shortwave infrared (SWIR) imagers can image effectively at the limits of atmospheric visibility. For all types of imagers, high-definition (HD) arrays represent the state-of-the-art.

“Without ISR, we are useless,” observed retired U.S. Air Force Maj. Gen. George Harrison in opening remarks to the “Airborne ISR” conference in London in October 2015, adding that electro-optical sensors are key to maintaining a battlespace edge in ISR — growing to equal radar.

The growing importance of electro-optical technologies and ISR capabilities can be seen in Air Force and Army contract awards during 2018, including three Air Force Research Laboratory (AFRL) contracts to help develop new enabling technologies for next-generation electro-optical and RF (radio frequency) sensors, communications, information processing, imaging, and signals intelligence (SIGINT).

Defense Engineering Corp. in Beavercreek, Ohio; optX Imaging Systems LLC in Lorton, Va.; and Lockheed Martin Space Systems in Louisville, Colo. each are working on different elements of the Research and Development of Electro-Optical and Radio Frequency Sensors (RADERS) project.

Key technologies

According to the Air Force Research Lab, RADERS comprises 12 electro-optical and RF technology areas:

• antenna technologies and electromagnetic scattering;
• electro-optic and infrared sensor technology;
• sensor information processing and integration;
• electro-optical infrared, spectral and common-aperture electro-optic/radio-frequency hardware and algorithms;
• waveform phenomenology, design and applications;
• ultra-sensitive receivers for signals intelligence;
• long-range day and night hyperspectral imaging research;
• standoff high-resolution imaging;
• infrared search and track (IRST) technology;
• passive concept exploration;
• laser radar (ladar) imaging, systems, components and applications; and
• RF sensor systems

The first of those is designed to advance antenna and electromagnetic technology for air, ground, and space-based applications, including radar, communications, satellite operations, and ISR systems from HF to W-band frequencies, with an emphasis on detection, tracking, and data fusion for difficult targets.

The second centers on sensors and algorithms for target sensing, detection, recognition, and tracking with thermal imaging, single-aperture electro-optical/RF sensing, hyperspectral imaging, and multispectral sensing.

The AeroVironment Mantis i45 EO/IR gimbal payload on the Puma AE small unmanned aircraft can target with great detail while operating at extreme long ranges.

Electro-optical infrared, spectral, and common-aperture electro-optical/RF hardware and algorithms will help develop the ability to detect low-signal targets in noisy and cluttered environments using electro-optical, infrared, hyperspectral, multispectral, and common-aperture electro-optical/RF sensors — including focal-plane arrays, hybrid focal plane arrays, and infrared cameras for high-resolution low-power portable midwave and longwave IR sensing.

Laser radar (ladar) imaging, systems, components, and applications will apply ladar to ISR, precision attack, and air-to-air engagements — particularly to detect, track, and identify difficult air and ground targets in challenging environments.

Another the Air Force Research Lab program — Precision Real-time Engagement Combat Identification Sensor Exploitation (PRECISE) — is a $33.6 million, five-year sensor-fusion effort to enhance the effectiveness of surveillance radar by blending in electro-optical technologies such as visible light, IR, multispectral, and hyperspectral sensors.

Electro-optical sensor payloads

In what some are calling a “game-changer” for ISR, the Air Force Research Lab is working with other research organizations in government, industry, and academia to develop the AgilePod — a modular, plug-and-play sensor subsystem designed for rapid sensor exchange and a variety of sizes to fit a wide range of manned and unmanned aircraft. On June 2018, the AgilePod flew three demonstration missions on an MQ-9 Reaper medium-size military unmanned attack aerial vehicle (UAV) — its first on a major Air Force weapons platform.

An Air Force sergeant launches a small unmanned aircraft system during training at Eglin Air Force Base, Fla.

“The AgilePod program began with a desire to bring agile manufacturing practices to the ISR enterprise, culminating in a wholly government-owned, open architecture ISR capability that was payload- and platform-agnostic,” Andrew Soine, a program manager with the Air Force Research Lab’s Materials and Manufacturing Directorate, said during the test flights.

“The program is really taking off, with proposed ISR and non-ISR applications that we couldn’t have foreseen only a few years ago. By owning the technical baseline, we’ve shown what can be done in relatively little time and cost when faced with emergent user needs.”

In October, the Army awarded a $454 million contract to L3 Technologies for a next-generation WESCAM MX-10D electro-optical infrared and laser designator sensor suite for the Army’s RQ-7Bv2 Shadow UAV.

“We are focused on developing new state-of-the-art multi-domain C6ISR [Command, Control, Communications, Computers, Cyber defense, Combat systems and ISR] technologies in support of the U.S. Army’s Modernization Strategy,” L3 CEO Christopher E. Kubasik said in the announcement.

“The market is demanding greater levels of sensor automation and integration,” says Cameron McKenzie, L3 Wescam vice president for global business development. “The object tracking function in MX products has recently undergone a significant upgrade. Operator burden has been significantly reduced as the system will automatically detect potential targets based on motion, and will annotate them.”

“This allows the operator to select which is to be tracked,” McKenzie continues. “As well, tracking robustness is significantly improved. This is of special importance to the unmanned aerial surveillance market as manual target tracking can be challenging due to the inherent latencies in remote UAS and payload control.”

Israeli electronics company Elbit Systems designs a variety of electro-optical night-vision systems for use in aircraft cockpits.

Expanding ISR

ISR, still predominantly air- and space-based, is expanding in ground-based, down to the individual warfighter, and maritime applications, from Navy aircraft carriers to Coast Guard small boats.

“In terms of ground combat ISR applications, electro-optical sensors are well-suited to forward observer roles, such as directing artillery fire, directing close air support, and observation post vehicle,” McKenzie says. “In such installations, the sensor can be mounted either directly to the vehicle or on a mast. The emphasis in the ISR role is on optical range performance and target geo-location performance.”

Elbit Systems Electro-Optics (Elop) in Haifa, Israel, is the world’s largest electro-optical provider outside the U.S., and produces a range of advanced electro-optical sensors for airborne, ground and naval applications, from position and orientation data to ISR.

Elop’s electro-optical land applications range from night sights and target-acquisition systems to payloads and perimeter security to laser designators and thermal imagers. These sensors are designed for extended early warning and rapid response capabilities for ground forces.

Thermal imagers, hyperspectral cameras, and similar sensors offer ISR solutions for aviation, such as the Multi-Spectral Infrared Countermeasure (MUSIC) family of direct infrared counter measures (DIRCM) solutions and the Surveillance, Warning, Obstacle Ranging and Display (SWORD) laser radar system. The Elop Compact Multi-Purpose Advanced Stabilized System (CoMPASS) family of electro-optical payloads feature day and night imaging sensors and laser designators for sea-based ISR, target acquisition and target designation.

According to the Air National Guard’s 2018 Weapons Systems Modernization Priorities, the service’s MC-12W medium-to-low altitude twin-engine turboprop aircraft requires an additional electro-optical infrared sensor to meet the demanding ISR tasks required by combatant and task force commanders.

An MQ-1C Gray Eagle unmanned aerial system, equipped with 3rd Gen FLIR sensors prepares to conduct a mission from Al Asad Air Base, Iraq

“The ANG MC-12W mission heavily relies on the ability of the crews to see the smallest details on the ground from miles away. Currently, the MC-12W is outfitted with a single MX-15DiD sensor on each aircraft. While this allows the MC-12W to complete a wide range of ISR tasks, it is extremely limited when it comes to fidelity and flexibility. This added system will double the amount of area to be seen by MC-12W crews and provide a substantial amount of situational awareness to the commanders on the battlefield,” the report said.

The Massachusetts Institute of Technology’s Lincoln Laboratory in Lexington, Mass., is capitalizing on recent advances in imaging architectures and real-time processing to develop a miniature 3D ladar sensor for small UAVs. The tennis ball-sized sensor is being designed to provide a one-billionth reduction in size, weight and power (SWaP) to generate 3D terrain and structure imagery through dense foliage.

By integrating the micro-ladar system on a UAV as small as a quadrotor, 3D imaging can become a feasible option for users who cannot afford large aircraft necessary for other systems, Lincoln Lab officials say. The micro-ladar could be used under clear and heavy foliage conditions for surveillance and reconnaissance missions as well as for humanitarian assistance and disaster relief operations.

Evolving user requirements

User requirements for electro-optical technologies constantly are evolving and expanding.

“Customers are also demanding entire mission solutions that include not just the sensor, but mounts and consoles,” says L3 Wescam’s McKenzie. “Consoles typically include advanced human-machine interface (HMI) and sensor automation functions, as well as displays, recorders, and microwave communications equipment. L3 has responded to this with the development of the MatriX package, a pre-engineered kit for specific aircraft platforms. MatriX currently supports UH-60 Black Hawk and C-130 Hercules platforms, with more to come in the future.”

The Marine Corps has electro-optical ISR projects in progress for UAV sensor payloads. The goal is to take advantage of reduced SWaP to give small UAVs modular, tailored, and specific mission capabilities that have been limited to large UAVs used by the Air Force and Navy. That will enhance the Marine Air-Ground Task Force’s (MAGTF) ISR collections, using RQ-21 Group 3 and RQ-20 Group 1 UAVs supporting Marine Expeditionary Units (MEUs), Marine Special Operations, infantry regiments, battalion and lower echelon operations.

Several research, development, and procurement efforts are in progress, says the Marine Corps Concepts & Programs 2018 report to Congress.

First is the Tactical electro-optical/IR SIGINT Integrated for Targeting (TEISIT) sensor, a Group 3 payload that is part of the Wide Area Surveillance and Persistent Surveillance (WASPS) program. The payload includes signals intelligence and electronic support; synthetic aperture radar (SAR) and electro-optical modalities. The sensor also will incorporate advanced processing algorithms to perform on-board data fusion, exploitation and an autonomous sensor control modality.

Second is the Spectral and Reconnaissance Imagery for Tactical Exploitation (SPRITE) project that provides wide-area ISR capability with simultaneous high spatial and spectral resolution. The system is capable of wide area electro-optical infrared and hyperspectral imaging.

Next is the AN/DSY-4 Spectral Bat sensor payload providing SIGINT and electronic support capability, including signals detection, geo-location, and a communications relay. These payloads complement the UAV’s organic electro-optical capability.

Image enhancement

Modern sensors include image enhancement functions for noise reduction, sharpening, and local area contrast enhancement, which help improve feature recognition and bring detail out of shadows. They also include advanced functions like object tracking, image blending, a laser rangefinder, one or more laser illuminators, and possibly a laser target designator. The laser rangefinder enables the operator to stay at a covert distance from the target and aids in target geo-location accuracy.

“The imaging and laser payload is supported by a four- or five-axis gimbal that steers and stabilizes the optical line-of-sight. Many low-cost turreted electro-optical/IR systems consist of only a two-axis gimbal. These feature poor stabilization performance as the servo-mechanism that performs the fast and fine motion required to stabilize the payload is directly loaded by the external airstream and other factors. A four- or five-axis configuration isolates the stabilization gimbal from the outer housing gimbal,” says L3 Wescam’s McKenzie.“The electronics responsible for image processing, tracking, and moving target indication, is integrated into the turret unit of a state-of-the-art sensor. This significantly reduces overall system weight as it eliminates an external electronics box and associated cabling, which simplifies aircraft/vehicle integration.”

ISR operators generally are interested in sensor range performance and automation features. When executing a surveillance or reconnaissance flight pattern, for example, longer target detection range enables the operator to execute wide leg spacing, while longer recognition and identification range performance means fewer deviations from the search pattern, resulting in shorter, more economical and more effective missions requiring fewer aircraft and fewer mission personnel.

“The primary contributors to better range performance have been the advent of HD cameras and better stabilization. Better image processing has resulted in better feature extraction,” McKenzie adds.

The last five years have seen a greater relative uptake of targeting sensors due to an increased emphasis on multi-role aircraft in fleets around the world. That has led to a renewed interest in large sensors for long standoff multi-mission surveillance and tactical platforms as well as small sensors for smaller, lower altitude platforms.

Complementary technologies

“Electro-optical sensors are a complementary technology to other technologies found in an ISR mission equipment package. Ground moving target indication and imaging radars are very useful in target detection given their long range and ability to penetrate atmospheric obscuration and foliage. That said, their resolution is limited, so they are not very useful for target identification,” says L3 Wescam’s McKenzie.

“That is the forte of a sensor that operates in the electro-optic band. Similarly, wide area motion imagery, signals intelligence and acoustical detection sensors can all be used to cue a directional electro-optical/IR sensor for target recognition and identification tasks.”

The Pentagon recently changed contractors to upgrade the F-35’s AN/AAQ-37 electro-optical Distributed Aperture System (DAS). When the original supplier, Northrop Grumman, elected not to bid on future DAS production, the Air Force awarded the contract to Raytheon, which offered a 45 percent reduction in cost per unit and 50 percent cut in sustainment costs. And, according to Lockheed Martin, five times the reliability and twice the performance.

The electro-optical DAS collects real-time, high-resolution imagery from six IR cameras mounted around the aircraft and sends it to the pilot’s helmet-mounted display, providing a 360-degree spherical view of the environment.

“The next generation is expected to be a continuation of recent trends in electro-optical/IR systems. We expect to see the incorporation of newer, higher resolution camera detector technology as chip production levels increase and costs decline to the point where it makes economic sense,” says L3 Wescam’s McKenzie.

“We expect to see more advanced image processing capabilities, with an objective to further reduce the operator’s workload. Finally, we expect to see higher levels of integration of electronic functions in the imaging,” McKenzie concluded.