Electronic binoculars from Northrop Grumman team to detect threats through brain activity
LINTHICUM, Md., 23 Aug. 2008. Everyone who has ever watched the Star Wars films from George Lucas has probably at one moment wished they had Jedi abilities such as mind control or what Lucas called Jedi reflexes – knowing something will happen a second before it does.
A team led by Northrop Grumman's Electronic Systems Sector is looking to bring a similar threat detection capability to warfighters as part of an advanced research contract to develop a panoramic day/night optical system that will utilize human brain activity to detect, analyze, and alert foot-soldiers to possible threats.
The Defense Advanced Research Projects Agency (DARPA), in Arlington, Va., awarded the contract, which is for the first phase of the Cognitive Technology Threat Warning System program, or CT2WS.
DARPA officials say the CT2WS will function as an intelligent neuro-optical system, using the stimuli sensed by brain activity to detect targets at long-range over a wide field of view.
Not able to rely on that mystical energy field called the Force, as Lucas's characters do, the academic and industrial consortium are looking to drive a breakthrough in soldier-portable visual threat warning devices using technology that has yet to be invented.
The first step for the Northrop Grumman engineers is developing a broadboard system and completing a preliminary design for the company's Human-aided Optical Recognition/Notification of Elusive Threats (HORNET) system.
"Northrop Grumman's HORNET system leverages the latest advances in real-time coupling of human brain activity with automated cognitive neural processing to provide superior target detection," says Michael House, Northrop Grumman's CT2WS program manager. "The system will maintain persistent surveillance in order to defeat an enemy's attempts to surprise through evasive move-stop-move tactics, giving the U.S. warfighter as much as a 20-minute advantage over his adversaries."
The human's subconscious mind is hypersensitive to visual images and the system will take advantage of that by using non-invasive electro-encephalogram (EEG) electrodes EEG measurement sensors placed inside the warfigher's helmet and attached to his scalp, House says.
A soldier's subconscious may detect a threat his conscious mind is not registering, House says. The EEG sensors will read the soldier's neural responses to potential threats, and then the readings will educate the system's algorithms.
For example the algorithms will identify when the warfighter's subconscious is detecting a moving target at 5 kilometers or one at 10 kilometers, House continues. As missions and threats evolve the algorithms will continue to be refined.
One of the main goals of the Hornet is to reduce the false alarm rate for target acquisition, House says. The human brain's visual process has very few false alarms, he adds. This should also significantly improve the range of standard binoculars, House notes
Lowering false-alarm rates is crucial for HORNET's intended applications, which include improvised explosive device (IED) detection and destruction, aided target recognition programs, and border surveillance for homeland security scenarios.
House declined to go into more specifics on the neurological technology, but referenced a public release from the Georgia Institute of Technology in Atlanta. Northrop Grumman is working with a Georgia Tech team led by Paul Hasler, an associate professor in the Georgia Tech School of Electrical and Computer Engineering, he says.
Hasler's team is researching neuromorphic engineering techniques to emulate how the human brain processes images, House says.
"The idea of this project is to build a visual device that is attentive, that can do the kind of low-level visual processing that your eyes do naturally," Hasler says in the release. "You would see a certain picture in your field of view, but the device would actually be looking over a much wider space – and if it found something interesting it would present you with that picture as well.
According to the Georgia Tech release "neuromorphic engineering is interdisciplinary, using fields that include biology, physics, mathematics, and computer science, as well as electrical and other types of engineering. Its aim is to develop artificial systems—such as vision devices or auditory processors or even robotic systems—based to a degree on natural biological systems."
"This is a highly interesting platform problem because you really are trying to build, at least initially, a simple representation of one layer of human brain cortex, maybe multiple layers if you can," Hasler continues. "You need a great deal of computational power, robustness, and performance, and it becomes possible because a configurable analog-digital system has a power efficiency improvement of up to 10,000 times compared to an all-digital system."
Hasler also comments in the release that this technology should lead to new "neural-inspired" approaches in military as well as commercial applications.
Northrop Grumman will also integrate current wide angle optics, digital imaging, and low-power digital signal processing (DSP), House says. Low-power is an important characteristic as this will be a portable warfighter system with heavy DSP requirements, he adds.
Following the first 12-month, $6.7 million phase of the project, DARPA has the option to extend the contract for two additional phases to develop the subsystems and final handheld assemblies. A prototype expected to be ready in 2010, House says.
In addition to Georgia Tech, Northrop Grumman's multi-disciplinary team includes SAIC in San Diego; Theia Technologies LLC in Wilsonville, Ore.; Sensics, Inc. in Baltimore; L-3 Communications Infrared Products in Dallas; Georgetown University, in Washington; Portland State University in Portland, Ore.; and the University of Colorado in Boulder, Colo.
