Homeland security and military personnel increasingly rely on intelligent sensor technology for surveillance and electronic intelligence.
By Courtney E. Howard
More than 1 million passengers and pedestrians, 327,000 vehicles, and 70,000 shipping containers enter the United States through its ports each day. “Hidden among legitimate travelers and trade are persons who seek to enter this country illegally or smuggle contraband, both at and between the ports of entry, including potential terrorists,” reads “The State of Homeland Security 2007; An Annual Report Card on the Department of Homeland Security” prepared for the House Committee on Homeland Security in the U.S. Congress.
The mission of the U.S. Department of Homeland Security (DHS) and the U.S. military is, in part, to ensure the protection, safety, and security of the nation and its citizens. Science and technology are playing an ever increasing role in this mission. In fact, the Directorate for Science and Technology, the primary research-and-development arm of DHS, is working in partnership with private businesses, national laboratories, universities, domestic and foreign government agencies, and the military to drive the development and use of high-tech solutions in support of securing the nation. Together, the homeland security and military communities are making significant strides in technology, particularly in the vital areas of intelligence and surveillance.
“Relative to anti-terror operations, they are likely the most important operations we have,” notes Ross Smith, president of Quantum3D Inc., a provider of commercial off-the-shelf (COTS), open-architecture, real-time visual computing solutions in San Jose, Calif. “Intelligence and surveillance are the only ways we can possibly detect a terrorist act in the planning stages and interrupt it in time to save lives. If you think about 9/11 or the train bombings in the U.K. or Spain, only intelligence, possibly amplified by surveillance, could have detected the planning of these events and possibly helped to interdict them.”
Meeting a critical need
“Intelligence and surveillance are critical for America’s homeland security and defense,” admits Andrew Partynski, president of DRS Homeland Security Solutions Inc., a division of DRS Technologies Inc. in Parsippany, N.J. “Being successful in detecting and classifying harmful threats helps ensure our physical well being. The arduous task is to detect and classify dangerous chemicals, materials, explosives, and people that may be threatening. The challenge is not only to correctly detect threats from a safe distance and ahead of time, but to do so very quickly in order to limit the impact on travel, trade, and commerce.”
It is through the application of today’s advanced radars, sensors, cameras, and software that this very real challenge should be met, Partynski suggests, rather than by building fences or walls. “These technologies—which apply to critical infrastructure, airports, seaports, land borders, ports of entry, and inland waterway and coastal borders—gather critical information while simultaneously limiting their impact on our daily lives.”
Extreme CCTV’s Infrared Imagers are available in 950nm infrared wavelength for covert illumination, making them ideal for defense applications including force protection, critical asset monitoring, and perimeter surveillance.
Engineers at DRS Technologies—an integrator of intelligence and surveillance technology and a manufacturer of component technologies, ranging from radars and infrared cameras to physical security systems and command-and-control systems—are part of the team lead by Boeing Co. in St. Louis that is working on SBInet, initiated by DHS in 2006 to develop an integrated system of personnel, infrastructure, technology, and rapid response to secure U.S. land borders.
The landmark program is designed to provide DHS and its agents, officers, and partners—including U.S. Customs and Border Protection, U.S. Immigration and Customs Enforcement, U.S. Citizenship and Immigration Services, and U.S. Coast Guard personnel—with the best possible solutions with which to detect, identify, classify, respond to, and resolve illegal entry attempts and other issues at U.S. borders. “SBInet is a program focused on transforming border control through technology and infrastructure,” explains a Boeing representative.
Appian Technology’s Stinger intelligent camera solution integrates the company’s Cobra camera with Quantum3D’s Embedded Visual Computing Subsystem (EVCS-1.0) and offers connectivity to and control of such electronic sensors and systems as dedicated short-range communication (DSRC), radio-frequency identification (RFID), speed, and red-light detection devices.
DRS engineers have provided systems engineering for the SBInet program’s intelligence and surveillance systems, as well as integrated tower-based surveillance systems at the borders. Managers at DRS Technologies, which is CMMI Level 5 certified in homeland security and defense, launched DRS Homeland Security Solutions Inc. specifically to address the needs of the homeland security market.
People are a critical asset that we always find in short supply, especially those with extensive training in intelligence and surveillance, Partynski admits. “These operators are not available in the vast numbers required to monitor all the systems that will be needed to secure our nation. The solution is to automate these surveillance systems so that most of the work is done by the system itself.”
Smart sensors
Technology solutions are needed that will reduce operator workload and enable one person to manage many surveillance systems efficiently and effectively. Enter smart sensors.
“Smart sensors are the goal for our future deployments of intelligence and surveillance systems,” acknowledges Partynski. “Automated systems that detect, track, and classify threats without radar experts or SIGINT (signals intelligence) experts, or even people continuously staring at camera video, will allow the U.S. to deploy a nationwide system that is secure, has low false alarms, and cues the operators only when a real threat is detected.”
Automated solutions, while bringing about a reduction in personnel costs, also contribute to time savings. A lot of technology out there is gathering tons of data, recognizes Smith. “The key issue,” he says, “is how to reduce the timeline between collection, processing, and assessment?”
The Stretch IP Camera Reference Design Kit (RDK), a multistandard, high-definition, production-ready design based on the S6105 software-configurable processor, is capable of performing H.264 compression on video streams to overcome bandwidth limitations.
“If you have to review the video from thousands of cameras manually, then it’s hopeless,” says Smith. “That is why we’re so keen on ‘smart’ applications in which the camera or collection system has the capability to perform preliminary analysis, and then report interesting stuff to humans who can further assess the data.”
“The camera itself processes the images,” says Smith of a smart camera project for Appian. “It can test them against a database and simply report the tag info, time, and location, or it can provide amplifying info, such as photos, if requested.” Taking humans out of the loop until they are needed saves a great deal of time, he continues.
Managers at Appian Technology PLC in Bourne End, England, selected Quantum3D’s Embedded Visual Computing Subsystem version 1.0 (EVCS-1.0) for use in its Appian Stinger Intelligent Automatic Number Plate Recognition (ANPR) camera. The new Appian Stinger camera recognizes vehicle license plates in traffic and reports the results to traffic and police authorities using integrated wireless and wired communication technologies, including GPRS/GSM, 3G, IEEE 802.11, and IEEE 802.3.
The Appian Stinger combines optical and sensor components from Appian’s high-performance Cobra camera with the Quantum3D EVCS-1.0. The scalable solution can be optimized to fit a range of environmental conditions and capture points using various camera and infrared illumination options. Users can adjust settings remotely in real time, as well as use the camera to connect to and control other electronic sensors and systems, such as dedicated short-range communication (DSRC), radio-frequency identification (RFID), speed, and red-light detection devices.
The Quantum3D EVCS-1.0 is a COTS, open-architecture, RoHS-compliant, conduction-cooled solution designed for machine-vision, traffic-surveillance, and security original equipment manufacturers (OEMs) and others who need advanced, embedded visual computing capabilities. Able to withstand harsh environmental conditions with extreme temperature fluctuations, the EVCS incorporates Intel Mobile CPUs and the Quantum3D Eidetix Advanced Video Capture and Processing System (AVCPS). This COTS subsystem enables the EVCS-1.0 to capture and process up to four simultaneous NTSC, PAL, or RS-170 composite video streams in Microsoft Windows or Linux operating environments. Its PC application compatibility enables the rapid deployment of desktop and notebook computer-based applications into fixed or mobile extended environments.
Sensors save lives
In addition to providing critical information on potential threats, intelligence and surveillance systems provide a safety net for people in remote areas of the U.S. border, Partynski says. “People regularly get into situations where they become lost or stranded or are victims of crimes. Having these technologies broadly deployed will help find people in trouble by locating them with advanced radars, sensors, cameras, and software or via their cell phones or radios. These technologies allow us to determine if activities are not normal and warrant someone to take a closer look and possibly assist.”
U.S. Navy personnel have been using Cross Match Technologies’ Man-Wearable Multi-Modal Jump Kits since 2006. The kit’s Biometrics System helps them identify persons of interest aboard vessels during specified interception operations. Expanded Maritime Interception Operations (EMIO), performed by the Navy’s Vessel Boarding Search and Seizure (VBSS) teams in all corners of the world, play a role in the prevention of contraband and illegal weapons smuggling and also in detaining and arresting known international terrorist suspects.
“The Cross Match Multi-Biometric System with the Quantum3D Thermite Tactical Visual Computer system brings the power of PC-based workstations to vessel boarding and seizure operations,” explains U.S. Navy Cmdr. John Funn, assistant program manager for Expeditionary Systems, Anti-Terrorism Afloat (PMS 480). “This system allows the EMIO teams to collect and forward biometric data from potential terrorists, smugglers, and other international criminals on intercepted ships.”
Navy personnel use the Cross Match Multi-Modal Jump Kit to gather biometric data, in the form of fingerprints, from persons of interest on boarded vessels. The data is then compared with fingerprint data of known criminals and terrorists in national and international police and national security databases, enabling positive identification of individuals and preventing the use of false identity documents.
“By having the Biometric System on-board the interdicted vessel, the personnel can be verified immediately,” Smith says. “It provides allied personnel with immediate, actionable information they can use to stop a criminal act while in progress.”
Included in the Cross Match kit is the small form-factor, lightweight Thermite TVC-2.0 Model 1000. The Thermite TVC incorporates PC compatibility, video capture and display, graphics capabilities, I/O options such as GPS, and mil-spec ruggedization, making it well suited to man-wearable 2D/3D graphics and video-intensive deployed unmanned aerial vehicle/unmanned ground vehicle (UAV/UGV) control, mission-planning, biometric, computer-assisted maintenance, and command, control, communications, computer, intelligence, surveillance, and reconnaissance (C4ISR) applications.
DRS Technologies’ MSTAR low-power, ground-surveillance radar provides 360-degree surveillance night and day and in all weather conditions. The MSTAR locates moving targets and classifies them as personnel, tracked or wheeled vehicle, helicopter, or sea surface vehicles.
Thermite TVC systems are designed with embedded computing and mobile graphics with video-capture and display options, extended-environment packaging with conduction-cooling technologies, and support for Linux and Windows operating systems to enable the rapid migration of desktop and notebook PC applications to deployed environments, says a Quantum3D representative. The man-wearable computer offers mobility, long battery life, ruggedization, and real-time visualization capabilities, of benefit to homeland security, military, and law-enforcement personnel.
Smith sees tremendous opportunities for more sophisticated, deployed processing in the field, as well as for back-end analysis in the command center, going forward. “We will also see a lot more surveillance information from systems on the move, such as UAVs and organic resources in squad cars, opposed to just stationary cameras,” he predicts.
Deployed developments
Video surveillance is a key element in systems used for border enforcement, seaport security, and critical infrastructure and key resources (CI/KR) protection, notes Joe Tillison, technology director, Avnet Electronics Marketing, North America, in Phoenix. “Adding sophisticated onboard processing to the traditional surveillance camera enhances it from merely a remote set of eyes to an intelligent system that can automate and assist in the recognition and identification of threats.”
These types of equipment are being used to automate mundane security tasks, such as watching crowds, off-limit areas, after-hours access, and anywhere motion in the field of view needs to be monitored for illicit activity, adds Tillison. Today’s solutions even can take advantage of facial-recognition algorithms to search crowds at controlled ports of entry for known persons of interest, he says.
Complementary metal-oxide semiconductor (CMOS)-based image sensors have matured, driving down the cost and power requirements of image capture and replacing older charge-coupled devices (CCDs). Low-power processors with sophisticated video processing features have also evolved. “These two things combined have enabled the rise of portable ‘smart cameras’ that can operate sophisticated software analytics onboard the camera itself, and then report alarms remotely using IP networking facilities,” says Tillison. “The cameras’ ability to perform object detection, crowd pattern analysis, secure zone intrusion detection, and so on boost the efficiency and accuracy of a human operator who is likely monitoring multiple banks of displays.”
Avnet released the Video Surveillance Design Kit to equip engineers with the latest innovations in processing and image sensing. The kit, combining CMOS image sensors from Micron Technology in Boise, Idaho, with a dual-core Blackfin processor from Analog Devices in Norwood, Mass., provides a hardware-development platform for professionals designing smart cameras with IP networking capability. The kit is designed to help developers combine control and signal processing with CMOS image sensors to analyze digital video for motion detection, threat analysis, and alerting capability—creating smart camera applications.
The hardware elements in Avnet’s kit combine with customers’ software IP to fit different video analytics applications for homeland security and defense. A fixed, remote smart camera with networking capabilities can continually monitor for incursions in a buffer zone and provide alerts with images to nearby border patrol officials, explains Tillison. Further, onboard video analysis coupled with a wide dynamic range sensor, such as Micron’s MT9V032, enable a smart camera to adjust to changing light conditions and use sophisticated pattern/motion analysis to avoid false alarms caused by wildlife or other errant sources.
“With its onboard compute power, a smart camera provides untiring vigilance in its search for known threats which might otherwise escape immediate attention,” Tillison continues. “This automated real-time analysis expands the reach and proficiency of enforcement agencies. Furthermore, once the video data is in the digital domain, the information can be easily and remotely stored or shared between agencies using existing IT (information technology) infrastructure.”
Although the benefits of networked smart cameras are many, they are not without technology limitations, namely network bandwidth and processing throughput. People must make trade-offs in camera resolution, frame rate, and data-compression techniques due to limitations in network bandwidth and processor performance.
Intelligent encoders
“Simply getting video to the central monitoring station is only the start of the problem,” says Mark Oliver, director of product marketing at Stretch Inc., a fabless semiconductor company making software-defined processors, in Sunnyvale, Calif. “To be effective, it must be disseminated to the field where it can be used to best effect.”
The compute power and resolution requirements of viewing devices in the field vary widely, presenting a challenge. “Current CODECs (coder/decoders) do not provide the flexibility needed to dynamically change the resolution, frame rate, and bandwidth requirements without computationally expensive processing,” Oliver continues. “The future for the industry will be a new generation of scalable video CODECs, the output of which can be directed to handheld devices and high-definition monitors. IP (Internet protocol) technology makes wide-scale video surveillance and long-distance transport of the resulting video possible.”
Continued growth in the number of video streams also makes monitoring difficult. This challenge sets the stage for the introduction of analytics to guide the monitoring activity and increase its effectiveness, according to Oliver. “Closer coupling of analytics with the camera electronics will enable bit rates and image processing to optimize data capture and storage, as well as bandwidth consumption.”
Technology advances are likely to overcome these impediments to the widespread adoption of smart surveillance devices in deployed defense and security applications. The use of smart sensors and smart cameras, as well as associated IT hardware and analytical software, will not only grow, but accelerate in the foreseeable future for homeland security and defense. “Evermore sophisticated video surveillance systems will continue to be a critical component of any security strategy,” predicts Tillson, “protecting property, and people.”
