Helmet-mounted displays are ready to leave the laboratory and head for the field
Display companies are near to perfecting the kind of flat-panel technology necessary for helmet-mounted displays, yet integration proves elusive
By Chris Chinnock
A little more than three years ago, engineers were integrating active-matrix liquid crystal display (AMLCD) technology from Kopin Corp. in Taunton, Mass., and active-matrix electroluminescent (AMEL) display technology from Planar Systems Inc. of Beaverton, Ore., into the first helmet-mounted displays (HMDs). Experts said designers could move these early miniature flat panels, which were smaller than an inch in diagonal, quickly into operational HMD systems.
Early optimism, however, has given way to a cautious re-evaluation of the utility of existing flat-panel display technology in helmet-mounted displays. Integration has come slowly. Even though today officials from Kopin and Planar have given their miniature flat-panel technologies operational production status, so far their use in HMDs has only been experimental. It now appears that production of HMDs with miniature flat panels is still two or three years off.
Some of the blame for slow HMD integration and acceptance comes from the way military officials develop and procure electronics. "Miniature flat panels are components for subsystems, such as head-mounted or heads-up displays, which themselves must be interoperable with larger systems onboard a military or aerospace platform," explains Henry Girolamo, a program manager at the U.S. Army`s Soldier Systems Command in Natick, Mass. "Everything must be analyzed, tested, and verified before a new technology can be fielded. As a result, military technology is often about five years behind commercial technology."
Money for advancing miniature flat-panel technology - sometimes called micro-displays - has dwindled over the last few years. Leaders of the U.S. Defense Advanced Research Projects Agency (DARPA) in Arlington, Va., are spending much less than they did only a few years ago. Meanwhile, Technology Reinvestment Programs (TRP) and advanced - technology programs have all but disappeared.
Compounding the situation are conditions at the U.S. Defense Department`s digital battlefield initiative. While leaders of this program will be primary users of HMDs, their technology development has slipped a year or two, and top Pentagon officials are diverting precious development money to support operations in Bosnia. All these factors are giving rise to working groups where military end-users can influence and pay for technology for specific needs across several platforms.
Despite these challenges, Girolamo says he is proud of the progress his "skunk works" team of rapid-development contractors has made in moving micro-display technology into military applications. As an example, he points to the U.S. Army Land Warrior program, where leaders are set to make a production decision in about a year and a half. "If they move ahead with the AMEL display that is part of the soldier`s helmet-mounted information system, they will only be about two years behind commercial technology," Girolamo says.
A successful evaluation of the HMD in the Land Warrior program could provide a springboard for much wider adoption of this technology for use on humans and in vehicles. Likewise, the Army`s Boeing-Sikorsky RAH-66 Comanche scout-attack helicopter program is shaping up to be the key evaluation platform for HMDs in many helicopter applications.
Crucial Land Warrior evaluation
Land Warrior is in place for the rapid fielding of technology intended not only to improve the deadly force that infantry soldiers wield on the battlefield, but also to improve their chances of survival. The companion Force 21 program focuses on similar needs, but is in place to develop technology for future phases of Land Warrior.
At Honeywell Sensors and Guidance Products in Minneapolis, the AMEL display is part of the Integrated Helmet Assembly Subsystem. This subsystem has day and night operation, laser detection sensors, an audio headset, and connections to the display control module contained in a backpack.
The night module works together with a conventional thermal imager. With a beam splitter, the AMEL device can overlay symbology onto the intensified image. The daytime module enables the system to present imagery from the soldier`s thermal weapon site or from a handheld daylight TV camera. In addition, the user can view maps and other computer-based data in the headset. The non-see-through display modules need only provide about 50 foot-lamberts of illumination.
Ron Lewandowski, engineering manager of display and orientation products at Honeywell, says his engineers chose the AMEL display over an AMLCD because of its wide operating temperature range of -40 to 75 Celsius, as well as its immunity to the effects of shock and vibration. "AMLCDs can cover this temperature range, but the heaters take more power than an AMEL, so they were not practical," Lewandowski says. "We are tracking over 30 companies currently developing miniature display technologies having potential for head-mounted applications."
Honeywell officials have delivered about half of the 65 Integrated Helmet Assembly Subsystems to Land Warrior prime Raytheon Systems Co. in El Segundo, Calif. This fall, soldiers of the U.S. Army 82nd Airborne Division at Fort Bragg, N.C., will evaluate the Land Warrior systems in rigorous field trials. If all goes as planned, initial production runs for the HMD should be about 5,000 units starting in late 1999. Over the 6-year life of the program, this could grow to 75,000 to 150,000 AMEL-based HMDs. "The ground-based segment is the fastest-growing part of the market right now," Lewandowski attests.
Lead Comanche role
Recently, Kaiser Electronics of San Jose, Calif., delivered to Sikorski Aircraft, of Stratford, Conn., a proof-of-principle HMD that incorporates the Kopin 1,280 x 1,024 micro-displays. Initial testing with this high-resolution HMD is now being undertaken in a simulator by a team led by Raj Kaushik, the manager of electro-optic systems. "So far, we are very pleased with the new HMD, particularly the fast video response, which we were concerned about," says Kaushik. The SXGA displays are the result of a new production line that Kopin is using in Taiwan. In June, the Army and DARPA awarded Kopin an additional $3.6 million to further development of the SXGA displays.
This fall, evaluators at the Army Air Crew Integrated Systems office in Huntsville, Ala., will decide which flat-panel technologies - direct-view and head-mounted - will be inserted into the Comanche helicopter. Many view this evaluation as critical to the future of flat panels in rotary-wing applications. Acceptance on Comanche will presage its use on other military helicopters such as the Army Boeing AH-64 Apache, Marine Corps Bell AH-1W Cobra, Bell/Boeing V-22 Osprey, Bell OH-58D Kiowa Warrior, and the U.K.`s Westland Lynx.
With Comanche, designers integrate the HMD into the helmet to enable the pilot to view imagery overlain with symbology such as air speed, heading, and threats. Imagery can come from three different sources: a head-steerable forward-looking infrared camera, a head-steerable low-light TV camera, or a miniature image-intensified charge-coupled device mounted on the helmet. The Comanche`s pilot helmet is a see-through design that enables the pilot to spend most of his time looking out the window, rather than looking down into the cockpit at his instruments.
The HMD is also a binocular device, meaning that two different sensor views are available for each eye. Data can come either from two sensors, or from a single image source through image processing. The HMD`s optics combine the two images into a 30-by-52-degree field of view with a 30-degree overlap. Two 1,280-by-1,024-pixel displays create a 1,716-by-960-pixel image, with light for the transmissive liquid crystal display delivered to the helmet through a fiber-optic bundle.
Officials of Kaiser recently delivered to designers at Sikorski a proof-of-principle HMD that incorporates the Kopin 1,280-by-1024-pixel micro-displays. Experts from a team led by Raj Kaushik, are carrying out initial testing of this high-resolution HMD in a simulator. "So far, we are very pleased with the new HMD, particularly the fast video response, which we were concerned about," Kaushik says. The SXGA displays are the result of a new production line that Kopin officials are using in Taiwan.
In general, see-through HMDs require much higher light output from the display so that users can see symbology or images against bright backgrounds. Kaushik notes that symbology is visible in daylight, but users can only view imagery on cloudy days, not in direct sunlight. Otherwise, it gets washed out. "Improving the contrast of the displays to minimize washout is an issue we are working on," Kaushik says, "but I am also concerned about miniaturizing the electronics and getting the cost of the systems down."
The next step in evaluation is to build a brass-board version of the HMD that experts can use for flight qualification sometime in 2000. While Kaushik says he believes the AMLCD technology is the best available today, he says he is ready to insert other flat-panel technologies if they prove able to meet the light output and other performance requirements for the HMD. For example, he is keeping a close eye on efforts to improve the brightness of Planar`s AMEL display.
Planar designers can produce SXGA-resolution devices with about 700 foot Lamberts of output and expect to achieve 1,000 foot Lamberts within a few months, says Bill Sproull, Planar`s general manager for AMEL. "I think 1,600 foot Lamberts is achievable within the year," Sproull says, which is the level necessary at the optics input for the Comanche HMD. Improvements in phosphors and electronic-drive circuitry are fueling these advances. Honeywell engineers are now integrating the high-brightness AMEL into their version of the Rotary Wing HMD.
Officials of the Air Crew Integrated Systems program office are also paying for a retinal scanning technology from Microvision of Seattle. Microvision officials have received a $4 million contract to develop a prototype binocular HMD system capable of displaying 1,280-by-1,024-pixel images in monochrome green. They will deliver the prototype in 1998 for evaluation on the Comanche program.
Designers also are evaluating helmet-mounted displays based on flat-panel technology for replacement of cathode ray tube (CRT)-based systems in other helicopters. At the end of May, contractors at TRACOR in Dayton, Ohio, completed the first week of flight testing of an AMEL-based HMD in a Sikorsky MH/HH-60G Pave Hawk search-and-rescue helicopter. The new AMEL system is one-third lighter and one-half the cost of the current CRT-based system, which uses a one-inch CRT and a fiber-optic image bundle to bring flight and navigational symbology to the pilot`s head and overlay it onto the night vision imagery.
"The pilots didn`t even know they had it on," asserts Roger Witkemper, TRACOR system engineer. "In fact, we had to dim the 120 foot Lamberts AMEL output because it was too bright." The new system will undergo testing for 90 days, but Witkemper says he is optimistic it will move quickly into production.
While experts are focusing most of their attention on helicopters, ground vehicle designers also are looking to develop helmet-mounted displays based on flat-panel technology. For example, leaders of the Army Battle Command Battle Lab at Fort Leavenworth, Kan., have asked Microvision engineers to develop a dual-eye, full-color, XGA-resolution monitor that M2A3 Bradley Fighting Vehicle crew members can wear like goggles. Honeywell officials say they hope to announce a new program soon to develop an HMD that provides a see-through turret capability as part of the mounted warrior program.
Slow cockpit integration
Using HMDs aboard fixed-wing aircraft arguably is the most difficult application for this technology, mostly because these displays must be bright enough to display symbology over a very bright background. Luminous outputs of 2,000 to 3,000 foot Lamberts are not an unreasonable expectation. Couple this with the safety concerns about high voltage near a pilot`s head and the need to eject from the aircraft while wearing an HMD, and the fact remains that no fixed-wing platform has yet to switch over to a flat-panel HMD. Some also cite money issues for delays in development.
Can flat-panel technologies achieve the brightness goals? "A year ago I didn`t think so," says Honeywell`s Lewandowski, "but now I am much more optimistic." As an example, he points to the progress that Planar designers are making and a new display under development at FED Corp. of East Fishkill, N.Y. FED designers are working on an organic light emitting diode (OLED) technology that holds promise for high light output in a fully pixilated format.
Boeing aircraft designers also are investigating Microvision`s laser-based HMD concepts for their Joint Strike Fighter cockpit simulator. Engineers at Saab AB of Linkoping, Sweden, are likewise using an HMD in their simulator for the JAS 39 multi-role fighter. A third, unnamed international avionics company, has taken delivery of a monochrome red binocular HMD as a retrofit to a CRT-based HMD. This device will provide a 40-by-30-degree circular field of view that helps provide acquisition information for fast-moving targets.
Wearables show promise
One of the larger growth areas for HMDs is shaping up for ground-based personnel such as special operations forces, military police, medics, maintenance technicians, inspectors, and logistics coordinators. Many of these users will link to body-worn computers, some of which will feature a voice-input interface to enable hands-free job performance.
Designers at Boeing in Huntsville, Ala., are working on two body-worn computing programs. Under the Special Operations Combat Management Systems program, Boeing engineers developed a computer based on an Intel 233-MHz Pentium microprocessor with wireless connectivity and I/O ports, that is worn in a vest. There are two display options: a handheld monochrome VGA panel or a monochrome VGA-HMD using a Kopin AMLCD. Designers have ruggedized all components to withstand water submersion and shock. The first four units are to ship to special operations forces at the end of May and will be field evaluated over the next 60 days.
A second program at Boeing, the Advanced Humionics Platform, is developing a next-generation processor together with Irvine Sensors of Costa Mesa, Calif. By stacking silicon processing layers, researchers believe they can shrink the computer to fit in a shirt pocket. Prototypes are not due until the end of 1999.
DARPA officials are also paying to develop a novel new display system for the Advanced Humionics Platform and other applications. MicroOptical Corp. of Boston has already developed three prototypes of a display system that look and feel much like ordinary eyeglasses. "We think these two programs will truly make computers melt into the human body," says Boeing`s Tom Runner, who is manager of the advanced mobile information systems group in Huntsville.
Leaders of the Army Soldier Systems Command are also buying 15 HMD units from VisionSciences of Minneapolis, a spinoff from the DARPA-SSCOM Honeywell Advanced Flat Panel HMD program. VisionSciences officials are responding to needs of field medics for light, mobile display equipment.
"Soldiers are being sent back to the U.S. for diagnostic imaging tests every time they hurt their knee, for instance," says VisionScience president Dan Cunagin. High-resolution monitors are much too big and heavy to carry into the field, so company designers are working on HMDs that will interface to laprascopic instrumentation. So far, they have built an audio/visual HMD that uses a Sony 1.3-inch miniature flat panel, but within six months they plan to build an SVGA monochrome version.
Maintenance and service of weapons is clearly expected to have a huge need for portable computer systems, some of which will require HMDs. Here, technicians will be able to access the latest schematics and procedural documentation, check inventories, and order parts - all from the field. In fact, the Army has just mandated that every future weapons platform must have embedded diagnostic capabilities.
What is particularly compelling about these applications is the broad range of non-military uses that will help drive technical innovation, volumes, and price reduction. For example, commercial truck, aircraft, and heavy equipment fleets will also benefit from the ability to access schematics and procedures on-site through body-worn computers. Experts at petrochemical plants or energy-generation utilities must perform inspections, which can be voice-input directly into PC-based template reports. And the list goes on.
Principle suppliers of these wearable or body-worn computers include Via of Northfield, Minn., Xybernaut Corp. of Fairfax, Va., and Interactive Solutions Inc. of Sarasota, Fla. In the last nine months alone, wearable computer conferences at MIT, Federal Express, and Xybernaut are helping to build awareness of the uses of wearables and the maturation of hardware and software.
Micro-display technology is also headed into consumer products. These will be part of digital still cameras, email readers on cell phones, viewfinders on smart cards, and other applications. Many companies are also developing HMDs that will cost about $500 for their laptop computers. Another area is as an entertainment accessory for displaying movies or games. Many believe the DVD Walkman is not far off.
Today, about 30 companies, mostly in the U.S. and Japan, are developing micro-displays for these and other applications. The next few years will likely see wide introduction of a host of new products based on these miniature displays - and military HMDs will benefit because of the cost reductions and technical advancements afford by a robust consumer and commercial marketplace.
Kaiser Electronics engineers built this proof-of-principle helmet-mounted display that incorporates Kopin 1,280-by-1,024-pixel micro-displays. So far, symbology is visible in daylight, but users can only view imagery on cloudy days, not in direct sunlight, so improving display contrast to minimize washout is a top priority.
Honeywell Sensors and Guidance engineers are designing the Integrated Helmet Assembly Subsystem, which has day and night operation, laser detection sensors, an audio headset, and connections to the display control module contained in a backpack. Its active-matrix electroluminescent display device can overlay symbology onto the intensified image.
Boeing designers in Huntsville, Ala., developed a computer based on an Intel 233-MHz Pentium microprocessor with wireless connectivity and I/O ports, pictured above, that is worn in a vest as part of the Special Operations Combat Management Systems program. Options include a handheld monochrome VGA panel or a monochrome VGA-HMD using a Kopin AMLCD.
Kopin stresses offshore fab
Engineers at Kopin Corp. in Taunton, Mass., have developed an AMLCD technology that marries single-crystal silicon IC fabrication with liquid crystal processing techniques. With their approach, they fabricate control circuitry in a semiconductor foundry. Then they transfer this thin circuitry to a glass substrate where they add a liquid crystal material layer. It is a transmissive LCD technology that enables designers to build high-density devices with the incorporation of peripheral-drive circuitry right on the display.
For years, Kopin experts struggled to move their technology out of the development phase and into production. Yield issues on their semiconductor line at MIT Lincoln Labs in Lincoln, Mass., and LCD processing facility in Westborough, Mass., was responsible for slippages on several programs.
Pressed to solve these problems, Kopin leaders entered into an agreement with United Microelectronics Corp. of Taiwan. "Using their high-volume semiconductor line has allowed us to improve yields, and actually revealed some other problems in liquid crystal assembly that were masked before," says Ron Gale, vice president of research and development at Kopin.
Yields are still a relative term, however. Most production is set for the company`s popular CyberDisplay, which is a QVGA (320-by-240-pixel) that is targeted for commercial applications in digital still cameras, cell phones, and other virtual viewfinder applications. Engineers can pattern more than 300 CyberDisplays on a 6-inch wafer, but only about 30 SXGA displays will fit on the same wafer. As a result, a half a dozen particles can be devastating for a wafer full of SXGA displays, but not particularly significant for a wafer full of QVGA displays.
Michael Presz, Kopin`s general manager in their Los Gatos, Calif., design center, says that yields on the CyberDisplays are running at more than 50 percent and that a 30-percent yield is necessary for commercial viability with the SXGA displays. So far, they have done two runs on the United Microelectronics line for the 12-micron SXGA devices, but yields are characterized as "good for this stage," meaning no row or column line outs. Production of these devices is set for early next year.
Government spending has also helped Kopin to develop new liquid crystal formulations to allow a very fast response time of less then 5 microseconds. Such fast switching is necessary to enable the displays to operate in color sequential mode. This means the display is successively illuminated with red, green, and blue light at about 180 Hz to produce a full-color image with a frame rate of 60 Hz. - C.C.
Planar moves AMEL into production
Engineers at Planar Systems Inc. of Beaverton, Ore., are developing a technology known as active matrix electroluminescent (AMEL). It is an emissive display that first fabricates a pixilated silicon control matrix and then deposits phosphors on top. These emit light in response to a voltage above threshold.
Planar officials are switching their AMEL technology into full production, and are supplying VGA-resolution monochrome devices to a variety of military and industrial customers. This device features a 24-micron pixel in a 0.75-inch diagonal package that provides 1,000 lines per inch of resolution. Today, about a couple of hundred devices per month are coming off this line. For high- volume applications, prices could be as low as $100.
By this fall, a VGA color device should be available, and by early next year an SXGA monochrome display using 12-micron pixels will move into production. For Planar, a product is in production when a device is fully qualified and an ISO 9001 process controls its manufacture.
Planar officials have been the beneficiary of contracts from the U.S. Defense Advanced Research Projects Agency and two Technology Reinvestment Program (TRP) awards. They used the first TRP, which ran from 1995 to 1997, to establish a manufacturing line for the 24-micron AMEL technology. Leveraging this program, Planar and a consortium of universities and corporate partners received a second TRP that will run through the end of 1999. Under this program, Planar will establish production for the 12-micron AMEL, as well as improve gray scale and color in direct-view EL displays. Each program was worth around $16 million, with 50 percent of that money coming from Planar and its partners.
To achieve a color AMEL device, Planar designers are working on two approaches. For one, they will use a white phosphor and filter using a stack of liquid crystal shutters in a subtractive color approach. By selectively removing colors from each pixel, designers can achieve a display with 512 colors, updated at a 60 Hz frame rate. This approach does not need to sacrifice resolution to achieve color, but has low overall light transmission. Heat generation and the limited temperature range of liquid crystal materials, narrows the operating temperature range of the color performance to about -20 to 50 degrees Celsius.
In a second approach to color, Planar will use the 12-micron SXGA display with a pixilated color filter approach. This does reduce the resolution of the display to a color VGA device, but maintains the wide operating temperature range with increased luminance. These could be ready by early 1999.
Also in the works is a project that will boost the current 32 gray levels in the monochrome display to 256 levels by the end of the summer. At about the same time, Planar experts plan to move ahead with a whole new interface design for the AMEL that would feature analog drivers. Today, those drivers are digital, but many military images sources, such as thermal imagers, still output analog signals. Analog drivers would eliminate the need for on-board analog-to-digital conversion electronics. - C.C.
Planar designers are developing a 24-micron pixel in a 0.75-inch diagonal active-matrix electroluminescent emissive display that provides 1,000 lines per inch of resolution. Pictured above is the Planar MicroBrite 640-by-480-pixel display.
An eyeglasses HMD
"In the early `90`s, we dreamed about a head-mounted display device that was just like a pair of regular eyeglasses," says Henry Girolamo of the U.S. Army Soldier Systems Command in Natick, Mass. "Now we are starting to see our dream come true."
Thanks to support from Girolamo`s group, as well as from the Defense Advanced Research Projects Agency, experts at MicroOptical Corp. in Boston have developed the first generation of such a device. While still in the early stages of development, the eyeglasses HMD is already showing images in an attractive form factor.
The idea is to embed mirrors, beam combiners, and lenses all within the ordinary glass lenses of a pair of eyeglasses. A micro-display image source is located on the temple of the eyeglass frame and coupled into the embedded optical system. When the display is off, the user can see through the glasses normally, but turn the display on, and an image floats out in front of him. Moving the micro-display on the eyeglass temple can vary the image`s apparent distance from him. Even better, the eyeglasses can also contain the corrective prescription of the user.
So far, MicroOptical has developed prototypes using Kopin`s QVGA CyberDisplay AMLCD, both color and monochrome versions. Since this device is only 320 by 240 pixels, standard VGA signals must scale down to a low resolution to fit on the display. Even at this resolution however, there may be industrial and commercial applications for the device.
For most military and aerospace uses, high resolution will be imperative. Recently, MicroOptical officials finished a prototype that uses Planar`s monochrome VGA AMEL micro-display. This design provide a field of view of 16 degrees, much better than the previous eight degrees, but does require somewhat bigger optics, increasing the eyeglasses` bulk. - C.C.
Designers from MicroOptical Corp. developed a prototype head-mounted display that resembles common eyeglasses. They embed mirrors, beam combiners, and lenses all within the ordinary glass lenses. A micro-display image source is located on the temple of the eyeglass frame and coupled into the embedded optical system.
Shining the light in your eyes
The idea behind the virtual retinal display is fairly simple. Instead of creating a conventional 2-D micro-display and using optics to magnify it for viewing, why not paint the image, pixel by pixel, right onto the retina or the eye?
Designers at Microvision Inc. of Seattle, have won several contracts to develop this technology for use in military and commercial helmet-mounted displays (HMDs). By using low-power red, green, and blue lasers as the image sources, they scan light in a 2-D pattern right through a person`s pupil and image it on their retina. The effect is like looking at a high-quality video image at arm`s length.
While compact red, green, and blue diode lasers are not yet here, company engineers are developing the optical systems that will enable their insertion as the sources mature. For example, two complementary Air Force contracts focus on developing very-high-resolution and very-wide-field-of-view scanning systems that will be useful for HMD applications.
The technology was originally developed at the University of Washington`s Human Interface Technology lab in Seattle, but switched to Microvision over the last year. Efforts to shrink the scanning system have resulted in a demonstration of a VGA device that uses a micro-electrical-mechanical system to preform the 2-D scanning. This novel device is only 1 mm square but can pivot in two directions at high speed. - C.C.
FED Corp. goes for OLEDs
FED Corp. of East Fishkill, N.Y., has shifted gears from field emission displays (FEDs) to a more promising area: organic light emitting diodes (OLEDs). While the company is still seeking patents and has not revealed much about their process, some of the initial results look promising.
For example, President Gary Jones notes that they are seeing efficiencies of more than 10 lumens per watt when the display is outputting monochrome green light. At that brightness level, the process, which is similar to one developed by Kodak in Rochester, N.Y., is to have a lifetime of 50,000 hours (to half brightness).
Experts have achieved those types of results with OLEDs on glass substrates. What FED is doing is developing the process to deposit the material on silicon substrates. The idea is to have standard silicon foundries fabricate the control matrix. Then, a sublimation process deposits two organic layers, p- and n-type, to produce a diode structure on top of the silicon. FED experts are using a molecular organic material that has high lifetime, but high manufacturing costs compared to the polymer organic materials favored by some other developers.
This May, FED experts demonstrated the first VGA OLED-on-silicon device that leveraged an existing CMOS control matrix design. Company officials are now designing their own SXGA circuitry that will come out of the foundry by September.
In addition, FED officials are putting in an automated production line for OLEDs that might be making monochrome displays by the end of the year. Color OLEDs are still under development. The company has received support from the Defense Advanced Research projects Agency, the Air Force, and The Soldier Systems Command.
FED officials plan to work with partners for military HMDs, and they are also eyeing commercial applications for FPD-HMD, such as a computer display peripheral. They recently acquired Virtual Vision, an early developer of commercial HMDs with an expertise in optics and HMD designs. - C.C.
