By Richard O`Connell Myriad Logic Inc.
Since its approval as an ANSI standard in June 1994, RACEway has gained acceptance as a way to increase VME bandwidth by as much as an order of magnitude. Today more than a dozen vendors offer RACEway commercial off-the-shelf (COTS) products such as digital signal processing, memory, data acquisition, and input/output equipment, based on the high throughput of RACEway, as well as on the availability of a wide variety of products, and compatibility with existing VME and VME64 systems. In high-rate, real-time systems RACEway delivers the performance other technologies promise for tomorrow, making it the architecture of choice for many major system integrators.
RACEway expands the internal bandwidth of VME and VME64 systems by providing alternate high-speed data paths with user-defined pins on the standard VME P2 connector. Using RACEway a board can establish a point-to-point, 160-megabyte-per-second connection to another RACEway-ready board in the chassis with a non-blocking cross-bar switch - the RACEway Interlink - implemented as a backplane overlay.
The Interlink dynamically switches connection requests to enable several board pairs to exchange data simultaneously at 160 megabytes per second. This makes internal bandwidth scalable in a RACEway system; every two boards enables an additional 160 megabytes-per-second board-to-board link. The internal bandwidth in a two-, four-, six-, or eight-board RACEway system is 160, 320, 480, and 640 megabytes per second respectively.
A second advantage of RACEway in real-time systems is deterministic performance. Designers can dedicate high-priority data paths to real-time functions to ensure that bandwidth is available when necessary. In a bussed system, meanwhile, real-time processes compete for bandwidth with other bus users, so it can be difficult to ensure that bus resources will be available precisely when needed.
Switched fabric interconnect
The RACEway specification, ANSI/VITA 5-1994, defines how to use the VME P2 connector`s A and C rows to get an additional, high-speed path to transfer large volumes of data between boards. RACEway uses these pins to craft a switched, point-to-point interconnect, also known as a "switched fabric." Since RACEway uses only P2 rows A and C, it can operate concurrently with VME or VME64 operations.
The central element of a RACEway interconnect is the Interlink crossbar switch - an active P2 overlay that mates to the rear of a VME backplane. The Interlink slots determine the boards that may participate in RACEway connections. Interlinks can encompass a variable number of slots; for example, Cypress Semiconductor offers four-, eight-, and 16-slot RACEway Interlink modules to configure systems in a variety of ways.
RACEway Interlink modules are generally one or more six-port crossbar switches - each switch a single chip that implements the RACEway Interlink standard. Designers can define Interlink overlays spanning as many as six slots with a single chip. They also can define Interlink overlays spanning more than six slots by cascading six-port - an effective mechanism for expanding Interlink modules, but that introduces additional latencies that the designer must consider.
A RACEway master board asks the Interlink for permission to exchange data via the fabric with a RACEway slave board by writing a routing word and slave address. The Interlink module routes the request to the requested slave board, which signals when it is ready to accomplish the data transfer. The Interlink manages actual data transfer on a 2 kilobyte burst basis, yet RACEway bursts always occur at 160 megabytes per second (32 bit synchronous transfers at 40 MHz). The Interlink, which interleaves bursts from several RACEway masters to a single RACEway slave, arbitrates access, recognizes priorities, and arbitrates requests when two or more RACEway masters attempt to access multiple-megabyte data blocks to a single RACEway slave. The RACEway specification also allows for adaptive routing, split reads, broadcasts, and multicasts.
RACEway and real time
The real-time system designer can use RACEway to channel data through the system over multiple data paths. In one example, a rotating set of three RACEway memory boards could buffer high-rate real-time data by writing acquired data to one memory board and switching to the second memory when the first memory fills, while a data capture board extracts data from the first memory for recording on high-rate tape or Fibre Channel RAIDs.
Similarly, acquisition switches to the third memory when the second memory is full, data capture switches to the second memory, and signal processors begin operating on data in the first memory. The signal processors can also use RACEway to pass data to additional signal-processing boards or to a high-performance workstation/supercomputer via a HIPPI or Fibre Channel RACEway board. In this example the VME bus is used for control only.
The above example illustrates a nine-board system with an internal bandwidth of 640 megabytes per second (via RACEway) and 80 megabytes per second (via VME64). This internal bandwidth is considerably higher than industry-standard bus systems such as PCI. While these numbers are theoretical maximums, designers can build systems today using COTS equipment that sustains several, 100-plus megabyte-per-second transfers between boards.
It is the scalability and determinism of RACEway channels that are the major advantages in real-time systems. In systems where data pass from one element to the next, scalability is vital to high throughput. In a standard bus architecture, the example system would require all boards to compete for bandwidth on one shared bus. Since the data pass from one element to another four times, the total system throughput in a bused implementation could be no more than one quarter of the maximum achievable transfer rate. In a VME64-only implementation, it would be difficult to sustain even a 10 megabyte-per-second input rate, while the RACEway extension to VME can realistically sustain 100 megabytes per second.
Deterministic performance is also important to real-time systems. Designers can dedicate individual RACEway links to real-time data paths to ensuring that the required bandwidth is available. In a bused system, the single data pathway may be occupied with a sequence of high-speed bursts or by relatively slow control messages. In typical VME64 systems, latencies due to bus contention can be one or more milliseconds. These latencies do not occur in a RACEway architecture.
RACEway building blocks
Since the adoption of RACEway as an ANSI standard, a growing number of manufacturers are supporting RACEway-ready versions of their VME products. COTS RACEway boards are available for data acquisition, digital signal processing, recorder interfaces, high-speed communication channels, and other functions. There are also chip-level products and foundation boards available to help develop application-specific RACEway boards.
Compatibility with VME is important to RACEway because it enables designers to use standard VME equipment such as system enclosures, and high-performance single-board computers with standard operating systems. Additionally, VME compatibility enables developers to re-use application- specific VME modules and capitalize on staff`s knowledge and familiarity with VME.
The variety of RACEway-ready products coupled with the ability to integrate these products with standard VME equipment enables the system designer to configure a system using off-the-shelf products. In fact, the range of VME/RACEway off-the-shelf products is broader than for any competing technology.
The HRS-2000
The HRS-2000 intelligent controller from Myriad Logic Inc. of Silver Spring, Md., is for the Sony DIR-1000 family of digital instrumentation recorders. The DIR-1000 currently operates at the fixed rate of 32 megabytes per second, with a 64-megabyte-per-second version to be demonstrated in January 1997. The HRS-2000 provides an ANSI-standard HIPPI connection between the fixed rate DIR-1000 and a wide variety of supercomputers, high-end workstations, and other embedded systems.
The HRS-2000 buffers and controls data flow between the non-real-time HIPPI channel and the real-time Sony recorder. These input/output rates typically require 100 to 140 megabytes per second transfer rates to the memory for 1 to 2 seconds at a time, with average rates of 64 and 128 megabytes per second for the 32 and 64 megabytes per second recorders, respectively.
Prior to the availability of RACEway, designers implemented the HRS-2000 using a VME64/VSB approach. A HIPPI board transferred data to one of three VME64/VSB dual-ported memories at an average rate of 45 megabytes per second (limited by VME64), while a DIR-1000 interface board transferred data from another of the memories to the Sony recorder using 34 megabytes per second bursts on VSB. Significant problems were encountered in managing access to the VME64/VSB memories while maintaining the real-time 32 megabyte-per-second rate to the recorder. Cross-talk problems were exacerbated by drive and termination specifications of the two high-speed buses operating on adjacent pins on P2.
In mid-1996 Myriad upgraded the HRS-2000 from VME64/VSB to RACEway. The resulting configuration requires only one memory board (reduced from three in the earlier version) and will support the 64 megabyte-per-second recorders. Since RACEway is not bused, the cross-talk problems experienced with the VME64/VSB version have been eliminated.
The new configuration also provides significant expansion potential. In the previous version, the VME and VSB buses were almost saturated, leaving no possibility for new features. In the RACEway version, the scaleable bandwidth enables designers to add boards and associated functionality.
Benchmarks
Benchmark tests determined realistic performance for various RACEway configurations. The first configuration consisted of two standard VME chassis, each containing one Myriad HIPPI-830R and one MicroMemories MMI 6490D. In the test the HIPPI board in chassis 1 repeatedly read fixed blocks from the memory board via RACEway and transmitted the blocks to the second chassis via the HIPPI channel. The HIPPI board in the second chassis received the blocks and stored them memory via RACEway. Testers calculated throughput by dividing the total number of bytes transferred over a large number of blocks; they included all software overhead. The results measured 91.9 megabytes per second with 1 megabyte blocks and 95 megabytes per second with 4 megabyte blocks. Since the HIPPI is a 100 megabyte-per-second synchronouschannel, achieving 95 megabytes per second was considered good.
A second test involved two HIPPI boards in a chassis with a single RACEway memory board. The first HIPPI board read blocks of data from the memory board via RACEway and transmitted the data out the HIPPI channel. The second HIPPI board received the data and wrote the blocks to another area in the same memory board. This test, which measures RACEway throughput to a single RACEway slave board from two competing RACEway master boards, measures the Interlink`s efficiency in arbitrating requests for the same RACEway slave. Performance grew with block size, rose to a maximum of 150 megabytes per second with block sizes of 4 megabytes, and illustrates that designers can sustain data rates as fast as 150 megabytes per second to a single RACEway board, and indicates that system-level throughput can be considerably higher with multiple transactions.
Contact Information:
- VITA, 10229 N. Scottsdale Road Suite B, Scottsdale, Ariz., 85253. Phone: 602-951-8866. Fax: 602-951-0720.
- Apcom Inc., Steven Neubauer, 8-4 Metropolitan Court, Gaithersburg, Md., 20878-4013. Phone: 301-948-5900. Fax: 301-948-1631.
- Atlantic Aerospace Electronics Corp., Ed Graham, 470 Totten Road, Waltham, Mass., 02154. Phone: 617-890-4200. Fax: 617-890-0224. Email: [email protected].
- Catalina Research Inc., Mike Bonato, 1321 Aeroplaza Drive, Colorado Springs, Colo., 80166-2247. Phone: 719-637-0880. Fax: 719-637-3839. Email: [email protected].
- Celerity Systems Inc., Mike Wojnar, 4000 Moorpark Avenue, Suite 201, San Jose, Calif., 95117. Phone: 408-247-3383. Fax: 408-247-3381. Email: [email protected].
- Cypress Semiconductor, Dave Horton, 3901 North First Street, San Jose, CA 95134. Phone: 408-943-2600, Fax: 408-943-6859. Email: [email protected].
- Echotek Corp., Gary Turchetta, 555 Sparkman Drive, Suite 400, Huntsville, Ala., 35816. Phone: 205-721-1911. Fax: 205-721-9266. Email: [email protected].
- Ixthos, Robert Kirk, 741-G Miller Drive, Leesburg, Va., 22075. Phone: 703-779-7800. Fax: 703-779-7805. Email: [email protected].
- MicroMemory Inc., Mose Jadon, 9540 Vassar Avenue, Chatsworth, Calif., 91311. Phone: 818-998-0070. Fax: 818-998-4459. Email:[email protected].
- Mercury Computer Systems Inc., Leigh McLeod, 199 Riverneck Road, Chelmsford, Mass. Phone: 508-256-1300. Fax: 508-256-5205. Email: [email protected].
- Mizar Corp., Keith Burgess, 2410 Luna Road, Carrollton, Texas, 751006. Phone: 972-277-4600. Fax: 972-277-4666. Email: [email protected].
- Myriad Logic Inc., Debra Benedict, 1109 Spring Street, Silver Spring, Md., 20910. Phone: 301-588-1900. Fax: 301-588-0605. Email: [email protected]
- Pentland Systems Ltd., Tracy Dorset, 1212 Baxter Drive, Plano, Texas, 75025. Phone: 214-517-9343. Fax: 214-517-0070. Email: [email protected]
- VMEtro Inc., Stacey Gonzalez, 1880 Dairy Ashford #535, Houston, Texas, 77077. Phone: 713-584-0728. Fax: 713-584-9034. Email: [email protected].
Richard O`Connell has been president of Myriad Logic since 1987, where he directed the development of Myriad`s line of VME and RACEway I/O products. Prior to joining Myriad, he held management and engineering positions in General Electric`s Aerospace Business Group where he designed satellite image processing systems. He has a BSE in electrical engineering and computer science from the University of Connecticut, and an MBA in information systems management from George Washington University.
