RTC Magazine

RapidIO technology is a fast growing interconnect and fabric standard for embedded systems, providing increased performance, improved efficiency and lower cost. RapidIO technology is supported by a broad ecosystem of leading vendors with multiple vendors shipping production switches, endpoints, FPGAs, boards, software and systems. Serial RapidIO technology offers a high-speed physical layer that can be configured to match bandwidth requirements with different speed variants and numbers of lanes.

Serial RapidIO builds on the communication industry’s common roadmap at the serial physical layer, using a variant of IEEE 802.3 Xilinx 10 Gigabit Attachment Unit Interface (XAUI) today for 3.215 Gbits/s. For future 5 and 6 Gbit/s versions, it is using a variant of the work done on the Optical Internetworking Forum’s (OIF) Common Electrical Interface (CEI).

RapidIO architecture has no inherent limitations preventing it from scaling indefinitely into the future, following or anticipating industry requirements. RapidIO technology has evolved over the past five years to a full system dataplane fabric, with extensions completed and in progress for:

• RapidIO Flow Control Logical Layer Extensions Specification

• RapidIO Data Streaming Logical Layer Extension Specifications

• Phase I: Encapsulation and Traffic Management Framework

• Phase II: Advanced Traffic

Management

• RapidIO Multicast Extensions Specifications

• RapidIO Next Generation Physical Layer Specifications

A comparison of the data rates between the different modes of Parallel and Serial RapidIO is given in Table 1.

Flexible Physical Interface

The RapidIO logical packet description is defined to be physical-layer-independent. This means that the RapidIO protocol could be transmitted over anything from serial to parallel interfaces, from copper to fiber media. The first physical interface considered and defined is known as the 8- or 16-bit link protocol end point specification (8/16 LP-LVDS). This specification is defined as having 8 or 16 data bits in each direction along with clock and frame signals in each direction.

The 8/16 LP-LVDS interface is a source-synchronous interface. This means that a clock is transmitted along with the associated data. Source synchronous clocking allows longer transmission distances at higher frequencies. Two clock pairs are provided for the 16-bit interface to help control skew. The receiving logic is able to use the receive clock for re-synchronization of the data into its local clock domain.

Since the Serial RapidIO specification is only defined in the physical layer (RapidIO technology defines the physical layer as the electrical interface and device-to-device link protocol), most of the controller remains the same. As a result, much of the design knowledge and verification infrastructure are preserved (Figure 1). This eases system-level switching between parallel and serial links. During the initial development stages of the Serial RapidIO specifications the designers decided to preserve as many of the concepts found in the RapidIO parallel specification as feasible. The parallel specification includes the concept of packets and in-band control symbols.

These were delineated and differentiated by both a separate frame signal and an “S” bit in the header. In the serial link specification this delineation is accomplished using spare characters (“K-codes”) found in the 8B/10B encoding technique. In this way, the sending device indicates to the receiving link partner the start of a packet, end of packet or embedded control symbol using these codes.

Comprehensive Link Protocol

A unique feature of RapidIO technology is that packet transmission is managed on a link-by-link basis. In the past, with synchronous buses, a mastering device had to exchange handshake signals with the target device. These signals indicated whether a transaction was acknowledged and accepted by the target device. With an interface such as the RapidIO specification defines, it is not practical to rely on a synchronous handshake since the receive port of a link is decoupled from the sending port. Therefore, many interconnects have ignored this issue and rely on an end-to-end handshake to guarantee delivery. However, this has the disadvantage of preventing precise detection and recovery of errors and forces far longer feedback loops for flow control.

To address this issue RapidIO uses embedded control symbols for link-level communication between devices. Packets are explicitly tagged between each link with a sequence number otherwise known as AckID. The AckID is independent of the end-to-end transaction ID. Using control symbols, the receiving device indicates for each packet whether it has been received along with additional

buffer status information. Receiving devices can immediately detect a lost packet, and through control symbols, can re-synchronize with the sender and recover it without software intervention. The receiving device then forwards the packet to the next switch in the fabric, and so on, until the packet reaches its final target.

Serial RapidIO allows longer transmission distances and thus involves longer loop latencies in providing feedback between the receiver and transmitter on a link. Consequently, the Serial physical layer specification increases the number of AckID values from 8 to 32.

Additionally, the Serial RapidIO specification now defines a transmitter-controlled flow control scheme whereby the receiving port provides information to its link partner about the amount of buffer space it has available. With this information, the sending port can allocate the use of the receive buffers of the receiving port. The sending port does not have to be concerned that one or more of the packets shall be forced to retry.

PCS and PMA Layers

The Serial RapidIO specification uses a physical coding sublayer (PCS) and physical media attachment (PMA) sublayer to organize packets into a serial bit stream at the sending side and to extract the bit stream at the receiving side. This terminology is adopted from IEEE 802.3.

Besides encoding for transmission and decoding for reception, the PCS function is also responsible for idle sequence generation, lane striping, lane alignment and de-striping on reception. The PCS uses 8B/10B encoding for transmission over the link.

The PCS layer also provides the mechanisms for automatically determining the operational mode of the port as either 1-lane or 4-lane, and provides for clock difference tolerance between the sender and receiver without requiring flow control. The PMA function is responsible for serializing 10-bit parallel code-groups to/from a serial bit stream on a lane-by-lane basis. Upon receiving data, the PMA function provides alignment of the received bit stream to 10-bit code-group boundaries, independently on a lane-by-lane basis. It then provides a continuous stream of 10-bit code-groups to the PCS—one stream for each lane. The 10-bit code-groups are not observable by layers higher than the PCS.

Robust Electrical Interface

Serial RapidIO uses differential current steering drivers based on those defined in the 802.3 XAUI specifications. This signaling technology was developed to drive long distances over backplanes.

For Serial RapidIO technology, two transmitter specifications were designated: a short run transmitter and a long run transmitter. The short run transmitter is used mainly for chip-to-chip connections either on the same printed circuit board or across a single connector such as that for a mezzanine card. The minimum swings of the short run specification reduce the overall power used by the transceivers. A user can further reduce the power by lowering the termination voltages.

The long run transmitter uses larger “voltage swings” that are capable of driving across backplanes. This allows a user to drive signals across two connectors and common printed circuit board material. To ensure interoperability between drivers and receivers of different vendors and technologies, AC coupling must be used at the receiver input.

The engineer’s interconnect choices may include use of proprietary, homegrown technologies, legacy interfaces or application-appropriate emerging standard technologies. The three leading choices are Ethernet, PCI Express and RapidIO technology. While the three interconnect technologies have some similarities, they are quite different in terms of technical merit. In many cases they can be highly complementary in the overall system architecture landscape.

RapidIO was designed specifically as a widely applicable, flexible, extensible system fabric for embedded infrastructure equipment including networking, storage and communication systems. PCI Express was formulated as an improvement on the Peripheral Component Interconnect bus, primarily for the commercial computing market. Historically, PCI, because of its ubiquitous nature and the consequent economies of scale, has been adopted within embedded systems despite not necessarily providing optimum functionality. There may be a similar desire to force-fit PCI Express into applications beyond the intent of the architectural scope of that interconnect. However, this is likely to be at the expense of inferior functionality, reduced performance, non-standard bridging and more complex system design than the adoption of an application-appropriate standard, such as RapidIO technology. Ethernet developed for system-to-system local networks requires heavy over provisioning for embedded applications and lacks determinism, reliability and robust error handling.

The RapidIO fabric provides a robust packet-switched system level interconnect. It provides a partitioned architecture that can be enhanced in the future. It enables higher levels of system performance while maintaining or reducing implementation costs. A RapidIO end point can be implemented in a small silicon footprint. Proven industry-standard signaling schemes (LVDS, XAUI) are used for the physical interfaces. Error management includes the ability to detect multi-bit errors and survive most multi-bit and all single bit errors. Even with all these capabilities, the RapidIO protocol overhead and latency are comparable to current bus technologies and significantly better than local area network-based fabric technologies such as Ethernet.

The RapidIO Trade Association
[www.RapidIO.org].