TECHNOLOGY IN CONTEXT
PCI Express
PCI Express Form-Factors
The PCI Express switched fabric interconnect technology has been widely adopted and is supported on a large number of board-level form-factors. Not all such form-factors support all variations of PCI Express. This overview is offered to help sort out the varieties.
STEVE COOPER, ONE STOP SYSTEMS
One aspect leading to the popularity of PCI Express is that it can be used in two different ways.
CPU-to-I/O tree: The first is a standard CPU-to-I/O communications bus, performing the same function that the PCI bus performs (but at higher speeds). This architecture assumes there is one, and only one, main CPU in the system. This CPU controls everything, from basic power-up to initialization to running the main operating system. It is possible to have other CPU chips providing intelligent I/O or secondary data processing, but there is always the one main CPU.
System-to-system networking: The second usage model is a CPU-to-CPU (or system-to-system) networking model. In this model, PCI Express will compete with other networking standards, such as Ethernet, but at a much higher speed. The key to using PCI Express in a network model is the availability of Advanced Switching (AS). AS is an extension to the PCI Express base specifications that adds extra routing and protocol encapsulation information onto each PCI Express packet. This extra information is added by intelligent switch chips, allowing boards that do not include advanced switching logic to intercommunicate with others that do include the logic.
Lanes are the key to higher performance, but proliferate incompatible versions: PCI Express is defined with a number of performance options. The most significant is the ability to route various numbers of lanes. A lane is defined as two differential pairs of signals, one operating in each direction. The simplest PCI Express connection consists of a single lane, and is identified as a x1 (pronounced “by one”) configuration. To achieve higher performance, designers can optionally run multiple lanes in parallel. Allowable lane options include x1, x2, x4, x8, x12, x16 and x32.
Since each of these various lane counts involves different numbers of connections, they typically require different connectors for each lane count that is supported on the interface board. Thus, each lane count option creates a new mechanically incompatible version of the form-factor. In order to reduce the number of incompatible options, each form-factor standard has limited the number of allowable lane sizes. For example, the desktop add-in boards are allowed to use only 4 of the 7 possible lane widths (x1, x4, x8 or x16). Other form-factors are more restrictive, with some allowing only a single lane width. Currently, there are no standard form-factors that utilize the x2, x12 or x32 lane configurations.
Some form-factors allow alternate buses, creating additional incompatibilities: The key to PCI Express’ high performance versus the older PCI and PCI-X buses is the change to higher speed transceivers that utilize serial signaling over differential pairs. This technique has become widely accepted as the path to higher performance in nearly all backplane or cabled systems. With this in mind, other bus structures have been defined that operate very similarly to PCI Express. Most notable is the Rapid I/O standard. Due to this phenomenon, a few form-factors have been defined such that they can accommodate PCI Express or Rapid I/O (or in some cases other electrical standards). Allowing multiple buses within a form-factor further proliferates incompatible options. In these cases, the boards might look identical (size and connector wise), however they cannot interoperate.
Connector types: There are several connector types used within PCI Express form-factors. Edge connectors are used where the edge of the add-in board (with gold contacts plated on) is inserted into a connector on the motherboard or backplane. This is the lowest cost approach since a connector is used for only one side of the connection. Pin-and-socket connectors are considered to be more reliable and impervious to dust and vibration, and are utilized on most of the industrial form-factors. The disadvantage is that a pin-and-socket connection requires a connector on each side of the connection, thus increasing cost.
Physical size: Form-factor discussions are mostly about the size and shape of the electronics modules. Some architectures define multiple shapes used within a system, such as the classic office PC that consists of a motherboard with smaller add-in boards inserted vertically. Other form-factors choose to make each board the same size, such as the 3U CompactPCI Express form-factor. In these systems, the CPU board is the same size as the I/O boards.