EPIC + PCI Express = EPIC Express: The Industrial Computing Platform’s Next Generation

When EPIC—the Embedded Platform for Industrial Computing—was introduced, it called for a future upgrade to the specification to support a serial fabric. PCI Express technology has emerged as the next-generation I/O solution of choice in many of the computing and communications industries as it is now migrating from the desktop to embedded applications. The PCI Express architecture uses the familiar software and configuration interfaces of the conventional PCI bus architecture, but provides a new, high-performance physical interface while retaining software compatibility with the existing conventional PCI infrastructure. The reason for adding PCI Express was to provide a “bridge to the future” while maintaining legacy support for the vast number of PC/104 expansion modules available worldwide.

Of the PCI Express hardware options on the market to date, none allow multiple boards with PCI Express to be stacked on top of each other. A stacking architecture is key for keeping costs low while maintaining a rugged, self-aligned group of boards. It also eliminates the need for a card cage or other external mounting frame.

Many questions arose about how to implement a PCI Express solution on EPIC. Should stackable, add-on legacy PC/104 I/O boards continue to be supported? If so, should both the PC/104 and PC/104-Plus expansion connectors remain? Is there a commercially available high-frequency connector that will allow PCI Express signals to be supported? If so, how many boards can be in a stack? Have the transmission line characteristics been modeled? Can a modular stack be defined that is position-independent yet requires no slot or address jumpers? Also, the enhanced capability of PCI Express is new and different from the legacy, parallel PCI bus architecture in that it supports scalable link widths in 1-, 2-, 4-, 8-, 16- and 32-lane configurations. Which lane widths should be supported and how many channels of each?

The five embedded SBC manufacturers that defined and created EPIC (Micro/sys, Octagon Systems, VersaLogic, WinSystems and Ampro Computers) conducted extensive technical discussions to hammer out these issues in order to fully define EPIC Express. The result was a definition that offers both an evolutionary and revolutionary solution. It is evolutionary since it supports legacy I/O, yet revolutionary since it is the first stackable PCI Express solution. Its PCI Express architecture is a state-of-the-art serial interconnect technology that keeps pace with recent advances in processor and memory subsystems, providing a general-purpose interconnect of choice for a wide range of applications, including graphics, storage, networking, etc.

It is also important to note that the PC/104 connector was maintained because of the number of I/O modules available worldwide from hundreds of suppliers. For low-speed, simple I/O such as relays, digital I/O, low-speed communications and certain A/D functions, this is the easiest, lowest cost and most straightforward implementation for a system designer. They can either select an off-the-shelf unit or design their own.

An EPIC board with the addition of PCI Express becomes an EPIC Express board. None of the dimensions or I/O Zones of an EPIC board change. The only difference is the replacement of the parallel PCI bus (as implemented through the PC/104-Plus connector) with a serial PCI Express connector. Similarly, the I/O boards that stack on top of EPIC Express are PC/104-size with the PC/104-Plus connector removed and replaced with a PCI Express stacking connector. Some have dubbed these “PC/104 Express” modules (Figure 1).


EPIC Express Implementation

With PCI Express, the number of serial lanes to a device can be increased to address current and future bandwidth needs of I/O devices. The PCI Express Base Specification defines the configuration of serial links as x1, x2, x4, x8, x16 and x32. A PCI Express link can be scaled on a device-by-device basis to meet different I/O controllers’ bandwidth and application objectives. It should be noted that the bandwidth of a x1 PCI Express link is nearly quadruple that of a 32-bit wide, 33 MHz PCI bus (as implemented with PC/104-Plus) with about one-fourth the number of pins. The EPIC Express technical committee considered both the application environment and practical implementation aspects of future bandwidth demands for small, industrial, stackable I/O modules and decided to support four x1 and two x4 links. However, one x16 link will be defined in the future to handle video applications.

There are two configurations that are currently defined and supported. The first is a replacement for existing, standard PCI devices using four x1 links (A-D). Similar to PC/104-Plus, it allows up to four boards to be stacked, yet requires only a single 28-pin connector. The connector is about 1/3 the size of a PC/104-Plus connector, which frees up valuable board real estate, quadruples the bandwidth of the link and eliminates the slot selection switch. This is called the Standard EPIC Express configuration.

The second configuration supports more bandwidth-intensive controllers and more PCI Express links. It supports the four x1 links on the first connector bank and the two x4 links (E &F), plus more clocks, power and ground. This adds two additional connector banks that take up approximately the same amount of area as a PC/104-Plus connector. This is called the Full EPIC Express configuration since it has more capacity and requires a larger connector with three 28-pin banks (Table 1).

The spacing between modules in an EPIC Express stack is maintained at 0.662-inches. This permits the same spacers and PC/104 cards to be used. The x4 EPIC Express expansion module must be closest to the root SBC. Stacked above that would be any x1 lane modules and finally PC/104-compatible modules. Even though the Standard and Full configurations require different EPIC Express connectors, a 1-bank, Standard EPIC Express Module can plug into a 3-bank Full EPIC Express Module but not vice versa. The reason is that a 3-bank connector is simply a 1-bank connector replicated two more times in a contiguous housing to support additional pins (Figure 2).


Automatic Link Alignment

One of the design goals of EPIC Express modules was that they not require any jumpers for address or slot alignment. The stacking design is physically similar to PC/104 and PC/104-Plus but the connectors employed are not through-hole style. Those were designed to implement a passive, parallel bus. By contrast, EPIC Express uses a pair of surface-mount connectors that allow one or more PCI Express controllers to be mounted on the I/O expansion module. This feature allows automatic link alignment, which eliminates the need for jumpers or a special stacking order. Boards not supporting PCI Express simply pass the signals up the stack from one connector to another.

An EPIC Express module with a single x1 PCI Express controller is always wired to Link A on the bottom side connector of its board. The top connector will have Link A wired from Link B on the bottom connector. The same methodology is maintained for Links B and C on the top connector since they are wired from Link C and Link D, respectively, on the bottom connector. Link D on the top connector is not connected.

An EPIC Express module with two x1 PCI Express controllers is always wired to Link A and Link B on the bottom side connector of its board. Therefore, Link A and Link B on the top connector will be wired from Link C and Link D, respectively, on the bottom connector. Link C and Link D on the top side connector will not be connected.

An EPIC Express module with a single x4 PCI Express controller is always wired to Link E on the bottom side connector of its board. The top connector will have Link E wired from Link F on the bottom connector. Link F on the top side connector will not be connected. Links A, B, C and D will be wired from the top connector to the bottom connector since there is not a x1 PCI Express controller on the board (Figure 3).


EPIC Express Connector

PCI Express is a high-speed, low-voltage, differential serial interconnect that allows two devices to communicate with each other. It is designed to offer higher bandwidth with fewer pins. Each link has a full-duplex, transmit and receive pair of signals operating at a signaling rate of 2.5 Gbits/s. This speed of 2.5 GHz results in 2.5 Gbits/s for each direction that provides a 250 Mbyte/s communications channel in each direction (500 Mbyte/s total).

The connector is the enabling technology that allows PCI Express to be implemented in a stack. Samtec’s Signal Integrity Division was instrumental in the definition and design process since they had done extensive work with their “Final Inch” differential connectors. They had worked with HyperTransport and were able to bring the same technology developed for their QTE/QSE connectors. They worked closely with the EPIC Express technical group and designed a special test rig and conducted testing to verify performance. The result is that a connector was specified that allows up to four boards to be stacked on top of an EPIC Express baseboard while maintaining proper electrical transmission line characteristics to support the data signaling rates.

Each connector bank is a 40-pin device with every third pin removed to support differential high-frequency signaling and a ground plane in the middle. The Standard configuration supports four x1 lanes and is a subset of the Full configuration. It is 1/3 the size of a Full connector, provides up to 16x the bandwidth of PC/104-Plus (parallel PCI bus) and has the same pin out for pins 1-40 (Figure 4). The Full configuration supports the four x1 and two x4 lanes on a single connector (Figure 5).

The assignment of the pin locations is important. They were selected to facilitate optimum routing of the PCI Express signal pairs as they are passed up the stack. Preliminary board layouts using the Intel-recommended PCI Express routing guidelines were conducted to ensure that the differential signaling environment could be maintained. This is necessary to minimize impedance mismatch, reflection and flight time, which degrade signal quality at these frequencies.

Generation 1 of PCI Express x1 links does not need a clock, but future generations double the serial data rate, which may require a clock, so pins are defined on the connector. Therefore, a clock is included for each link in the EPIC Express pin definition.

The transmit and receive pairs for the links, clocks for each link, power and reset are supported. However, PCI Express card Presence Detect and Wake are not. These two signals are for hot swap applications. Since EPIC Express expansion modules bolt together, it is not logical to implement hot swap. SMBus and JTAG are not currently supported. The +3V power rail is not supported since most all EPIC SBCs are +5V only. Therefore an EPIC Express expansion module will generate the requisite onboard voltage from the +5V rail.

On a Standard EPIC Express implementation, the first bank connector contains four x1 data links (A-D), three clocks (A-C), +5V, ground, auxiliary 3.3V and reset (PERST#). A Full EPIC Express implementation adds two x 4 links (E-F), three clocks (D-F), more +5V power pins, +12 volts, -12 volts plus four reserved pairs for possible functions to be determined in the future.

The designers of EPIC Express are delivering on their promise of upgrading to the latest technology. EPIC Express defines the addition of a PCI Express connector to both the EPIC SBC and PC/104-sized I/O expansion modules. The compact and incremental nature of EPIC SBCs using PC/104 expansion modules has proven beneficial in a wide range of embedded applications including test equipment, medical instruments, communications devices, transportation systems, military/COTS, data loggers, security, robotics, semiconductor manufacturing instruments and industrial control systems.

Unlike the other mid-sized boards, ample board space has been reserved to support the broad base of PC/104 I/O modules as well as PCI Express without sacrificing I/O capacity. Due to this architecture, compliant implementations can smoothly migrate from legacy ISA-based systems to the fast serial interfaces of the future. For more detailed information and a copy of the EPIC Express specification, visit www.epic-express.com.

WinSystems
Arlington, TX.
(817) 274-7553.
[www.winsystems.com].