Serial Switched Fabrics

Leveraging PICMG 2.16 Packet Switching in VME64x Systems

VITA 31.1-2003, Gigabit Ethernet on a VME64x backplane, is one of the newest, ANSI-approved additions to the VME family of specifications based on the IEEE 1101.10/1101.11 mechanical standards that are used for rugged, reliable systems in applications for Military/Aerospace, Industrial and Telecom markets, among others.


  • Page 1 of 1
    Bookmark and Share

VITA 31.1 provides a standardized way to implement an Embedded System Area Network on an enhanced, and compatible, VME backplane without the need for external Ethernet cables, while maintaining standardized backplane connector pin-outs. Operating independently over the P0 connector, VITA 31.1 provides a 10/100 or Gigabit link between any VITA 31.1-compatible boards within the chassis. This enables a tried and tested method of implementing a Local Area Network-based multiprocessor architecture by leveraging readily available Ethernet hardware, TCP/IP software and clustering, and other network management tools.

Using standard networking techniques offers a simpler, more efficient method of remote booting boards within the chassis, and simplified intercommunication between boards, particularly where different operating systems and CPU architectures are being used within the same chassis. It can also be used as a method of offloading tasks from the standard VME backplane, thereby improving overall application efficiency. This would be particularly suitable for applications such as data acquisition, radar data processing or applications requiring accurate time synchronization across multiple boards within the system.

A traditional method has been to connect VME boards to an external Ethernet switch. The configuration utilizes all the benefits of a Local Area Network (LAN), whereby the boards can communicate with each other across the LAN. Benefits of a LAN include multiple virtual communication paths, throughput and improved reliability. However, the disadvantage is that the cabling and the Ethernet switch are external to the VME chassis (Figure 1).

To overcome this disadvantage VITA decided to standardize on a much simpler and more reliable connection method. Allowing the Ethernet ports to be accessed via specific pins on the VMEbus’ P0 connector enables all the Ethernet signals to be routed, in copper traces on the backplane, to a slot(s) that contains a 6U-size Ethernet switch. Rather a VME switch, the new scheme uses a 6U CompactPCI switch that complies with the PICMG 2.16 standard for which standard hardware is readily available. The VITA 31.1 standard requires a new backplane that includes the traces for the Ethernet links as well as some slots for CompactPCI Ethernet switches. The switch uses one slot leaving up to 19 other slots for VME node boards. In redundant systems two switches will be needed.

This new VITA 31.1 backplane, which contains VME slots and CompactPCI slots, is compatible with existing VMEbus systems. In VITA 31.1, the boards in the VME slots are known as “node boards” and the boards in the CompactPCI slots are know as “fabric boards”. Figure 2 shows how node boards are connected to a fabric board. The fabric board normally contains many high-speed switches that allow any node to talk to any other node at full Ethernet speeds. This high-reliability method of communication has a single point of failure—the fabric board. To overcome this VITA 31.1 supports a dual redundant system with the use of two fabric boards (Figure 3). The VITA 31.1 backplane supports all the additional Ethernet link traces.

Switched Fabric Architecture

Today, the 6U x 160 mm VME board is more often a self-contained computing node. Peer-to-peer communications between these intelligent devices is better supported in a switched serial environment than over a bus. The interconnected mesh, as provided by the Ethernet switch shown in Figure 1, could be referred to as a switched fabric and the network topology configuration as a star.

The use of IEEE 802.3 10/100/1000 Base-T Ethernet as the data link layer is part of the PICMG philosophy of using industry standard, well-understood technology, and adapting it for reliable, rugged systems. Ethernet was chosen as the data-link layer because it is cheap, simple, reliable and the software is well proven. These are the same reasons why Ethernet is the networking technology of choice for networks worldwide.

So, VITA 31.1 defines a switched fabric architecture, as contrasted with a bus architecture (e.g., VME). A switched fabric is an interconnected network of switching devices and the topology is a star (not a bus) as shown in Figure 2. The switched fabric architecture supports point-to-point connections (or links) from every node board to the centralized fabric board(s). System performance is not diminished with extra nodes. The overall performance is scalable, as each node board can utilize the full bandwidth of its own point-to-point link. Plus, the availability of two fabric slots/boards supports redundant links, increasing the system reliability.

Scalable Performance

Since the VITA 31.1 specification, like PICMG 2.16, is designed for packet-based traffic, that traffic can be routed across a link from one node board to any other node board via the fabric board by way of a packet switching backplane. For each link there is a full-duplex Gigabit connection between each node board and each fabric board (two pairs for transmit and two pairs for receive, as well as grounds). The VITA 31.1 specification is

designed for a 21-slot chassis. Therefore, a VITA 31.1 chassis can support a maximum of two fabric boards and 19 node boards. Thus the total bandwidth for the system is up to 40 Gbits/s.

Like a Local Area Network, the switched fabric architecture can cater to different speed Ethernet links specific to a given node board. The node boards and fabric boards use auto-negotiation (as defined in IEEE 802.3) to select compatible rates between 10/100 and 1000 Mbit/s Ethernet.

Redundancy and Reliability

To improve a system’s reliability, and hence availability, the objective is to eliminate single points of failure. The VITA 31.1 specification is designed to carry significant amounts of traffic, and so reliability is a key issue. The VITA 31.1 backplane is designed to accommodate up to two fabric boards. Each node board can connect to the two fabric boards in a dual star topology with one fabric board used as a redundant system element (Figure 3). If one fabric board fails, traffic can be routed through the secondary board without the system going down.

Some system designers will refer to the term “high availability”. The full requirements of a high-availability system are hot-swap capabilities, fault tolerance and failover, redundant internal communication between system components, and software to monitor and control redundancy. VITA 31.1 does not cover all the requirements of high availability. VITA 31.1-2003 defines the pin assignments and interconnection methodology for implementing a 10/100/1000 Base-T Ethernet switched network on a VME64x backplane.

VITA 31.1 Backplane

VITA 31.1 is based on the PICMG 2.16 CompactPCI Packet Switching Backplane specification, and adopts the same connectors and pin-out for the fabric slots that are used in PICMG 2.16. This allows VITA 31.1 applications to utilize PICMG 2.16 switched fabric boards (i.e., Ethernet switch) that are already commercially available from multiple vendors. It is also interesting to note that the VME bus on P1/J1 and P2/J2 is optional. In fact, these backplanes could have no VME bus whatsoever, relying on the point-to-point Ethernet switched fabric for data transfer

The backplane for VITA 31.1 is designed for backward compatibility with legacy VME boards as well as providing a standard for specifying a set of differential pin-out connections on P0/J0 for one or two Gigabit Ethernet links. The impact on available I/O pins is minimal. Node slots/boards utilize pins on the

2 mm HM P0/J0 connectors, while fabric slots/boards utilize pins on the 2 mm HM P3/J3 – P5/J5 connectors (PICMG 2.16 CompactPCI fabric slot pin-out using 2 mm HM connectors). To ratify this as a standard, ANSI approved VITA 31.1-2003 Gigabit Ethernet on VME64x in June 2003. This standard defines the pin-outs of two Ethernet connections on the P0 connector (Figure 4).

There is no doubt that some applications will benefit from moving toward switched fabric architectures, like VITA 31.1 and PICMG 2.16, to supplement and possibly replace traditional buses. Switched fabrics are attractive because they abstract the data transfer and get away from the software complexities of address register mapping. They also minimize the number of interconnect signals required and eliminate bus loading skew problems that limit the performance of parallel buses. A bus is also a single point of failure; it only takes one VME board to fail and hold up the VME bus.

Concurrent Technologies
Ann Arbor, MI.
(734) 971-6309.