The last major shift in standard interconnect technology involved the mass
deployment and adoption of the PCI interconnect in the early 1990s. While shifts
of this magnitude do not occur often, the market is about to witness another
significant transition (Figure 1). This transition will affect two previously
separate markets, computing and communications, with the adoption of a single
Due to the widespread adoption and implementation of the PCI bus and the resulting
investment in hardware and software, the industry is leaning toward an evolutionary
rather than a replacement technology to protect their investments. In July 2002,
the PCI Special Interest Group (PCI SIG) released the PCI Express specification
v1.0 to its members. The specification defines a serial bus structure for chip-to
chip and add-in card applications, the functions provided today by the PCI interconnect,
a parallel bus-based architecture.
PCI Express is a serial interconnect, which results in lower pin counts, lower
power and full duplex transmission. It provides improvement in areas of scalability,
reliability and quality of service. In terms of software, PCI Express is fully
compatible with PCI at the application level. PCI Express addresses many of the
limitations of PCI’s parallel bus-based architecture and is well positioned
to become the successor to the PCI interconnect for the PC, traditional server
and direct attached storage markets.
By providing a manageable transition from PCI, PCI Express has steadily gained
support from key industry players, who are making substantial product development
investments. However PCI Express’ greatest asset, its compatibility with
PCI, limits its use in complex systems. PCI Express is designed to operate in
systems with a single host processor connected to a multitude of peripheral devices.
It is limited in its ability to handle multiprocessor applications, found in communications,
storage and blade servers, which have more sophisticated communication models
involving multiprocessing or peer-to-peer communication.
StarFabric, which is complementary to PCI Express and provides many of the same
features, is a serial switched interconnect solution that is in production today.
Both StarFabric and PCI Express provide complete software transparency and a high-speed
switched serial PCI-compatible interconnect. PCI Express will have a physical
layer with four times the bandwidth of today’s StarFabric. Depending on
an application’s performance needs and cost requirements, either technology
StarFabric is deployed today in numerous PCI expansion products that allow flexible
system configurations. However, most of the applications that are using StarFabric
today are based on multiple processors and require peer-to-peer communication.
These applications, which include communications, medical imaging, automated test
equipment, blade servers, video servers, storage and military/industrial signal
processing, use StarFabric’s path routing capability. PCI Express does not
have the features necessary to support these types of applications because it
does not support multiprocessing, peer-to-peer environments.
During the development of the PCI Express specification, the industry realized
that a certain class of applications would require a superset of the PCI Express
features. The Advanced Switching Interconnect Special Interest Group (ASI SIG)
was formed to develop a specification that would build this functionality on top
of the PCI Express Physical and Data Link layers. Advanced Switching (AS) further
enhances the capabilities of PCI Express by providing protocols suitable for a
variety of applications, including multiprocessing and peer-to-peer computing.
In January 2004, after more than two years of specification development effort
initiated by the Arapahoe Working Group, the ASI SIG announced the approval and
release of Version 1.0 of the Advanced Switching core specification. Companies,
such as Agere, Alcatel, Huawei, Intel, Siemens, Vitesse and Xilinx were joined
by other semiconductor vendors and major players in the communications and compute
markets, all of which had expertise in developing advanced serial interconnects.
Today, more than forty companies are involved in developing the architecture through
their sponsorship of the ASI SIG. With such broad support and participation from
market leaders in the systems and silicon business, the AS specification will
now enable designers of silicon, software, test equipment and design tools to
move ahead with product implementation.
Advanced Switching Implementation
AS is currently being targeted for applications such as converged servers, advanced
storage, communication access/edge infrastructure and blade servers, which up
until now have not been well served by industry standard interconnects. Instead,
these applications have had to rely on proprietary solutions for the combination
of high availability, distributed processing, QoS features and multi Gbit/s performance.
AS is well-suited to penetrate these markets because of the economic advantages
that come with participating in an industry standard market, while utilizing PCI
Express peripherals. The development of software and hardware tools to simplify
design and manufacturing will further lower the cost of implementing switch fabric
Already, the industry is seeing announcements for new product lines based on PCI
Express and AS. StarGen, which developed StarFabric and made key contributions
in developing the AS specification, has announced a product line of switches and
bridges called StarXpress, which is based on PCI Express and Advanced Switching.
StarXpress components are targeted for backplane interconnect of high-performance
communications, storage, blade server and embedded products.
StarGen has leveraged its architectural and implementation expertise gained through
the development of StarFabric to develop its StarXpress product line. StarXpress
provides a smooth upgrade path from PCI Express to AS. StarXpress switches will
support both direct connection of AS end points and PCI Express components, as
well as hybrid platforms with PCI Express and AS components. AS fabrics based
on StarXpress switches and bridges, whether using PCI Express or AS end points,
can be deployed in storage, communications and compute applications
Advanced Switching Protocol Interfaces
AS provides a high level of flexibility for system architects, allowing a number
of different I/O protocols to share the fabric. The protocol is identified in
a header attached to the data packet. In addition, the data payload can contain
either a native AS packet or encapsulate a packet in its native format—such
as Ethernet, SONET, TCP/IP or PCI Express.
In February of 2004, the ASI-SIG announced four data transport protocol interfaces
(PIs) that will serve in a wide variety of applications. Additional PIs will be
developed by the ASI SIG Technical Working Group and made available as industry
standards over time, ensuring against obsolescence and providing for use in unforeseen
applications. Finally, vendor-specific PIs are allowed in the specification for
proprietary systems running on industry standard AS hardware. The announced Protocol
PI-8: PCI Express Encapsulation: This is the standard tunneling scheme for passing
native PCI Express packets through the AS fabric, offering the simplicity of complete
software compatibility with PCI Express peripherals within an AS environment.
A system can contain a mix of PCI Express and AS components to offer the best
features of both technologies.
PI-9: Socket Data Transport (SDT): A low overhead protocol that provides direct
hardware implementation of the well-known, socket inter-processor communication
interface’s read/readv/readn and write/writev/writen data movement model.
SDT will move massive amounts of data with minimal processor overhead.
PI-10: Simple Load/Store (SLS): An extension of the PCI load/store model that
offers a low overhead model for transporting data across the fabric. SLS provides
a simple load/store abstraction that would allow PCI, PCI-X, PCI Express, HyperTransport,
RapidIO, and virtually any other interconnect that used a load/store model to
interoperate within an AS fabric via translation of their native protocol into
the common SLS protocol. SLS is a trusted communication model that provides the
advantages of efficiency, low overhead and low latency.
PI-11: Simple Queuing (SQ): A simple messaging protocol that uses queues in place
of specific addresses to move messages across an AS fabric. SQ allows multiple
endpoints to share a single queue resource, thus minimizing the context required
for concurrent communication.
A Roadmap to Converging Markets
PCI Express components will be available in sample quantities in mid 2004, about
a year ahead of AS silicon. Initial components will be bridges to the various
flavors of PCI and switches with assorted port counts and lane widths. CPUs, chipsets
and NPUs with PCI Express ports will show up in late Q2 2004. Systems that use
PCI Express instead of a parallel PCI bus to connect to peripherals may ship as
early as Q4 2004. With its point-to-point architecture, PCI Express will offer
many advantages over the various PCI-based parallel buses in bandwidth, performance
AS is also causing quite a stir in standard interconnect technology. Numerous
AS chips are expected to be introduced in the first half of 2005, with additional
announcements continuing throughout the year. Switches, bridges to PCI and PCI
Express, as well as CPU chipsets, network and I/O processors will be included
among the initial AS products. Deployment of AS systems is expected to begin in
the second half of 2006. The availability of AS prototype silicon signals a major
change in the computing and communications interconnect world. The end result
will produce components that meet the most demanding performance, feature and
cost goals, making AS the dominant multi-point, peer-to-peer interconnect in its
performance range. Proprietary solutions will eventually be replaced with a family
of modular building blocks with wide availability and support.
PCI Express and AS are complementary technologies that share physical and
datalink layers (Figure 2). AS leverages PCI Express to provide switch fabric
functionality for multiprocessor and peer-to-peer communications and compute
applications. Their mutual success is tied to the development of an industry
standard switch fabric infrastructure. AS is dependent on PCI Express to create
the commodity marketplace for peripherals, devices, cards, cables, connectors
and tools, while AS completes the fabric solution with advanced features that
enable converged data communication solutions.
What Will the Future Hold?
PCI Express and Advanced Switching address the limitations of PCI as a system
interconnect. Wider and faster incarnations of the PCI parallel bus standards
did little to address its most serious inadequacies. The shared-parallel bus’
single point of failure was an obstacle to increasing reliability and providing
higher levels of system availability. This, combined with a limited system scale,
shared bandwidth and strictly hierarchical data routing, was impacting system
design, performance, reliability, fault tolerance and scalability.
With technical contributions and support from a wide range of interested parties
in the semiconductor, equipment and software industry, the ASI SIG has defined
the standard that, in conjunction with PCI Express, will be the successor to PCI
and will provide the interconnect technology for inter-chip to inter-chassis communication
for the next generation of converged communications and computing systems.
Steve Christo chairs the ASI SIG Marketing Working Group.