RTC Magazine

Designers of complex, distributed systems stand to gain the most from the emergence of switched fabrics. Table 1 shows some of the common requirements that these designers are faced with and how those requirements are typically met with existing bus backplane technologies. The complexity of software needed to implement current solutions to these common requirements increase system development and post-deployment maintenance costs. In some cases, the required complexity results in projects that fail to complete development.

Switched fabrics such as StarFabric, PCI Express Advanced Switching (AS) and Serial RapidIO give new freedoms to designers to implement their systems. In the case of StarFabric, the solutions are available today. For others, solutions are just around the corner. On VME, the new VITA 41 spec calls for the fabric interconnect to be implemented on the P0 connector. This leaves the backplane backward compatible with earlier specifications.

As the “switched fabric wars” rage and the switched fabric trade associations race to establish the physical and electrical specifications for their technologies, niceties such as the development of abstracted, full-featured communications protocols are left to others to define, and some would argue that this is for the best. An exception to this is InfiniBand, which defines a very rich API. One software solution for leveraging switched fabrics to move messages or large amounts of data with ultra-low latencies in closely coupled, multiprocessor systems was described in an earlier issue (see “Software Solutions for Interprocessor Communications,” p. 56, RTC, August 2003).

However, for applications requiring a more loosely coupled architecture supporting dynamic loading, hot failover and other advanced features, we suggest that Publish-Subscribe protocols—already popular with users of Ethernet interconnects—are an ideal choice. Switched fabric features such as Scalability, Quality of Service and High Availability fit very well with the way publish-subscribe operates.

Publish-Subscribe Communications

There are three common communication models: point-to-point, client-server and more recently, publish-subscribe. Point-to-point is like a phone call. You know the address of the remote node, establish the connection, and then communicate. A phone call and a TCP/IP socket connection are examples of point-to-point or connection-oriented communications.

The client-server model was created to help scale the point-to-point model. Multiple client nodes can establish connection to a known address where a server waits to establish connections with each client. Clients can then make requests of the server and get replies. Overlaid on the concept of software objects, client-server underlies remote method invocations. This model is well established and is the basis of Microsoft’s DCOM and the Object Management Group’s (OMG) Common Object Request Broker Architecture (CORBA) standards.

For developers building real-time distributed applications, the client-server model has some distinct disadvantages: 1) The server represents a bottleneck and potential single-point of failure; 2) The request-reply semantics require two messages to get the data for each client, which increases bandwidth load and transaction latency; and, 3) It is often based around a remote method invocation or “object-centric” design that is not suitable for many distributed real-time applications that simply need to communicate data and not objects. Shoehorning object-centric communication models into “data-centric” systems frequently leads to unnecessarily complex system designs and significantly degraded performance.

Publish-subscribe excels at real-time data distribution. Publish-subscribe is characterized by a set of data producers and data consumers. Where client-server has a request-reply form, publish-subscribe is more a “push” model. That is, after the publishers and subscribers have identified themselves on the network, the data is pushed onto the network by the publishers. Subscribers can then pull the data off the network anonymously—no requests or polling are required.

Another advantage is anonymous communications—publishers and subscribers don’t need to know each other’s physical address. This is in direct contrast to the connection-oriented communications models. The middleware keeps track of which subscribers want which data from which publishers. This makes complex data distribution patterns quite simple to program. This anonymity also makes it simpler to set up redundant publishers for fault-tolerant systems. It’s also straightforward for nodes to come into and leave the network and for applications to be moved from node to node as required in load-balanced systems.