RTC Magazine

Creating the right environment for networked, distributed applications to share data “on demand” and with proper reliability is a complex undertaking. This is even more challenging when inadequate responsiveness or errors in these “mission-critical” systems can result in financial disaster, lost production, social harm or even loss of life. The need for solutions that feature real-time, scaleable, low-overhead and fault-tolerant operation is growing faster than ever. Today the increasingly heterogeneous nature of these systems is also adding complexity, driven by the need to seamlessly incorporate Web-based, server-based and embedded applications into single systems with predictable (deterministic) and very fast performance.

Driven by the limitations of traditional client-server architectures, and following the trends toward more loosely coupled and dynamic systems, the Object Management Group (OMG) recognized the demand for a standard that would follow a different paradigm. OMG organized members with experience in both the “underlying” technologies (networking and information-management) as well as “user-level” requirements (distributed, real-time and mission-critical system characteristics), to join forces and define the Data Distribution Service (DDS) for real-time systems. DDS utilizes the powerful publish/subscribe pattern to implement an information backbone to which any application can dynamically connect in order to publish and/or subscribe information. The DDS middleware must then take care of the proper data dissemination, taking into account multiple and dynamic quality of service (QoS) requirements regarding delivery (latency, reliability) and durability (persistence).

Now that several COTS DDS implementations are becoming available, quantitative characteristics (such as functional coverage of the spec) as well as qualitative characteristics (such as footprint, performance and usability) will determine their suitability as an information backbone that actually reduces mission-system complexity during all lifecycle stages from design to deployment.

Especially within the mission-system domain, requirements for standards-based middleware solutions are becoming eminent since homegrown solutions don’t tackle the evolutionary nature of these networked systems, neither on the application level nor on the underlying technology level.

Profiles for Information Distribution

The DDS standard specifies a coherent set of profiles that target real-time information availability for domains ranging from small-scale embedded control systems up to large-scale enterprise information management systems. Each DDS profile adds distinct capabilities that define the service levels offered by DDS in order to realize this “right data at the right time at the right place” paradigm (Figure 1).

The Minimum Profile utilizes the well-known publish/subscribe paradigm to implement highly efficient information dissemination between multiple publishers and subscribers that share interest in so-called “topics.” Topics are the basic data structures expressed in the OMG’s IDL-language. They allow for automatic generation of typed “readers” and “writers” of those topics for any mix of languages desired. This profile also includes the QoS framework that allows the middleware to “match” requested and offered QoS parameters—the minimum profile offering basic QoS attributes such as reliability, ordering or urgency. The Minimum Profile specifically targets resource-constrained environments.

The Ownership Profile offers support for replicated publishers of the same information by allowing a “strength” to be expressed by each publisher so that only the highest-strength information will be made available to interested parties.

The Content Subscription Profile offers powerful features to express fine-grained interest in specific information content (content filters). It also allows applications to specify projection views and aggregation of data as well as dynamic queries for subscribed topics by utilizing a subset of the well-known SQL language while preserving the real-time requirements for the information access.

The Persistence Profile offers transparent and fault-tolerant availability of “non-volatile” data that may represent persistent settings to be stored on mass media throughout the distributed system. Alternatively, “state” data can be preserved in a fault-tolerant manner outside the scope of transient publishers, which allows the late joining of applications and dynamic reallocation.

The DLRL Profile extends the previous four data-centric publish/subscribe (DCPS) profiles with an object-oriented view on a set of related topics, thus providing typical object-oriented features such as navigation, inheritance and use of value types.

Information at the Heart

The information-centric paradigm is characterized by getting “the right information at the right time at the right place.” Even with the definitions as stated before, this still is a rather nebulous statement that requires more analysis since getting information on a pager is rather different from shooting down a ballistic missile. The three aspects of “right information,” “right time” and “right place” need to be further analyzed.

Following the definition, the right information should be explained as those related pieces of (published) data that are required by a subscriber. What is required is expressed by the subscriber’s subscription. Apart from services allowing a subscriber to express its interest in information, interfaces are needed that allow fast access to this information once it is available. When evaluating available standards and practices for accessing information, it was kept in mind that run-time flexibility is needed to achieve the goals and related evaluation criteria for dynamic and evolutionary systems. An associative information model provides the required flexibility level by allowing selective and run-time specification of autonomously produced data items and their relations. SQL as a powerful and standard technology is used to express the required fine-grained (content-filtered) interest in information using aggregation, selection and projection.

In a time-critical or real-time environment, getting information at the right time means making sure that information is made available when needed. In order to become available, the published data making up the information must be delivered to subscribers anywhere in the system and at anytime in the system. This includes late-joining subscribers. This decoupling “in space and time” requires mechanisms both for efficient and effective distribution. Applying policies to specify required QoS levels is a powerful way to separate service-quality “need,” as perceived by applications, from service-quality “offering”—the dynamic availability of enterprise resources and networks.

Information classification then implies the ability to “attach” QoS-profiles to publishers and subscribers as well as directly to the data itself, decoupled and outside the local scope of these applications so that enterprise-rules may be applied without affecting (the design of) individual applications. Finally, the right time implies fine-grained notification mechanisms to react in real-time to availability of the proper information (content). In information-critical and real-time systems, the lifecycle of the data is also an important aspect of information. Lifecycle-based selection, for instance, enables consumers to process first-time appearances of information with higher priority than updates of already known information.

Getting information at the right place requires the knowledge about where the information is needed. Publications and subscriptions serve as declarations that inform the middleware where certain data is produced and where it is needed, thus offering real-time interaction with the information. Just as in real life, information production is often asynchronous from information consumption. This fact can be exploited by providing subscribers with a private information cache. Implementing such a cache as a local in-memory database allows extremely fast access on the information of interest. There are also a number of non-functional characteristics that can help shape a DDS implementation.

To ensure scalability, flexibility and extensibility, an optimal DDS should have an internal architecture that utilizes shared memory to not only “interconnect” all applications that reside within one computing node, but also to “host” a configurable and extensible set of services. These services provide pluggable functionality such as networking, providing QoS-driven real-time networking based on multiple reliable multicast channels. They also ensure durability by providing fault-tolerant storage for both real-time state data as well as persistent settings. A remote control and monitoring “soap-service” provides remote Web-based access using the SOAP protocol. This pluggability ensures suitability in resource-constrained environments.

An implementation such as this also utilizes a shared-memory architecture where data is physically present only once on any machine, and where smart administration still provides each subscriber with his own private “view” on this data (Figure 2). This allows a subscriber’s data cache to be perceived as an individual database that can be content-filtered, queried, etc., using the content-subscription profile. This shared-memory architecture results in an extremely low footprint, excellent scalability and optimal performance when compared to implementations where each reader/writer is a communication endpoint with its own storage (i.e., historical data both at reader and writer) and where the data itself still has to be moved, even within the same platform.

Another key non-functional characteristic is to ensure that the middleware is easily configured on the fly by specifying only the needed services to be used as well as configuring those services for optimal matching with the application domain—networking parameters, durability levels, etc. Easily maintainable XML-file(s) are utilized to configure services. The middleware configuration is also supported by means of a MDA toolset, allowing system/network modeling and automatic generation of the appropriate XML configuration files.

In one implementation, for example, a 100% Java-based tool can be used to greatly aid the design, implementation, test and maintenance of the DDS implementation. During the design phase, the information model is established by first defining and registering topics in a run-time environment. This environment can be both a host environment as well as a target environment. The tool can then enable the creation of publishers/writers and subscribers/readers on the fly to experiment and validate how this data should be treated by the middleware regarding persistence, durability, latency, etc.

During the implementation phase, where actual application-level processing and distribution of this information is developed, the tool allows injection of test input data by creating publishers and writers on the fly as well as validating the responses by creating subscribers and readers for any produced topics.

During the test phase, “snapshots” can be made of writer and reader history caches. Then the total system can be monitored by inspection of data and behavior of readers and writers based on statistics, such as how long data has resided in the reader’s cache before it was read. It is also possible to monitor the data-distribution behavior in terms of memory usage and transport latencies.

Maximum flexibility for planned and “ad-hoc” maintenance can be offered by allowing the Java tool to remotely connect via the Web-based SOAP protocol to any “reachable” systems around the world as long as an HTTP connection can be established with the computing nodes of that system. Using such a dynamic connection, critical data may be logged and data sets may be injected into the system to be maintained—for example, new settings that can be automatically persisted using the QoS features as offered by the persistence profile supported by the implementation.

Today, DDS implementations are beginning to come to market with second-generation fully compliant OMG-DDS implementations that offer support for all the DCPS profiles (minimum profile, ownership profile, content subscription profile and persistence profile) as well as the DLRL object profile. In these, several run-time modules can cover the full OMG specification as well as provide total-lifecycle support by integrated productivity tool suites that target the complete system lifecycle.

PrismTech
Burlington, MA.
(781) 270-1177.
[www.prismtech.com].