RTC Magazine

The increasingly complex systems in use today require increasingly sophisticated management tools. Even a desktop computer benefits from power and fan management, and the benefits increase for larger systems. The larger and more heterogeneous a system becomes, the more data must be gathered and organized to properly maintain it. Intelligent Platform Management Infrastructure (IPMI) is one standard that has been developed to address these issues.

The central player in the IPMI architecture is the management controller (MC). An MC, typically a microcontroller, monitors sensors and provides access to identifying information for one or more components. An MC runs independently of the component that it monitors. A centralized baseboard management controller (BMC) communicates directly with system management software, and allows the operator to access all of the management information in the system. The BMC also provides a system event log (SEL) that stores critical events received from other MCs.

IPMI defines a set of messages that can be sent from one MC to another. Many of the messages relay sensor or configuration information. Other messages are specific to communication between the BMC and system software. Still other messages are used to relay message data from one chassis to another. IPMI messages may be sent using a number of protocols, of which the most important is the Intelligent Platform Management Bus or IPMB. IPMB messages are sent over an I2C bus.

Using IPMI, an operator can securely establish a connection to a local or remote BMC. He can examine serial number and version information for installed components, and get a list of recent temperature or voltage spikes from the SEL. He can configure the BMC to page him if a failure condition or other event of interest occurs. The system software to accomplish this is the same, regardless of whether the system is a desktop computer or a multi-chassis data center.

ATCA Enhances IPMI

IPMI is a broad specification, but there are areas it does not address. It has only minimal support for power management, and does not support hot-swap devices at all. It is primarily a protocol and data specification, and so is not useful for hardware integration. The AdvancedTCA (ATCA) specification from the PCI Industrial Computer Manufacturers Group (PICMG) was designed to address these issues.

The ATCA specification focuses on modular computing components. It defines chassis, shelves and node boards. A chassis holds one or more shelves. Each shelf has a fan tray, a redundant pair of power supplies and slots for up to 16 node boards. Node boards may provide power and signals to rear transition modules. The ATCA specification defines connectors, voltage levels and physical dimensions for all of these components, so that components from different manufacturers will interoperate. It also extends IPMI by adding messages to support hot-swap, power control and electronic keying, or E-Keying.

ATCA defines a shelf management controller, or ShMC, which provides much the same functionality as a BMC in an IPMI system. It controls and monitors all the subsidiary MCs in a shelf, and consolidates the configuration information and sensors for all of the managed components. It communicates with the other MCs via dual-redundant IPMB, which can be either a single bus or a radial configuration (Figure 1). System software communicates primarily with ShMCs. ATCA also defines an Intelligent Platform Management Controller, or IPMC.

An IPMC typically resides on an ATCA node board, which it controls and monitors. An IPMC relays sensor events to the ShMC, and performs power and E-Keying functions in response to messages from the ShMC. A related PICMG document, the Advanced Mezzanine Card Base Specification, describes a type of node board called an AMC Carrier, which supports up to eight hot-swappable AMC modules.

The IPMI specification allows third parties to define additional types of sensors as needed. ATCA defines a hot-swap sensor type to facilitate hot swapping. Every hot-swappable entity has a hot-swap sensor associated with it. This sensor generates an event when the entity transitions from one hot-swap state to another. For example, when an ATCA node board is inserted into a running shelf, the IPMC on the node board sends a hot-swap event to the ShMC. The event message indicates that the ATCA node board has transitioned from the “not present” state to the “present” state. Although it is normally implemented in software, the hot-swap sensor can be read just like any other IPMI sensor.

Electronic Keying

ATCA defines a very flexible system for interboard I/O. The majority of the pins that connect an ATCA node board to a shelf are differential signal pairs. The signals are connected to a backplane in the shelf and are grouped into channels that in turn are grouped into interfaces. There are three interfaces: base, fabric and update channel. The base interface is a 10/100/1000BASE-T dual-star network among two hub boards and the remaining 14 ATCA node boards. All shelves must implement the base interface, so node boards have a standard way to communicate with the outside world.

The update channel interface consists of ten differential pairs, and is typically used to connect two adjacent node boards. It may be used for redundancy interlock or to allow two boards to exchange large volumes of traffic with no impact to the rest of the system. The fabric interface is a general-purpose interconnect, which can be routed in many different ways. The ATCA specification describes channel routing assignments for a variety of backplane configurations.

The channels in the fabric interface can potentially implement many different interconnect technologies. The ATCA specification lists Ethernet, InfiniBand, StarFabric and PCI Express, and refers to these as “link types.” In addition, it provides a way for OEMs to specify new link types using globally unique identifiers, or GUIDS. A link type in combination with a set of channels is called a link. The ability to implement arbitrary links on a node board leads to potential communication problems between node boards that are using the fabric interface. The solution is electronic keying.

When a node board is introduced into an ATCA shelf, the ShMC queries the node board to determine what links it uses. The ShMC then uses its knowledge of the shelf’s backplane configuration to determine which links are connected to compatible links on another node card. The ShMC sends commands to both node boards to enable every link that is connected to a compatible link. All other links on the newly inserted node board are explicitly disabled by commands from the ShMC. This scheme allows new interconnect technologies to be deployed safely without modifying the shelf hardware or software. A similar technique is used to interconnect AMCs on AMC Carriers.

As you can see, the ShMC has a lot to keep track of. It must manage state, power and sensor information for up to 16 node boards and over a hundred AMCs. It must deal with E-Keying and log and forward events. It must adjust fan speeds in the fan tray according to temperatures reported by node boards. It must be able to deal with one or more encrypted streams of messages from system software, and manage the associated user and password information. Node boards require varying levels of complexity from their management controller, with AMC Carriers on the most complex end of the spectrum. AMC management controllers (MMCs) have the lowest level of complexity, though this depends somewhat on the AMC payload.

Supporting ATCA

Would-be manufacturers of ATCA hardware face a familiar decision. On the one hand there are the integration issues and licensing fees associated with using a third-party management controller. On the other hand are the cost and risk of implementing a management controller from scratch. Either approach can divert resources from the main development effort. The relative importance of the costs will vary from product to product. Licensing fees for example are more important in a low-cost, high-volume product.

Vadatech’s management controllers, for example, range from the VT001 for shelf or VT020 for AMC Carrier management (Figure 2), to the VT027 and VT026, for node board and AMC Module management, respectively. These management controllers can be configured and integrated by the manufacturer or by Vadatech. The VT001/VT020 software was developed in C++, and was designed to accommodate different hardware configurations without requiring a change to the executable binary. The configuration files provide a layer of abstraction from the hardware. Internally, extensive modularization and abstraction of sensors and other devices provides another layer. The configuration files assign a unique set of resources to each process.

Vadatech
Henderson, NV.
(702) 896-3337.
[www.vadatech.com].