Shelf Management

Hardware Platform Management Building Blocks Speed ATCA and AMC Product Development

Developers of ATCA and AMC products often choose to use externally acquired platform management building blocks so they can focus their own scarce development resources on unique added values. Such building blocks must meet certain key requirements.


  • Page 1 of 1
    Bookmark and Share

Suppliers and consumers of telecom equipment are aggressively adopting the AdvancedTCA (ATCA) and AdvancedMC (AMC) architectures defined by PICMG. One key reason is that these architectures include an advanced hardware platform management infrastructure based on the Intelligent Platform Management Interface (IPMI). This infrastructure is a key enabler for the stringent service availability requirements that are typically mandated in telecom equipment and the mix and match interoperability that is crucial to the promised reductions in capital and operating expenses.

Implementing the relevant aspects of this platform management architecture in each type of ATCA/AMC component is not a trivial matter, however. Even the challenge of thoroughly understanding the management aspects of the relevant specifications is substantial. The IPMI specification is over 400 pages long; the management parts of the ATCA and AdvancedMC specifications account for hundreds of additional pages.

Building Block Candidates

Figure 1 shows the logical elements of an example ATCA shelf, focusing on the platform management aspects. An ATCA shelf manager communicates inside the shelf with IPM controllers, each of which is responsible for local management of one or more Field Replaceable Units (FRUs), such as boards, fan trays or power entry modules. Management communication within a shelf occurs primarily over the Intelligent Platform Management Bus (IPMB), which is implemented on a dual-redundant basis as IPMB-0.

ATCA boards that are AMC carriers add another level to the management hierarchy. Carrier IPMCs handle on-carrier management, including communication over a local IPMB-L with the Module Management Controllers (MMCs) that provide onboard management for the AMC modules.

An overall System Manager (typically external to the shelf) can coordinate the activities of multiple shelves. A System Manager communicates with each shelf manager over an Internet Protocol (IP)-capable transport, usually Ethernet.

As shown in Figure 1, the key interoperability interfaces in the ATCA/AMC architecture include IPMB-0, which connects the shelf manager (or more precisely its shelf management controller or ShMC) to the IPM controllers in the shelf. In addition, IPMB-L connects a Carrier IPMC to the MMCs on the AMCs installed on the carrier. There is also a system manager interface, by which a system manager communicates with one or more shelf managers.

The ATCA/AMC management architecture uses IPMI-based messaging across all these interfaces. PICMG has extended IPMI with dozens of commands to cover the unique needs of a modular platform designed for high availability. A key goal of this architecture is to ensure that independently implemented building blocks can communicate seamlessly across these interfaces, enabling coherent management of the entire shelf.

Compliant and Interoperable Across Key Interfaces

To achieve the necessary compatibilities, management building block suppliers must be thoroughly familiar with the specifications they’re implementing. Ideally, they are directly involved in the PICMG subcommittees that create the specifications. Although there are some moves in that direction, there are currently no formal compliance processes for PICMG specifications. In the meantime, building block suppliers must thoroughly test their products for compliance.

In addition to internal compliance testing, independently implemented management building blocks must be compatible across the interoperability interfaces shown in Figure 1. Since 2002 (even before the ATCA specification was initially adopted), PICMG has been running ATCA/AMC Interoperability Workshops (AIWs) to help ensure this compatibility. AIWs happen about three times a year and typically involve 15-30 PICMG member companies. Each company brings instances of their products for testing. During the AIW, participants systematically apply dozens of test scenarios (see sidebar “ATCA/AMC Interoperability Workshop Scenarios”) with different combinations of components. For instance, each distinct board type is tested with each distinct shelf manager type.

Experience in AIWs has directly resulted in improvements in the ATCA and AMC specifications. For instance, independent implementers interpreted early specification requirements for FRU state transitions differently. Those and many other similar areas of potential incompatibility are now clarified in the specifications, but more will undoubtedly be found as testing gets broader and more intensive. Management building block suppliers must actively participate in AIWs to help ensure interoperability of their components and their customers’ products as well as to help uncover any specification ambiguities that remain.

Adaptable Hardware and Software

Management building blocks must be adaptable in both the hardware and software dimensions. For instance, building blocks could be supplied with reference schematics for appropriate hardware components and source code for the corresponding software components. Of course, it is also important for the largest possible fraction of board developers to be able to use the reference design and firmware with only minor changes.

The article, “Accelerating the AdvancedTCA Ramp with Adaptable Shelf Management,” in the May 2005 issue of RTC magazine covers this topic for shelf managers. For example, differing shelf requirements may mandate different placements of dedicated shelf manager FRUs in a shelf. A shelf manager building block needs to be adaptable in this and many other respects.

At the ATCA/AMC board/module level, adaptability is also critical, since the range of complexity and application focus for these boards is very wide. For instance, different developers and their customers may have quite different philosophies regarding the number and type of IPMI sensors (e.g., voltage or temperature) needed on a board. An IPMC building block must provide good flexibility in this area.

As another example, consider the E-Keying feature of the high-speed fabrics implemented in the ATCA/AMC architectures. Here, data structures stored as non-volatile FRU information associated with each board and shelf component indicate what protocols and widths of interfaces are supported. The shelf manager checks this data and issues commands to the communicating nodes that enable only compatible ports on each node. An IPM controller needs to communicate with the fabric devices on its board and enable or disable their ports, based on these directions from the shelf manager. How it does this communication may differ greatly between a node board that only attaches to two fabric ports and a multi-fabric switch board that attaches to dozens of fabric ports.

Reliable Firmware Upgrade

Management firmware intended for use in highly available ATCA systems must be reliably upgradeable without a physical visit to the equipment. Reliable upgrade typically implies that a separate copy of the upgraded firmware is downloaded to the IPM controller in such a way that the original firmware is preserved. If anything goes wrong at any time during the upgrade process, the original firmware can be restored and the IPM controller can go back to using it while the difficulties in the upgrade are sorted out.

The IPMI specification reserves a set of command codes for firmware upgrades, but doesn’t define any specific command code points or associated semantics. Therefore, the firmware upgrade protocol by which commands using these codes implement an upgrade is currently specific to the developer of the IPM controller firmware. Work is underway within PICMG to standardize this process, but the results are not yet available.

IPMB-0 is the only mandatory interface to an IPM controller so it is a natural choice as a primary upgrade interface. Upgrade protocol packets can be forwarded from (and corresponding responses forwarded back to) an upgrade facility that either resides with or remotely accesses the shelf manager, which can play a proxy role by forwarding the packets over IPMB-0. Figure 2 shows this process. Figure 2 also shows how upgrade protocol packets can reach the MMC on an AMC, using the Carrier IPMC as a second-level proxy.

Management Building Blocks for ATCA/AMC

One set of ATCA/AMC management building blocks is the IPM Sentry family. Figure 3 shows some key family members and type(s) of in-shelf management controller targeted by each one. IPM Sentry management solutions are designed to address the requirements areas highlighted in earlier sections. IPM Sentry component developers are deeply familiar with the relevant PICMG and IPMI specifications; they are active in specification evolution, contributing substantial content.

At the shelf level, the SO-DIMM-sized ShMM-500, when mated with the shelf-specific carrier board, can be placed flexibly within a shelf; reference schematics for carrier board are provided. At the board level, reference schematics are integrated directly into the design of the managed board to optimize customization, cost and board real estate requirements.

Shelf manager source code is available, but there is also shelf-specific code for most shelves. Board management firmware source code is available to all board developers, but many can limit their changes to just a single source file to establish configuration details for their boards. At the shelf level, the ShMM-500 includes hardware support for reliable firmware upgrade. If newly loaded firmware does not confirm its operation quickly, hardware automatically reboots with previous firmware. At the board level, BMR reference designs provide an option to save existing firmware before new firmware is loaded. If new firmware doesn’t work, automatic restore and reboot on previous firmware occurs.

Pigeon Point Systems
Scotts Valley, CA.
(831) 438-1565.