MicroTCA.0 Spec Adapts and Extends PICMG Hardware Platform Management

MicroTCA is a new architecture, with definite differences from existing ATCA and AMC platform management approaches. Still, the management architecture preserves compatibility for overall system managers that need to manage MicroTCA shelves along with ATCA shelves, while including significant differences for inside-the-shelf management.


  • Page 1 of 1
    Bookmark and Share

The entire MicroTCA architecture was built around a key ground rule: MicroTCA Must Work With AMC.0-compliant AdvancedMC Modules. This ground rule is essential to preserve the potential for AMC modules to be used in either an ATCA carrier board or a MicroTCA “carrier”—essentially a subrack for AMCs to plug directly into a backplane, plus all the necessary supporting resources, such as power, management, interconnect fabrics and cooling. While there undoubtedly will be AMC modules that are focused for use in one or the other of these environments, the goal of the MicroTCA initiative is to encourage an ecosystem of modules that could be used in either context.

These management issues affect two kinds of AMC/MicroTCA “carriers.” One is an ATCA board that includes connectors for adding AMC cards defined by the AMC.0 specification. The other is a MicroTCA chassis with a backplane that accepts up to 12 ATC modules as defined by the MicroTCA.0 specification. On the management front, this ground rule means that MicroTCA carriers must provide the following for each AMC slot:

• An Intelligent Platform Management Bus-Local (IPMB-L) interface for platform management interactions with the module. The module is required by both specifications to be isolatable so that faults on IPMB-L to one module don’t have to affect the IPMB-L connection to other modules.

• A presence signal to indicate to the carrier (be it an ATCA carrier card with AMC connectors or a MicroTCA backplane chassis) that a module has been inserted in a slot.

• An enable signal to allow the carrier to release the module management controller (MMC) on the module from reset when a module is inserted in a slot and ready to have platform management activated.

Figure 1 shows these key management interfaces of an AMC module. The additional interface shown at the bottom of the module represents as many as 21 high-speed fabric ports that can be implemented on each AMC. As indicated in the figure, AMCs do not connect to the dual redundant IPMB-0 that plays a role in both ATCA and MicroTCA. Each MicroTCA carrier can handle up to 12 AMC modules.

On an ATCA carrier, the carrier IPMC provides local management for the AMC slots it implements and represents any installed AMC modules to the shelf manager over the dual redundant IPMB-0. How are these responsibilities handled in MicroTCA?

MicroTCA Carrier Manager

The carrier manager handles most of these responsibilities in MicroTCA. It interacts over IPMB-L with the AMC modules and represents them to the MicroTCA shelf manager. In addition, however, the carrier manager represents and interacts with the enhanced module management controllers (EMMCs) of MicroTCA-specific module types, including power modules, cooling units and OEM modules. These interactions occur over an intra-carrier IPMB-0 that is functionally equivalent to ATCA’s IPMB-0.

A carrier manager is one of the MicroTCA management components that reside on or interact with a MicroTCA carrier hub (MCH) module. MicroTCA MCHs provide the key management and interconnect facilities of a MicroTCA carrier. For instance, an MCH can include up to 84 lanes of fabric switch facilities, servicing the fabric needs of the installed AMC modules. MCHs can also be implemented on a redundant basis, in which case the fabrics are likely organized as a dual star. Even if dual MCHs are implemented, at any given time there is only a single carrier manager to represent the carrier, and all the resources it hosts, to upper level management.

The final management component that is always implemented on an MCH is the MicroTCA carrier management controller (MCMC), which provides local management for the MCH. An MCMC handles, for instance, E-Keying of the fabrics on its MCH and implements any MCH management sensors, such as the required temperature sensors. The MCMC also uses I2C to access a carrier FRU Info device (typically an I2C SEEPROM) to gather management information about the carrier. This information indicates, for instance, what module slots the carrier implements and, for the AMC and MCH slots, what fabric connections the carrier backplane implements.

In a redundant MCH configuration, each MCH implements one MCMC. In that case, the single carrier manager works with both MCMCs for matters concerning their respective MCHs, such as their temperature sensors or fabric resources. Figure 2 shows these resources, along with the MicroTCA shelf manager component.

MicroTCA Shelf Manager

MicroTCA Shelf Manager is either local to or remote from the MCH. Both options are shown in Figure 2. A local MicroTCA Shelf Manager is implemented on the MCH directly. In this case, the specification allows the shelf-carrier manager interface to be private. Alternatively, a remote MicroTCA shelf manager is implemented in some off-MCH way. In this case, the shelf-carrier manager interface is required to be based on the Remote Management Control Protocol (RMCP, as defined by IPMI) and implemented over UDP.

Figure 2 doesn’t attempt to represent any of the wide range of implementation choices for a remote MicroTCA shelf manager, but to name just two, it could execute inside a carrier on one or a pair of redundant AMC modules, or even on some processor that is outside the MicroTCA shelf altogether. In either case, the shelf manager communicates with the carrier manager over some IP-capable fabric, typically Ethernet.

As with ATCA, the MicroTCA shelf manager is responsible for representing the MicroTCA shelf to an upper level system manager (not shown in Figure 2). However, the detailed responsibilities of a MicroTCA shelf manager are significantly different from those of an ATCA Shelf Manager. While the MicroTCA shelf manager retains responsibility for supervising the cooling of the entire shelf, it doesn’t get involved with power allocations or E-Keying, among other things. Both of those missions are the responsibility of the MicroTCA carrier manager.

A MicroTCA shelf may have up to 16 MicroTCA carriers—implying an upper limit of 16x12 or 192 (!) AMC modules—plus all the support modules of those carriers as well. For a multi-carrier shelf, the shelf-carrier manager interface is logically equivalent to ATCA’s in-shelf IPMB-0, since it provides access for the shelf manager to all the carrier managers and the modules that each of them represent.

When it comes to how MicroTCA systems are powered, again, there are big contrasts from the way in which AMCs on an ATCA carrier board are powered. In the ATCA case, the carrier is fully responsible for providing power to the modules it hosts, based on the power budget is has been allocated by the shelf manager.

MicroTCA Power Architecture

MicroTCA power modules (PMs) represent a fundamentally different approach to power than is used for the AMC modules on an ATCA carrier, and necessarily so. MicroTCA power modules are (as the name suggests) distinct physical elements that can come from a different vendor than any of the other components of a MicroTCA carrier. This requires a substantial architecture defining how they interact with the other elements of the carrier. In contrast, the power subsystem of an ATCA carrier is entirely defined by the carrier implementer, following the AMC.0 specification, except where it connects to the specification-defined AMC interfaces.

MicroTCA power modules can be redundant, which means that one of the PMs in a MicroTCA carrier can be designated to back up the other one to three PMs. If any one of the primary PMs fail, the redundant PM will automatically (and under direct hardware control) take over the load of the failed PM. On an ATCA carrier, in contrast, if the power subsystem fails, the entire carrier is considered to have failed and all its responsibilities must be transferred to some other entity (perhaps a backup carrier).

Local management for each PM is handled by an EMMC that coordinates with the carrier manager over the intra-carrier IPMB-0. Some functions of the carrier manager can only be accomplished with the assistance of the PMs in the carrier. For instance, the PMs sense and control the radial presence and enable signals that respectively indicate 1) that a module is occupying a slot and 2) that the management controller on the module should be released from reset. For each of these signals, 1-n lines connect to the PM, depending on the number of slots in the backplane. Figure 3 shows the key management interfaces of a MicroTCA power module, including its responsibility for the radial presence and its ability to enable signals of its assigned modules.

Controling Cooling Units via Carrier Manager

The final key element of a MicroTCA carrier is one or two cooling units (CUs). A CU’s local management is also handled by an EMMC, which communicates with the carrier manager over IPMB-0. Just like an AMC module, a CU has a presence output and an enable input, which are handled by its assigned PM.

A CU implements the same set of fan control commands that are defined for ATCA fan trays. Unlike PMs, however, the effects of a Cooling Unit can extend beyond its home MicroTCA carrier. For instance, if multiple carriers are stacked vertically in a shelf, cooling units at the bottom of the stack may be responsible for cooling all of them. Because of this potential inter-carrier role, the control of CUs is the responsibility of the shelf manager, not the carrier manager.

A carrier manager forwards commands to its CUs, but does not have a management role with respect to them. The configuration data for a MicroTCA shelf details the modules that each CU cools so that the shelf manager can appropriately adjust the fan level for the right CU when a module needs more or less cooling. Figure 4 shows how CUs fit into the management framework for MicroTCA, along with the other elements discussed above.

Just as for ATCA, implementation of the various management controllers for MicroTCA (as shown in Figure 4) is a serious engineering project. The hardware platform management section of the MicroTCA.0 specification is 110 pages long and it leans heavily on the corresponding sections in the ATCA and AMC.0 specifications, which account for another 285 pages of dense requirements and guidance.

For most builders of MicroTCA products, it is far preferable to make use of an existing hardware plus software solution for these mandatory controllers, compared to implementing them from scratch. Pigeon Point provides such solutions, not only for MicroTCA, but for the corresponding ATCA- and AMC-defined controllers as well. Using such solutions allows a product developer to focus on their own value adds, rather than spending time on the mandatory management facilities. As shown in Figure 4, Pigeon Point offers solutions for AMC MMCs, plus MicroTCA EMMCs and MCMCs (including a µShelf Manager that executes on the MCMC hardware).

Pigeon Point Systems
Scotts Valley, CA.
(831) 438-1565.