RTC Magazine

The ARM instruction set architecture is implemented in a very wide range of CPU and microcontroller devices. Different levels of that range of devices can be applied across different types of management controllers defined by the widely implemented xTCA management framework. xTCA encompasses both the AdvancedTCA (ATCA) and complementary MicroTCA (µTCA) platforms, as well as the AdvancedMC (AMC) hot swappable modules used in both platform types. So which implementations are candidates for ARM among xTCA management controller types?

The simple answer is “all of them,” but typically with different levels of the ARM architecture.  Figure 1 shows the following controller types for both xTCA platforms, in order of increasing complexity:

• Module management controllers (MMCs) for AMC modules and enhanced MMC (EMMC) for µTCA infrastructure modules like Cooling Units or Power Modules.

• IPMCs and carrier IPMCs, which monitor ATCA boards, including boards that can carry AMC modules.

• µTCA shelf managers, carrier managers and MicroTCA carrier management controllers (MCMCs), which manage the AMC and infrastructure modules in a µTCA shelf, which is typically modest in size. Often, all of these logical functions are implemented by a single physical controller, as suggested by the dashed line boundary in the above Figure.

• ATCA shelf managers, which handle management coordination for ATCA shelves, each of which can include dozens of boards and modules.

The above figure also annotates each module type with a level of ARM implementation that could fit the resource and compute power needs of that type. Two generations of ARM implementation levels are cited. “Cortex-Ax” devices are application CPUs capable of running full scale operating systems such as Linux, while “Cortex-Mx” devices are microcontrollers that, for Linux, can only run the µCLinux subset. In fact, low-end Cortex-Mx devices usually have modest amounts of built-in RAM and flash; they could only run µCLinux with substantial external memory resources. For previous ARM generations, these levels can map to ARM9 and ARM7, respectively. Of course, implementers may choose to use different (including more capable) ARM implementation levels than those listed in the above figure for the various xTCA controller types.

Key Characteristics for Board or Module Management

One key need for such a controller is small size. ATCA boards tend to be very dense with functionality, and the required management controller needs to leave as much space as possible for that functionality. Figure 2 shows an example IPMC for an ATCA board; the example is based on a Cortex-M3 microcontroller, built into the Microsemi SmartFusion intelligent mixed signal FPGA. As shown in the figure, only a tiny fraction of the board area is used by this example IPMC implementation.

As the second figure demonstrates, an IPMC has numerous functional responsibilities, but an efficient implementation leveraging the compact ARM Thumb-2 instruction set variant can fit two copies of the necessary firmware in the 256 Kbyte of internal flash memory implemented in a typical Cortex-M3-based microcontroller, such as the SmartFusion A2F200. Lower tier xTCA management controllers, such as MMCs and EMMCs, can be implemented in even smaller Cortex-M3s, such as the SmartFusion A2F060, with its 128 Kbyte of built-in flash.

Two copies of the firmware support safe firmware upgrades in the field; if a newly downloaded firmware image is flawed, the controller can automatically fall back to a backup instance of the firmware. In a SmartFusion-based IPMC, critical supplementary functionality can also be implemented in the internal FPGA fabric; even that fabric can be upgraded remotely.

The focus on reliability of ATCA IPMCs continues with the ATCA specification required watchdog timer, which ensures that controller hangs, if any, can be recovered with a reboot of the IPMC firmware. In a Cortex-M3-based IPMC, the memory protection unit (MPU) can be used to further enhance reliability. Critical memory areas, such as system or subsystem control registers, can be protected from inadvertent access (especially writes!) by careful configuration of the MPU regions. For SmartFusion IPMCs, one such critical subsystem is the programmable analog functionality, which can participate in sensitive functions like supervision of the onboard power rails.

Further background on using intelligent mixed signal FPGAs for management controllers, including for xTCA, is provided in two recent RTC magazine articles:

• Using Intelligent Mixed Signal FPGAs for Hardware Platform Management, in the October 2010 issue, introduces Cortex-M3-based mixed signal FPGAs, several management frameworks in which they can be used, and key features such as a “LAN Attach” connection for the controller, using a 10/100 Ethernet MAC that is often included in medium range Cortex-M3s.

• Customizing a Microcontroller for Hardware Platform Management, in the June 2011 issue, covers the opportunities to integrate board functionality, such as the MicroTCA infrastructure management functions shown inside the dashed boundary line of the first figure or the power rail management shown in the second figure, into a management controller based on a Cortex-M3-enabled mixed signal FPGA, thereby saving board real estate and cost.

Capabilities Needed by the Overall Controller for an ATCA Shelf

This controller, an ATCA shelf manager, has substantially more responsibility than a board or module level controller. An ATCA shelf can have up to 16 boards, each of which can have two to four, or even more AMC modules, potentially with additional managed field replaceable units (FRUs) handling shelf infrastructure functions like cooling, power entry, sensors and shelf description storage. The resulting number of management controllers in an ATCA shelf can reach into the several dozens. Given the scale of its responsibilities, an ATCA shelf manager typically needs a higher level of ARM processor, at least an ARM9 or Cortex-Ax.

Further confirming its need for a higher level of ARM processor, an ATCA shelf manager usually supports a range of interfaces to the system manager, representing the next level of management above the shelf, potentially with oversight responsibility for tens, hundreds or even thousands of shelves. The top row of the logical shelf manager section in Figure 3 shows a typical set of system manager interface options.

These include the Remote Management Control Protocol (RMCP), which is standardized by the Intelligent Platform Management Interface (IPMI) specification and mandated by ATCA. This interface is quite low level and many ATCA users prefer a more abstract management interface to their shelves.

There is also the Hardware Platform Interface (HPI), which is standardized by the Service Availability Forum. HPI provides a set of application programming interfaces (APIs) for managing an abstract model of a shelf, with its FRUs and controllers. An HPI service built into an ATCA shelf manager can be dramatically more efficient, especially on startup of a complicated shelf, and can leverage the redundancy facilities of its containing shelf manager. There is an open source implementation of HPI called OpenHPI, but it lacks these integration benefits.

Other options include a Command Line Interface (CLI) that is typically proprietary to the shelf manager vendor and SNMP, which provides access to shelf management variables via the Simple Network Management Protocol. The framework and protocol of SNMP are standardized, but each shelf manager vendor implements a proprietary set of management variables. Similarly, an HTTP facility provides a web interface via a standardized protocol and proprietary web pages.

Furthermore, a logical ATCA shelf manager is typically implemented on dual redundant physical counterparts, coordinated via hardware and software redundancy interfaces—HRI and SRI, respectively. To enable seamless switchovers from an active shelf manager to its standby counterpart when necessary, the two units must have a shared hardware level state and every software subsystem in the shelf manager that maintains state for its aspects of shelf operation needs to share that state with its redundant counterpart. 

Redundancy is typically extended to the communication links used by the system manager to reach the shelf manager, with each instance cross-linked to two hubs in the shelf, usually those that implement the ATCA Base Interface via 10/100/1000 Base-T Ethernet.

With all the above functionality, an ATCA shelf manager is usually implemented on a full-fledged operating system, such as Linux. Such operating systems require a memory management unit (MMU); shelf manager reliability considerations also require the process isolation and memory protection features of an MMU. The µCLinux subset does not require an MMU and does not address these reliability considerations. An ARM processor used for an ATCA shelf manager would usually be based on the Cortex-Ax or ARM9+ architectures. 

Cost optimization is still important at this level, so a system on chip (SoC) implementation that includes dual Ethernets, built-in support for SD RAM and cost-effective non-volatile storage is normally required. Beyond the dual Ethernets needed for reliable communication with the system manager, a third Internet Protocol capable link to implement the Software Redundancy Interface between the physical shelf managers is highly desirable. This link could be Ethernet- or possibly USB-based.

Example ATCA Shelf Manager with Two ARM Processors

As indicated above, an ATCA shelf manager platform typically includes an MMU-capable SoC, such as the Freescale i.MX287 processor used in the Pigeon Point ShMM-700R diagramed in Figure 4. The i.MX287 processor includes an ARM9-based core and a multitude of built-in interfaces, allowing for a very cost-effective implementation. 

In the Pigeon Point Shelf Manager, the shelf adaptation layer of the third Figure interprets a Hardware Platform Description Language (HPDL) description of the shelf. That description resides, along with xTCA- and IPMI-defined data structures, in the shelf description area, allowing the shelf manager to automatically adapt to the characteristics of any shelf variant into which it is installed. This interpretation layer makes additional requirements on the ARM9-based CPU.

In addition to supporting all the other system manager interfaces shown in the third Figure, the Pigeon Point Shelf Manager implements a built-in HPI subsystem called IntegralHPI, which fully supports redundancy and collaborates with the main part of the shelf manager during shelf startup to dramatically reduce the interval before HPI-manageability of a shelf is possible.

Complementing the i.MX287 is a second controller, the Cortex-M3-based A2F060 SmartFusion device. The A2F060’s built-in FPGA fabric implements the hardware redundancy interface and special logic that supports dual redundant images for the shelf manager firmware. Furthermore, the A2F060’s I2C ports are, by design, suitable for implementing xTCA’s dual redundant IPMB-0; board voltages are monitored by the A2F060 programmable analog subsystem. Here, the SmartFusion device is handling a reduced version of the responsibilities of the board and module level controllers discussed earlier. The Pigeon Point Board Management Reference (BMR) solutions for those xTCA controller types package those capabilities for that context.

Different variants of Microsemi SmartFusion Cortex-M3, the Freescale i.MX287 ARM9, and combinations thereof can serve the entire range of xTCA management controllers in a cost-effective fashion. 

Pigeon Point Systems

Scotts Valley, CA.
(831) 438-1565.
[www.pigeonpoint.com].