Managing Small Modules
Using I2C for "Behind-the-Scenes" Management
Hardware platform management implementations must interface with a variety of components with different characteristics. The Inter-Integrated Circuit Bus provides a straightforward and extensible basis for building powerful management into complex systems.
MICHAEL THOMPSON, PENTAIR/SCHROFF AND SERGE ZHUKOV, PIGEON POINT SYSTEMS
Page 1 of 1
The Advanced Telecommunications Architecture (ATCA) and MicroTCA specifications include hardware platform management capabilities because the target markets demand high reliability and maintainability. The specifications intentionally omit the implementation details of the hardware and firmware for the management devices. This allows the designer to consider all of the trade-offs and decide what is best.
From the external, management-oriented view (Figure 1), a shelf is a collection of field replaceable units (FRUs) and their associated sensors. When you look at the actual FRU designs you will find that they are often intelligent, incorporating a microprocessor or an FPGA with a microprocessor core running shelf manager or Intelligent Platform Management Controller (IPMC) firmware. An FRU can also be non-intelligent, but managed by an intelligent FRU. In either case, the external representation of the intelligent and non-intelligent FRUs and their sensors is the same.
The Intelligent Platform Management Interface (IPMI) architecture, substantially enhanced by the ATCA and MicroTCA specifications, is used as a standardized method of interacting with the FRUs. This architecture defines messages that travel on the Intelligent Platform Management Bus (IPMB), an I2C bus that interconnects the FRUs inside the shelf. Using IPMI abstracts the physical implementation of the FRU. The only requirement is that the FRU implementation must comply with the IPMI behavioral and I2C electrical specifications.
Approaches to Management Solution
ATCA supports both intelligent and non-intelligent FRUs. Non-intelligent FRUs are usually managed by an IPMC on another intelligent FRU or by the shelf manager (the managing controller). The managing controller represents itself and its own sensors (such as temperature or voltage sensors) to the shelf manager, as well as representing its managed FRUs and their sensors. A managing IPMC can represent AdvancedMC (AMC) modules to the shelf manager, using an architecture defined by the AdvancedMC specification.
An IPMC uses IPMI/I2C to communicate with the shelf manager over the IPMB, but the communication interface between the IPMC, its sensors and its managed FRUs is not defined. Each IPMC needs to implement I2C for the IPMB so it is natural to also use I2C-based voltage, current, temperature or presence sensor devices and I2C-based nonvolatile FRU data storage.
The I2C sensor devices that are on an FRU controlled by the IPMC are not directly connected to anything outside of that FRU. The I2C devices on the FRU are accessed in a standardized way via IPMI commands. For the purposes of hardware platform management, the devices in the IPMC are usually exposed as collections of sensors and FRU inventory devices.
For each supported sensor, the IPMC has an associated Sensor Data Record (SDR). IPMI commands are defined to read the contents of the SDR. The SDR contains all information about the sensor: its type, format of the analog reading, measurement units and parameters for conversion from the single-byte “raw” reading to the numeric “processed” reading. The processed reading is expressed in measurement units that make sense to the user like the number of RPMs for a fan tachometer sensor or the number of volts for a voltage sensor. The SDR also contains the human-readable name of the sensor, initial values of thresholds and hysteresis, and, for discrete sensors, the mask of supported states and assertion and deassertion state masks.
So, the IPMC normally represents I2C sensor devices as IPMI sensors, mapping the corresponding I2C device register value to the IPMI analog reading and describing the attributes of the device in the corresponding SDR. I2C devices and signals used as control inputs and outputs are represented as settable sensors. I2C-based EEPROMs are often used to store “FRU information.” The FRU information is a collection of structured records defined by the IPMI or PICMG specifications that describe an FRU and its properties (such as inventory information like serial and model numbers).
A key question for ATCA shelf developers is how to manage the auxiliary FRUs that make up the shelf infrastructure, such as fan trays, power entry modules and alarm panels. In one approach, as shown in Figure 1, each of these FRUs is represented by an IPMC. Some shelf vendors and their customers prefer that approach.
The IPMC-based auxiliary FRU architecture can make shelf maintenance a headache because there can be more than six IPMCs on those FRUs (in a shelf that has two power modules, three fan trays and an alarm panel, for instance). The different types of FRUs in the shelf usually have their own specific firmware versions, adding to the maintenance complexity. Different revisions of the IPMC firmware may behave differently so you need to be careful about mixing FRU revision levels. You may also find that newer revisions of the IPMC firmware may not run on older FRU hardware revisions.
An alternative shelf management architecture has a single microprocessor, usually on the shelf manager (or perhaps a redundant pair of microprocessors, normally running the same firmware), managing multiple non-intelligent chassis FRUs. This dramatically reduces the number of microprocessors and the amount of independently managed firmware in the shelf. Since there is only a single copy of management firmware this also makes system maintenance much easer. When the firmware in the single microprocessor is upgraded, the behavior of all of the FRUs is upgraded at the same time. In the example that follows, the shelf manager manages six non-intelligent FRUs in the shelf. The shelf manager has I2C interconnects for management links and also uses I2C to connect to the devices on the FRUs. Just as on an IPMC, the microprocessor on the shelf manager manages the non-intelligent FRUs in the shelf and makes the sensor devices on managed FRUs in the shelf appear to be intelligent.
Figure 2 shows a shelf management architecture in this second model, where the auxiliary FRUs are non-intelligent and are “owned” by the shelf manager, rather than having local IPMCs.
Compared with a regular IPMC, the shelf manager has more challenges and possibilities in managing these auxiliary FRUs. The main challenge is that a shelf manager should ideally support ATCA shelves of different types, each of which may have a different population of non-intelligent FRUs, sensors and control devices. It is good to avoid the need to add new firmware to support each new shelf design. This will also prevent the shelf manager firmware implementation from expanding in size and complexity over time. To implement this flexible management architecture you need a generic method that describes a specific shelf design and lets the shelf manager interpret this description dynamically. Compared to a regular IPMC, the shelf manager can utilize a more powerful processor, so this approach becomes practical.
The Pigeon Point shelf manager uses this approach to provide generic support for different types of shelves, shelf manager carriers and non-intelligent FRUs. These are described in a textual format in a special language designed by Pigeon Point Systems, called Hardware Platform Definition Language (HPDL). An HPDL description of a shelf includes the list of non-intelligent FRUs with their properties, describes the I2C devices and the corresponding signals that exist on the FRUs, describes mapping between the I2C devices and IPMI sensors, and also includes the SDRs for these sensors.
The shelf description in HPDL is created by the shelf manufacturer, possibly in collaboration with Pigeon Point engineers. The HPDL description is then compiled into a binary format, compressed and stored in the Shelf FRU information device. The Shelf FRU information device is a redundant (typically an EEPROM-based) storage device that contains, according to the ATCA specification, important information about the shelf. Shelf FRU Information is mandatory for any ATCA-compliant shelf, and its format, based on variable-sized records, allows vendor-specific extensions. This is a good location for storing the HPDL description of the shelf. The shelf manager reads the Shelf FRU Information at initialization.
When the shelf manager firmware encounters HPDL description records in the Shelf FRU Information, it reads and interprets them. Based on that information, the shelf manager firmware creates corresponding managed FRUs and sensors, and maps subsequent operations on managed FRUs and sensors to reads and writes on the corresponding I2C devices.
A Management Architecture
Designing a shelf that utilizes HPDL with just I2C devices on the FRUs can present some challenges. I2C devices like temperature sensors and EEPROMs have a limited number of possible bus addresses. Fortunately, I2C bus switches are available so that the I2C devices can be arranged in a hierarchy that eliminates the address conflicts. Also, there are limitations on the total capacitance that the I2C devices can put on the I2C bus. An I2C switch alleviates this restriction because it reduces the total number of devices that are connected to a single I2C bus at the same time.
Figure 3 shows the overall I2C-based FRU management architecture of a Schroff ATCA shelf. The Schroff shelf manager incorporates a Pigeon Point ShMM-500R Shelf Management Mezzanine. The RMI Alchemy Au1550 processor on the ShMM-500R has two I2C buses that implement the IPMB-0 management links. It also has a master-only I2C bus that is used to connect to the I2C devices on the shelf manager and auxiliary shelf FRUs.
The shelf manager includes an On Semiconductor I2C ADM1026 that provides FRU data storage, fan speed monitoring, fan speed control, temperature sensing, voltage measurement and GPIO—nearly all of the sensors on the shelf manager. HPDL records describe the I2C devices on the shelf manager, and how the shelf manager’s sensors are mapped to the I2C devices or the pins on the I2C devices. The shelf manager’s master-only I2C bus needs to be connected to the I2C devices on the shelf’s non-intelligent FRUs. As stated earlier, a hierarchy of I2C buses is necessary because of address limitations and to limit capacitance on the I2C bus. An NXP PCA9545 1x4 I2C switch and an LTC4300-controllable I2C buffer expand the single master-only I2C bus to five I2C bus segments that connect to the off-board I2C devices. HPDL records describe the I2C switch and the I2C bus hierarchy so the shelf manager firmware knows where to find all of the devices on the non-intelligent FRUs.
The shelf alarm panel (SAP) has an onboard I2C temperature sensor, three exhaust air temperature sensors, a SEEPROM for FRU data storage and PCA9555 GPIO devices. Since the SAP is a separate FRU from the shelf manager, these I2C devices are described in the HPDL records, but they are associated with the SAP FRU.
The power entry modules (PEMs) and fan trays use the same LM75 I2C temperature sensor, SEEPROM and GPIO devices as the SAP. There are not enough I2C addresses available for all of the devices to be connected to the shelf manager at the same time. The I2C switch on the shelf manager allows us to only connect to a few of the devices at a time, so the hierarchical design works. Additional HPDL records describe all of these I2C devices and associate them with their corresponding FRUs.
The shelf requires FRU data storage for board and product data, for power management and interconnect information, and for the HPDL data. This shelf includes SEEPROMs on chassis data modules (CDMs) on individual I2C buses for redundancy. The Shelf FRU Information SEEPROMs are described in HPDL records that are stored in the SEEPROMs on the shelf manager.
The shelf supports redundant shelf managers. Because of the possibility of a hardware or software failure, only a healthy and active shelf manager can enable its off-board I2C buses and utilize the I2C devices on the Shelf FRUs. A watchdog timer on the shelf manager will disable off-board access to the I2C devices if there is a hardware or software failure on the shelf manager.
There are a small number of I2C devices on the shelf manager that measure the local air temperature, control and monitor the cooling fans, store FRU data, and control and monitor digital I/O. The shelf manager firmware running on the Au1550 makes its onboard I2C devices appear as intelligent sensors that are associated with the shelf manager FRU.
The HPDL records that describe the fan tray of the shelf architecture described above, are shown in Figure 4.
From the implementation examples above, we can see that I2C has proven to be an excellent choice for both the TCA-specified in-shelf management links and for the behind-the-scenes connections to the sensor and storage devices on the FRUs in a shelf.
Pigeon Point Systems
Scotts Valley, CA.