RTC Magazine

Serial Attached SCSI (SAS) technology offers a wealth of benefits to the enterprise server, storage enclosure and storage networking customers. These include high reliability and performance, mixed enterprise/desktop drive support and improved economies of scale. SAS is even being evolved into an island SAN support fabric. As the T10 Standards Committee continues its charge forward, SAS as a storage interconnect and storage networking technology continues to gain momentum not only in midrange environments but also in the enterprise—a space that was once dominated by channel-oriented technologies.

Although SAS was originally envisioned as a serial replacement for parallel SCSI, its overall development, in terms of functionality and performance, has grown by leaps and bounds. Most recently, the T10 Standards Committee completed work on and ratified the standard on which 6 Gbit/s SAS is based. 

6 Gbit/s SAS Technology

SAS is the evolutionary follow-on to the parallel SCSI interface. Like other serial storage technologies such as Serial ATA, SAS was originally envisioned as only a point-to-point connection mechanism to replace a multi-drop parallel bus, but through two generations of serial technology it has evolved to become much more. 6 Gbit/s SAS not only provides for a faster interconnect rate, versus 1.5 Gbit/s and 3 Gbit/s, but it delivers a wide array of enterprise features including zoning. It also provides for enhanced receive performance based on adaptive decision feedback equalization technology (DFE). Similarly, spread spectrum clocking (SSC) was added for electromagnetic interference (EMI) mitigation. Numerous other features were added but will not be covered here. The industry objectives for 6 Gbit/s SAS include:

- Preserve 3 Gbit/s SAS usage models
- Maintain 1.5 Gbit/s SAS, 3 Gbit/s SAS and SATA compatibility
- Reduce the number of cabling and connector options
- Double transfer performance to 6 Gbit/s SAS
- Improve cost/performance and power/bandwidth ratios
- Reduce number of connections per Gbit/s
- Support 10m copper interconnects 

6 Gbit/s SAS achieved these industry objectives and more. In fact, it provides further extensions to the base 6 Gbit/s SAS specification. It also delivers additional enterprise features including optical interface support, as well as enhanced power management and control.  

The 6 Gbit/s speed enhancement addressed several of the industry objectives for 6 Gbit/s SAS. Not only did the speed grade improvement provide for better overall system and interconnect performance, it paved the way for the adoption of SSDs (Solid State Drives)—one of the few target mechanisms that can take advantage of the multi-gigabit transfer rates offered by 6 Gbit/s SAS. Maintaining backward compatibility with earlier generations, such as 3 Gbit/s SAS, was required for legacy device support and required significant engineering development both in the standards body and the component community. In addition, the overall improvement in bandwidth allows more devices such as drives and controllers to exist in the same domain without bumping into bandwidth limitations resulting in poor performance and ultimately I/O starvation.

Also key to supporting legacy environments and cabling distances (up to 10m copper), was the adoption of decision feedback equalization (DFE) technology in 6 Gbit/s SAS interfaces. A decision feedback equalizer is a nonlinear equalizer that uses previous detector decisions to eliminate inter-symbol interference (ISI) that can result in bit errors on the serial link. In other words, the distortion caused by previous serial bits is subtracted to produce a more accurate version of the bit being sampled. The result is a serial receiver that is substantially more tolerant of the challenging signal environments seen in more complex storage systems that make use of the higher frequency data links provided (Figure 1). Without DFE technology, it is unlikely that support of aggressive channel models would have been possible at the 6 Gbit/s SAS signaling rate and beyond.

Another key electrical feature of the 6 Gbit/s SAS specification is the introduction of Spread Spectrum Clocking (SSC) requirements (center and down-spread) for 6 Gbit/s SAS links. Support is optional at lower speed links such as 3 Gbit/s SAS and 1.5 Gbit/s SAS. SSC is employed to reduce the peak amplitude of radiated emissions. This is often useful in electrically “noisy” environments where electromagnetic interference (EMI) is of concern. SSC is accomplished by varying clock rates to spread emissions over a wider range of frequencies. Figures 2, 3 and 4, respectively, illustrate a non-spread signal, a down-spread signal and a center-spread signal. 

SAS Zoning

Enterprise storage system adoption of SAS started with the advent of 3 Gbit/s SAS. Customers of this type of storage system need high levels of functionality, control and security. Although many vendors implemented proprietary zoning schemes in early SAS components, 6 Gbit/s SAS sought to standardize zoning for better interoperability. Zoning has many different applications in enterprise storage systems. However, it is most often utilized in environments with multiple initiators, such as blade servers or clustered file server environments. The purpose is to associate a subset of each of the storage devices with a particular initiator.

Zoning is accomplished using a structure roughly analogous to that of a file system. A zone is the equivalent of a folder or directory and can either be “hard” or “soft.” In hard zoning, each device is assigned to a particular zone, and this assignment does not change. In soft zoning, device assignments can be changed dynamically by the storage administrator to accommodate variations in traffic patterns or data demands. Zoning not only improves storage efficiency by allowing multiple non-shared data entities to access and share the same domain, but it also blocks initiators from accessing each other or any dedicated devices. Further, zoning provides for improved security when multiple organizations share a common domain of storage, such as cloud computing environments.

Advanced Connectivity 

Unlike other enterprise storage technologies, SAS was not originally envisioned as needing to support any connection technology other than passive copper and then only with limited channel lengths (up to 10m). It quickly became clear that an active interface, either copper or optical, would be required as intra-datacenter interconnects demanded greater flexibility. To accommodate these needs, the T10 working group introduced advanced connectivity options for the SAS protocol in 6 Gbit/s SAS.  

The SAS response to these market demands is the Advanced Connectivity Roadmap, which offers improved capabilities beyond today’s widely deployed Mini-SAS connector with the denser and more flexible Mini-SAS High Density (HD) interconnect. The Mini-SAS HD offers remarkable improvements in SAS capabilities with enhancements targeting four main areas. It provides more box-to-box, server-to-storage and rack-to-rack connections, with increased deployment options and delivery of more flexible capacity within a SAS domain.  

Advanced Connectivity provides superior connections because of characteristics such as double the density of the Mini-SAS connector to support higher port-count densities. It is also electrically improved—less cross-talk, better signal-to-noise ratio and improved passive signaling. Cable lengths are also improved due to active copper cables lengthened to 20m and optical cables lengthened to 100m. 

It should be noted that active copper connections, supporting distances greater than 20m, are also possible using the standard external Mini-SAS connector. This is accomplished by reassigning a ground signal on the connector, thus providing power to the active components within the cable.  Deploying this standardized approach to achieving longer cable runs on the Mini-SAS connector is entering the market and will become a common usage model for 6 Gbit/s SAS.  

Unlike the Mini-SAS HD connector, the Mini-SAS connector lacks the cable management facilities and is limited to 6 Gbit/s SAS operation. Several vendors, cooperating at industry plugfests, have proven the robustness of these active copper environments on Mini-SAS and find it more than adequate for many application environments.

In addition, connectivity management is now possible due to these added enhancements including connection discovery, cable plant management, improved serviceability and reliability, plus lower total cost of ownership (TCO) through ease of error isolation and servicing. Finally, converged connectivity is achieved by using a single connector to support a variety of interconnect types, having a provision for consistent management across these connections and using a consistent method for port management and scaling.

Power Management/Control

A traditional server or storage enclosure may make use of a variety of SAS components that run the gamut from HBAs and RAID controllers, to expanders, signal conditioners and target devices that include SSDs as well as standard rotating media devices. Each component communicates with its peer through high-speed physical layer structures (PHYs). In general, PHYs are responsible for up to 70% of the total power consumed by a given component. 

As shown in Figure 5, the percentage of power consumed by the expander device in its I/O (PHY) alone represents more than 70% of the total device power consumption in a typical operating configuration. While other SAS components may have higher power consumption in their core area due to a higher core logic-to-PHY ratio, PHY power consumption still represents the lion’s share of overall dissipation in high-speed storage silicon components. Component power consumption is directly related to cooling burdens associated with a server or storage enclosure. In fact, direct cooling costs are second only to equipment operating costs, in terms of electricity usage in data centers today.

In the past little emphasis was placed on chip power efficiencies since lower output amplitudes and operating frequencies generally equated to lower power demands. For example, in today’s server systems, the cost of operating a 1U server in a datacenter environment often exceeds the actual cost of the server itself in a single year of operation. Given the escalation in overall energy costs over the last two decades, the information technology industry has come under great pressure to effectively manage power at the component level. As such, system vendors are increasingly turning to their component suppliers and to technical standards bodies in an effort to spur invention that can help enable overall power savings.

Effective management of a SAS component’s PHY power consumption provides the biggest benefit potential to overall SAS system power efficiency. 6 Gbit/s SAS addresses this management need with three tiers of PHY power states:

- Active PHY power (full power)
- Partial PHY power (lower power)
- Slumber PHY power (lowest power)

Of the three states, partial and slumber both refer to lower PHY power conditions than the active state. In the active PHY power state, a SAS device PHY is fully enabled and is capable of transmitting information and responding to received information without needing to change the PHY’s power condition, although the PHY may consume more power than when in a lower power condition.

While both the partial and slumber PHY power states operate at lower power consumption levels than the active state, they differ in the treatment of pending connection requests.

While in the partial power state, a connection request will wait for up to 10 microseconds for the PHY to achieve full power to honor the connection request, and then begin transmitting or receiving information. If a connection request is made to a PHY in the slumber power state, the connection request will be rejected using the OPEN_REJECT (RETRY) mechanism and the requesting device will retry its connection attempt at a later time.  During this type of wake-up attempt, the “slumbering” PHY is given 10 milliseconds to reach its full power state or else the link is reset and the connection attempt is aborted. 

Maxim Integrated Products
Sunnyvale, CA.
(408) 737-7600.
[www.maxim-ic.com].