RTC Magazine

It all started out innocently enough. The IT manager for a major food manufacturer hired a consultant to scan the company’s PCs, find out which ones were out of date, and then apply the necessary patches – pretty standard stuff. So it was a big surprise when he received a call from a control systems engineer in the production department asking about unusual network activity coming from the enterprise network. They discovered that the consultant had accidently scanned a range of IP addresses that were assigned to programmable logic controllers (PLCs) on the plant floor. The scanning caused every single PLC to crash, bringing the plant to a complete standstill and leading to the loss of over $1 million of work in process.

This was a rather expensive outage for the company in question; fortunately, cookie dough doesn’t explode when it hits the plant floor. However, managers at other types of plants such as chemical, oil and gas, or nuclear facilities have to consider the potential safety issues that can be caused by cyber security incidents in their Supervisory Control and Data Acquisition (SCADA) and control infrastructure, in addition to the potential financial impact.

An industrial control system, at its simplest, is a digital process controlling a real-world event. This could be a robotic arm on a manufacturing floor, a cooling pump in a nuclear reactor, or a valve in an oil pipeline. Most control system networks were designed as isolated, self-contained end-to-end networks. However, more and more organizations are interconnecting control system networks with corporate enterprise networks to maximize accessibility and reduce cost.

Multiple business drivers compel this increase in interconnectivity. Geographically dispersed systems are expensive; an oil company managing refineries all over the world wants to reduce costs by consolidation. Improved responsiveness leads to cost savings; electric Independent System Operators (ISOs) need real-time generation output for managing shortages, and selling excess. Business must be agile; just in time manufacturing enables quick response to volatile supply and demand. Maintenance costs can be minimized by enabling remote debugging and repair; integrating SCADA and IP networks eliminates the expense of additional cabling in an industrial environment, which can cost up to $3000 per foot. 

Even efforts to protect the networks can lead to unintended consequences. Regulations such as the Critical Infrastructure Protection (CIP) standards from the North American Electric Reliability Corporation (NERC) drive integration, as electric ISOs seek access to production data in real-time to demonstrate compliance. Ironically, the pursuit of security itself can lead to exposure! Protective measures such as centralizing access control to minimize tampering, or extending closed-circuit TV monitoring or VoIP to remote stations, require increased accessibility.

Challenges of Interconnectivity

Clearly, interconnectivity is the wave of the future – but many control system components were conceived in the past. Control devices, and the PCs that manage them, are very vulnerable – not only to malicious attacks using malformed network data, but also in many cases to even high levels of well-formed network traffic. PLCs and remote terminal units (RTUs) are typically optimized for high-performance real-time I/O, not for robust network interfaces. In addition, control networks run continuously for weeks or months at a time, and many systems cannot be shut down even for a few minutes without significant financial or safety impact. As a result, the PCs in these networks are often not up to date with security patches or anti-virus definitions.

Secondly, control networks are usually poorly segmented, with little or no separation between different subsystems or even different physical locations. If a problem occurs in one area of the network, it will spread rapidly to other unrelated systems elsewhere in the network. Poor segmentation also makes it very difficult to locate the origin of a problem and resolve it at the source. In the early days, control networks started as very simple ‘islands’ of automation, but they have steadily grown in size and complexity over time.

The third common issue is the existence of multiple points of entry into these networks. Many control network managers will swear up and down that their control systems are not connected to the enterprise network or the internet, but authorized penetration testing often shows otherwise. In addition, there are often other transient paths of entry that don’t even show up on a network diagram: VPN connections, laptops or even USB memory sticks traveling in and out of the plant can easily carry viruses right into the heart of the plant network.

Theoretical vulnerabilities lead to real-world incidents with far-reaching consequences including loss of productivity, revenue, and even loss of life. A Zotob worm infestation caused the shutdown of 13 assembly lines, affecting 50,000 workers, at Daimler Chrysler in 2005.  Failure of two water recirculation pumps due to excessive traffic on the control system network forced the manual shutdown of the reactor at the Browns Ferry Nuclear Plant in August of 2006. A year later, the National Transportation Safety Board found that an unresponsive SCADA system at Olympic Pipe Line Company contributed to a pipeline rupture and subsequent fire that killed three young men in Bellingham, WA. 

Most IT managers have significant experience addressing cyber security issues in enterprise networks, so why can’t managers of control and SCADA networks simply apply the same technologies in their systems? Control systems have unique requirements that until recently have not been addressed by available security solutions. These requirements include harsh physical and electrical environments and support for the unique communication protocols that are common in industrial networks. Such systems also require the ability to install, configure and test these security solutions in a ‘live’ operating network without putting the plant at risk.

A Way Forward

An aerospace company needs to implement secure connections between their enterprise network and the manufacturing plant. Because of the size of the product being built, the manufacturing tooling - mounted on mobile crawlers - roams throughout the entire facility. Essentially, the product being manufactured remains stationary while the production line moves around the product, the reverse of most production sites. Due to the mobility requirement, the crawler must use wireless connectivity with its potential security challenges, to connect to the network.

Open standards from the Trusted Network Connect (TNC) of the Trusted Computing Group (TCG) enable a solution that combines products from multiple vendors, all interoperating via standard interfaces. The cornerstone of this system is the MAP, or Metadata Access Point, which acts as a ‘clearing house’ for a wide variety of transitory data.  A key element in the operation of the TNC-based security solution, the MAP provides flexibility and interoperability that simply cannot be achieved with proprietary solutions.

A Tofino Security Appliance from Byres Security protects each crawler. Tofino provides firewall services to insulate the PLCs from disruption and permit only the specific network connections required for correct plant operation. In addition, Tofino VPN services secure all network connections to the crawler over the wireless network. When initially deployed, the appliances first check in with the MAP to collect their corporate security certificates.  Next, they retrieve their security policy (firewall rules and VPN security associations) via the same server. And if unauthorized network traffic is blocked by the firewall on a crawler, the Tofino can report this information to the MAP in real time. IF-MAP, an open protocol with support from diverse vendors, could then be used to respond in a variety of ways, including alerting the network security team, logging the incident in a database, or even changing security policy if appropriate. 

MAP-based functionality can go far beyond the crawler’s security appliance and other network-based security products. A wide variety of MAP-capable devices enable the company to implement highly optimized security solutions, customized to their specific needs (Figure 1). As each crawler moves around the plant, MAP-aware wireless access points report each crawler’s location to the MAP; the crawler’s security policy can be configured based on the physical location of the crawler. For example, if the crawler is located in a service bay, firewall policy allows a PLC engineering workstation to upload new firmware or logic programming into the controller. Such activity is prohibited when the crawler is in use on the plant floor. Other systems that interface to MAP, such as physical plant security products, can be configured to interoperate in ways that would be impractical, if not impossible, using proprietary solutions.

SCADA - One Type of Control Systems Network

As shown in Figure 2, the presence of a centralized controlling element differentiates a SCADA – Supervisory Control and Data Acquisition – network, as opposed to other control systems where multiple components might make individual control decisions. A SCADA network generally includes a control center and will usually have a backup control center to ensure availability and disaster recovery. Operators interact with the system via a Human Machine Interface (HMI).

The control center has a main terminal unit (MTU) that connects to one or more remote terminal units (RTUs)– which could be PCs or embedded OSs. The RTU communicates with an intelligent electronic device (IED) – this might be a microcontroller or a programmable logic controller (PLC) which runs the logic for running, controlling and monitoring a machine or robot.

These components have a range of connectivity – anything from modems to wireless to microwave to the Internet. They communicate with each other across a suite of protocols including DNP3, a distributed network protocol; ICCP, an inter-control-center communications protocol; Modbus, a protocol that originated on serial links but has since been ported to IP; or OPC, an open connectivity protocol originally based on Microsoft object linking and embedding (OLE) and distributed component object model (DCOM).

Open Standards for Network Security

Trusted Network Connect (TNC) is a work group of the Trusted Computing Group (TCG), an industry standards organization focused on strong security through trusted computing. TNC is completely vendor-neutral; the full set of TNC specifications is freely available for anyone to implement, and TNC-based products have been shipping for almost five years.

TNC standards provide an architecture and open interfaces that allow interrogation of an endpoint to determine its integrity and compliance with security policies.  When an endpoint requests access to the network, a policy server queries the endpoint, determines user identity and endpoint health, and makes an access control decision based on the resulting information.  The policy server sends a policy decision to an enforcement point, telling it whether to permit access, deny access, or quarantine the endpoint.  TNC interfaces standardize communication between these components at the network, transport, and application layers (Figure 3).  

TNC’s IF-MAP standard extends the TNC architecture to allow data sharing across a huge variety of security and networking systems.  The Metadata Access Point, or MAP, is a central clearinghouse for endpoint metadata; MAP clients can publish, search for, and subscribe to notifications about that metadata.  Any networking and security technology can be a MAP client; examples include intrusion prevention system (IPS) platforms, vulnerability scanners, dynamic host configuration protocol (DHCP) servers, physical security systems such as badge access solutions, and even application servers. These components can act as sensors adding data to the MAP and/or act upon information received from other components.

Now, more than ever, organizations interconnecting control system networks with corporate IT networks need to be aware of potential risks. Planning, processes, and technology are required to adequately reduce exposure, mitigate the risks associated with a hyper-connected environment, and prepare the infrastructure to securely handle change.

The current trend toward higher levels of integration between enterprise and control/SCADA networks will continue to accelerate as operators seek improved productivity and return on investment (ROI). However, this ROI will not be realized without significant improvements in control system security. TNC and MAP provide an open ecosystem of interfaces, tools, and products that enable robust and flexible security architectures to be deployed quickly and cost-effectively. Moreover, integration of specialized security products demonstrates that open standards from TNC enable management of security policy for both the enterprise and control networks from a single set of tools, offering high levels of security in a very cost-effective solution.  

Byres Security.

Lantzville, BC, Canada.
(877) 297-3799.
[www.tofinosecurity.com].

Juniper Networks.
Sunnyvale, CA.
(888) 586-4737.
[www.juniper.net].

The Trusted Computing Group.
[www.trustedcomputinggroup.org].