Internet of Things: A Fancy Way of Saying “More Embedded Linux, Please”

Spanning server to deployed device, configurations of Linux have reached levels of flexibility, performance and real-time capability that enable it to function at all levels in the Internet of Things.


James Kirkland, Red Hat

To recap the past decade or so of embedded computing:: Linux is the dominant operating system in embedded devices and its use is increasing, while the deployment of custom or in-house operating systems has taken a sharp downward turn, according to the 2014 Embedded Market Study by UMB Tech. This remains true, even when taking Android and all of its associated “Linux or not” questions out of the equation, with the next contenders being Windows, custom OSes, and RTOS as a distant fourth.

Why is Linux so dominant? It’s not the smallest, fastest, or lowest energy OS. Memory requirements ranging from 2 MB to 512 MB preclude its use on many smaller devices. A custom OS that has been pared down and optimized for a given microprocessor or System-on-Chip (SoC) will probably deliver better performance. Figure 1 which shows the 2014 Embedded Market Study results when participants were asked, “What are the most important factors in choosing an OS?”



Figure 1. An Internet of Things solution typically adds layers of complexity to device workloads.

These results show that the top three concerns are not technical; rather, they relate to the business requirements around the long-term maintainability and commercial viability of a deployed solution. The open source nature of Linux ensures that the source code is available, that tech support is available (through the vast community of contributors and through services provided by enterprise vendors), and that there are no royalties for the software. The open source model with its published APIs and its inclination to establish or follow industry standards means that Linux is compatible with other software and systems, more so than a proprietary OS would be. As open source software, Linux can be customized, and these customizations can then in turn be contributed to upstream projects, fueling the cycle of innovation.


Configurable Footprint and Deployment Flexibility

Linux can be configured to fit on many different device types from a small flash memory drive to a gateway server or controller using SSDs or traditional hard drives for storage. Linux distributions include tools that allow developers to omit user space packages with unwanted functionality and significantly reduce the OS footprint. Similarly, developers can modify the OS to meet their specifications. Modifying or configuring an existing OS is more efficient than building a custom one yet achieves many of the same objectives.

The answer to the question of Linux’s importance to the Internet of Things, and therefore embedded systems, lies partly in recent technical advances and partly in changes to embedded devices themselves and their usage. Hardware certifications have kept up with the pace of new processors, boards, SoCs, etc. Better tools simplify the job of deploying to multiple types of devices, and the Linux kernel now includes real-time capabilities. These are just some of the innovations that make Linux suitable for more embedded use cases than before.

As for the devices themselves, while some are becoming smaller as in the case of wearables and sensors, many are getting “smarter,” especially in enterprise or industrial settings. These smart devices are called upon to perform a wider variety of increasingly complicated tasks that require more processing capacity. As the use cases grow more complex some devices start looking more like mini-servers. Figure 2 shows the three-tier architecture of a typical enterprise internet of things solution. In this example, the embedded device could be running an encryption application, lightweight messaging and ESB middleware in addition to the operating system. The embedded “mini-server” or controller is also running a combination of data-processing application  and middleware plus a portion of a business rules engine that allows it to control the device based on data it has received from the datacenter or from the device itself. Shifting some of the data processing and business rules logic away from the data center and into the field reduces the latency and cost of routing all data to the data center.


Figure 2. An Internet of Things solution typically adds layers of complexity to device workloads.

Take, for example, smart meters. Today, most of these are not all that smart functioning primarily as sensors, but they will eventually become hubs for smart houses serving as in-and-out conduit for all the systems in the house such as air conditioning, security or electric car chargers. In some areas, technology trials are taking place where the smart meter does more than transmit energy usage data—it allows energy companies to manage demand. It can delay or decrease the amount of electricity being consumed by controlling lighting, cooling, etc.

For devices like these increasingly smart meters, a robust OS like Linux, with its memory-management capabilities and services such as network management and military-grade security, is a necessity instead of overhead as might previously been thought to be the case.


Connectivity Requires Security

Another aspect of these smarter devices and their deployment scenarios is the degree to which they are connected to other devices and to enterprise data centers. Connectivity is a hallmark of the Internet of Things, a space more and more embedded devices find themselves inhabiting. Smarter devices, that is. Devices collecting, processing and transmitting more data, connected to critical resources pose a significant security risk—even when the devices are on a private network. Some of the recent retail data breaches occurred when private networks where data was being transmitted in clear text were compromised.

Many distributions of Linux include a security-enhanced option called SELinux which has earned the highest government certifications. SELinux has built-in context-defined, policy-based security. With other operating systems and some Linux distributions, security is incumbent on the system architects and the amount of effort they put into it.


Real-time Linux Comes of Age

Real-time capabilities are critical in any use case where physical state must be considered and operations must occur in a specified order according to regimented timing. Robotics, manufacturing, and control systems are just a few of the areas where a real-time system is found in embedded devices.

As an example of Linux’s real-time capabilities, the U.S. Air Force has replaced RTOS in the anti-missile systems on its Lockheed AC-130s. The original system consisting of a MIPs chip running RTOS has been exchanged for a less expensive solution: a Linux laptop that has been certified as meeting real-time specifications connected to a bus that detects the missile and dispenses the counter measures, such as chaff and flares. Linux was able to meet requirements for predictable ordering of operations that used to be reserved for an RTOS.

In 2006, real-time capabilities were added to the 2.6 Linux kernel. Since that time, better schedulers and locking algorithms have increased determinism and decreased latency. Some Linux distributions provide real-time variants while others provide a kernel patch (PREEMPT_RT) that can be used to upgrade Linux to function as a real-time system.

An RTOS offers comprehensive real-time capabilities—both hard and soft real time. Linux does not have a deadline scheduler to support hard real time where processes are forced off the CPU in order to allow the process with a firm deadline to complete on time. Linux provides only soft real time where every process receives a fair share of resources and every effort is made to respect deadlines by increasing a job’s priority. For scenarios where soft real time is all that’s needed, Linux can offer a less expensive and more flexible alternative to an RTOS.


Linux Tools for Configuring Real-Time Environments

In addition to core real-time capabilities, Linux includes a set of tools for optimizing and configuring how real-time systems perform. Developing an application that performs well on a real-time system requires attention to how processes consume resources to avoid latency traps. For example, instead of using global locks, use specific resource locks to ensure that other processes don’t get blocked by a process that can’t be pre-empted. The Linux tuna and ltrace tools can help identify latency traps so that you can program an application in such a way that processes aren’t waiting on resources. Tuna allows you to change scheduler and interrupt request settings at the CPU level or by thread. Ltrace is a debugging tool that identifies shared library and system calls and is useful for seeing where an application might have run into resource contention.

Another Linux feature, control groups (or c-groups), allows you to control resource allocation. For example, you can use control groups to pin all the hardware interrupt requests to a certain CPU and ensure that a process gets higher priority on that CPU. This avoids one of the latency traps caused when a process gets bounced between CPUs, loses access to its cached data and has to pull data back from memory. Control groups also let you assign I/O and network resources to certain processes. Whereas control groups let you dynamically manage resources, for an embedded deployment you would define resource allocations in a configuration file.

The Linux OS gives you fine-grained control of resources and the tools to deliver optimizations as part of the embedded application. Whether you are deploying a traditional embedded device or an Internet of Things solution, Linux allows you to optimize for a given environment and hardware without the effort, cost, and risk of customizing an OS. When you choose to embed an enterprise-class Linux platform, you are choosing to build highly reliable and widely tested technology into your embedded system.


Red Hat, Raleigh, NO. (514) 448-5100.