Standard Middleware Helps Systems Adapt
The Yocto Project: New Possibilities for Intelligent Embedded Systems?
Bringing together and integrating software components for embedded designs can be a tedious and uncertain task. The creation of a project framework under which these components can be identified, evaluated and integrated can save time and effort, freeing talent for more value-added activities.
LAX MANDAL & DAVID STEWART, INTEL
Page 1 of 1
Anyone familiar with the embedded technology industry will acknowledge that the approaches to developing embedded systems are very fragmented. They are fragmented in the way the systems are designed, in what I/Os and OSs are implemented, what middleware or applications are used, and because of the fact that there is limited or no interoperability between these systems. Embedded developers usually adopt a headstrong methodology or a sense of design philosophy that may not be compatible with others. While this has worked in the past and has helped embedded systems become entrenched in our lives and businesses, a new breed of systems is emerging. These systems are connected and serve as end points of a networked deployment or installation, and they will need a more consistent approach to development. These, as International Data Corporation puts it, are “intelligent systems.”
An intelligent system is essentially an embedded system that is connected, enables intelligent operations and is in lock step with other such systems to either distribute workloads or create a chain of values delivery. This encompasses a breed of systems that would promote the needs for stream-lined applications, cloud connectivity and possibly a much greater level of user-interaction. Intelligent systems will also provide an opportunity for developers and businesses to create and deploy end-to-end solutions as opposed to traditionally limited point solutions.
The biggest bump on the road to deploying such intelligent systems widely is the ability to efficiently develop such systems themselves. This is particularly due to the lack of synergies and portability of various software components between cross-architectural hardware solutions. Developers spend an extraordinary amount of time and energy on building solutions that may not be utilized in their next project. In most cases developers also resort to “beg and borrow” tactics toward software development in order to be able to fulfill short-term needs that may result in unattended gaps. There has so far not been a framework that allows developers to easily develop cross-architecture solutions, easily port solutions from one project to another while not losing consistency in software architecture, and seamlessly integrate standard middleware and applications. Without such a framework, it will be difficult to: a) broaden the scope of the intelligent systems beyond mundane processing capabilities and b) accelerate intelligent systems deployment.
The need for such a framework is more apparent in the open source world than in the closed source world, where device support is dictated up-front. The Yocto Project is one such initiative by the Linux foundation whose goal is to bring down the barriers engrained in current methodologies and software development practices by providing a universal framework for development.
So What Is the Yocto Project?
The Yocto Project (www.yoctoproject.org) is an open source collaboration project that provides templates, tools and methods to help developers create custom Linux-based systems for embedded products on any hardware architecture.
The approach of the Yocto Project is a bit different from the traditional approaches in the Linux world. The Yocto Project is not the end game for a hardware platform, but a means to achieve the end game objectives. In plain words, it’s not an embedded Linux distribution; rather, it creates a custom one for you. Figure 1 shows the major components of the Yocto Project and how information flows from start to completion.
In the Yocto Project packages are brought in from various sources and run through the build system to ultimately build runnable Linux images.
In a Yocto Project build, specified open source packages from around the Internet and your local intranet are accessed and the associated source code is fetched. These sources are then patched and compiled into binary packages. Rather than lock system developers into a single packaging format, the Yocto Project supports all the most popular package systems (RPM, DEB and IPK). These packages are then emitted into a package feed that can be used to build runnable Linux images. The package feed can be used to supply additional binaries to the devices under development. In addition, the build process can create an application development SDK, which is based on the popular Eclipse development environment. Images can be tested on a device emulator (called QEMU) or on real devices, and tools are provided in the Yocto Project to debug or profile the resulting system.
The Yocto Project brings together the elements needed to make the embedded Linux development process easier and faster and is easily reusable or updated. Thus there are a number of implementation engineers who have created ready-to-go Linux distributions on Intel platforms in a tiny fraction of the time previously required.
For an example of the Yocto Project’s philosophy, consider this quote from one of the project’s documents, the Yocto Project Kernel Architecture and Use Manual:
The complexity of embedded kernel design has increased dramatically. Whether it is managing multiple implementations of a particular feature or tuning and optimizing board specific features, flexibility and maintainability are key concerns. The Yocto Project Linux kernel is presented with the embedded developer’s needs in mind and has evolved to assist in these key concerns. For example, prior methods such as applying hundreds of patches to an extracted tarball have been replaced with proven techniques that allow easy inspection, bisection and analysis of changes. Application of these techniques also creates a platform for performing integration and collaboration with the thousands of upstream development projects.
With all these considerations in mind, the Yocto Project kernel and development team strive to attain these goals:
• Allow the end user to leverage community best practices to seamlessly manage the development, build and debug cycles.
• Create a platform for performing integration and collaboration with the thousands of upstream development projects that exist.
• Provide mechanisms that support many different work flows, front-ends and management techniques.
• Deliver the most up-to-date kernel possible while still ensuring that the baseline kernel is the most stable official release.
• Include major technological features as part of Yocto Project’s up-rev strategy.
What is also interesting is the support that this initiative is generating from the key components players in the embedded industry. Industry leaders such as Wind River, Texas Instruments, Montavista Software, Panasonic, Intel and many others have pledged support for the project. This signals a broader understanding that Linux-based system developers see the opportunity to focus on higher value add in areas such as solution stack buildout, applications and user experiences rather than be content with solving the system integration challenges. In essence, the Yocto Project provides a prescription for system integration challenges, so developers can easily shift their focus and system deployments can take advantage of that by considering rolling out value-added services.
Now Give Me an Example, Please…
Late last year, engineers contributing to the Yocto Project embarked on a Network Attached Storage (NAS) project to demonstrate how possible it would be to create solutions built with a fast time-to-market objective. Based on an Intel Black Sand (Intel Atom N450) reference single- board computer platform, the project was initiated to integrate a samba server and various related functionalities.
Shane Wang, lead engineer on the NAS project said, “In order to test Yocto’s usability to build images for embedded devices, we chose a personal NAS based on Intel’s Black Sand hardware, and expected to complete the project in a rather constrained time frame. We did some investigation on different packages, prioritized some key features, and worked out an NAS solution based on the Yocto Project from Oct. 12 to Oct 28. Now the code for Baryon is ready, the demo is also ready, and the data for preliminary profiling and tracing with ADT is ready.”
The team began by surveying the features of some existing NAS products on the market and selecting a number of file sharing services to offer, which already existed as open source projects. The team was able to quickly integrate Windows file sharing (Samba), home media sharing (Media Tomb), RAID and some Linux-oriented services such as NFS, FTP, SSH and RSYNC. Because an NAS typically has no display screen, management would be done using a browser interface, and the open source Webmin project was integrated with the Yocto Project’s built-in web server. Finally, the Application Development Toolkit created by the Yocto Project was used to analyze and tune performance of the NAS, invoking such tools as blktrace, perf, oprofile, iozone, perftop and netperf. Creating a finished quality product in only three weeks was not the goal, nor would this likely be achievable. However, the result is amazingly competitive compared to commercial offerings, as outlined on the Baryon project web page, and is a clear demonstration of the power of using the Yocto Project to quickly create a custom Linux device. More details of the Baryon project can be found at https://wiki.yoctoproject.org/wiki/Baryon.
What this example showcased was that system integration is less arduous and can free up a lot of developer time that otherwise would be spent trying to do a great deal of manual integration and validation. Now that precious developer time that was freed up can be utilized toward integrating value-added capabilities. This time can also be utilized in creating additional product offerings, or possibly some cross-architectural work based on the setup that was used in the previous design, to meet the niche market requirements.
Cross-architectural systems based on similar development frameworks can be ported back and forth much more easily, which is a huge plus in an environment where there are high expectations around homogeneity of applications. This enables the deployments to have access to higher-value capabilities, in particular with regard to manageability and security. In a connected environment, the ability to manage a network of intelligent systems and to keep it secure would enable a new dimension of operational excellence and easier delivery of services, thus increasing productivity significantly. Various market sectors such as industrial factories, energy distribution, networking, retail, automotive, machine to machine, etc., can benefit from this progression of OS-level simplification to enrich services deployment.
What Else Do I Need to Know?
The Yocto Project is certainly not indicative of the way the Linux marketplace has operated. It may not be the ultimate model either. However, what it provides is a set of benefits that the embedded industry can utilize to get to the next step of evolution. Various market analyses point at an explosion of connected end-point systems that are proposed to be intelligent systems catering to local and networked computational needs. This enables opportunities to deploy value-added services, increasingly from a cloud computing perspective (both public and private cloud-oriented delivery). However, to be able to be in lockstep with these evolutionary potentials, systems development needs to be easier and faster.
The road to success for embedded systems depends on the agility of the industry to evolve from mundane devices to intelligent systems. Only time will tell how successful the Yocto Project can be, but it’s very obvious that the approaches toward developing solutions and integrating systems need to change. The Yocto Project enables one such change.
Santa Clara, CA.