Developers are barraged with claims that developments based on open source software are less expensive that those developments using commercial OSes. This paper presents data based on development information from more than 2700 developers that discuss this issue.
By Jerry Krasner, Embedded Market Forecasters
Theodore Levitt, the renowned Harvard professor used to tell his students “People don’t want to buy a quarter-inch drill; they want a quarter-inch hole”. Following this line of thought, prices may vary depending on the type of drill required but the true cost of getting their quarter-inch hole depends more on the cost of the drill used as well as the time required of someone to drill it. Similarly, the acquisition of the RTOS is not for the RTOSes sake – but rather a choice in search of a cost effective and time sensitive design/development solution.
The RTOS enigma can be seen in the same light. It matters little if the drill bit is free or if it comes at a price. The true cost includes more than the acquisition cost.
Intuitively, the idea that a “free” RTOS reduces development costs can be no more than fanciful thinking – it depends on facts, not on anecdotes.
Anecdotes are not data – they supplement ideas for which no data exists (true or false) to provide guidance to developers, managers and CFOs
Every day developers are barraged with claims and counter claims regarding the use of RTOSes, development tools, programming languages, and more. FUD has become less than an idea and more of an institutional malaise. It frequently masquerades as a reasonable idea – open source software, for example, is “free” or so their proponents proclaim. But is there any comprehensive source of user data that can validate this claim? In this paper we will present a financial comparison (Cost of Development) between the RTOS industry, a specific example of a commercial RTOS, Linux developments and open source developments. The results may be surprising to some and not so to others.
The data presented herein is based on the results of the 2016 Annual EMF Embedded Developer Survey (1298 respondents) and is nearly identical to the data derived from 1337 respondents in 2015.
In developing this cost data (using our unique EMF Dashboard www.embeddedforecast.com) we were able to do a simultaneous comparison of the data presented in the following Tables by determining the number of software developers per project, the time period from design start to project/product shipment, and the percentage of designs completed behind schedule.
EMF, based on the responses to their 2016 Survey of Embedded Developers (1298 respondents), created six cadres of data by filtering the responses to developers using any RTOS; those using a commercially available (3 examples); those using Linux and those having reported using open source software. As this is a vendor agnostic paper, the commercial RTOSes are referred to as Commercial A, Commercial B and Commercial C.
The data is accurate – just the names have been withheld.
Simultaneously arranging them in matrix format comparative data was derived from the database as follows:
- Number of software and hardware developers per project
- Number of months between project start and project completion (development duration)
- Percentage of developments completed behind schedule and the number of months of behind schedule activity
These data can be used to calculate the average Total Cost of Development.
- Multiplying the number of developers per project by the development duration yields “Base Development Cost in Man Months”
- Multiplying the number of developers per project by the percent of behind schedule completions and by the number of months the development was behind yields “Behind Schedule Costs in Man Months”
- Combining these “Costs in Man Months” yields “Total Cost of Development” measured in man months.
By assuming that on average a developer man month (salary plus overhead) equal to $10,000, the comparative Total Cost of Developments for the six cadres can be shown in dollars.
Table I presents the comparative results.
What we learned from Table I:
- One needs to look at the factors that create total direct costs – not at any one particular data point. Developers can improve time-to-market by adding more developers to the project – but at an increased cost. Comparing behind schedule completions to each other can also be misleading. The cost of the loss in schedule time depends on the number of project developers as well as the number of months lost to schedule
- The Average Cost of Development (measured in man months) is measured by multiplying the number of developers required for a project by the number of months required to complete the project.
- Although the average number of developers/project is better for Commercial RTOSes A and C , Commercial RTOS B had many fewer behind schedule completions and yet RTOS A had a 33,4% lower cost of development than RTOS B
- What is important to cost sensitive managers (including CFOs) is the 6 cost figures presented in Table I. Namely Average Man Months/project; Man Months lost to behind schedule completion; Man Months lost to Cancellation; and Total Average Man Months required for completion.
- In addition, it makes sense that projects that require few months to complete don’t experience the burden of lost opportunities that longer projects may incur. Our data cannot account for this
- To complete the cost analysis one needs to include the RTOS acquisition cost. To be fair in calculating comparative costs between commercial RTOSes and open source developments, one needs to add acquisition costs to the cost figures presented in Table I
Looking at Table I might lead some readers to conclude that the average cost of development is the same for all RTOS developments, which includes Linux developments and open source developments. This is not necessarily so. There are a number of high power commercial RTOSes for which the cost of development is very high as well as a number of commercial RTOSes whose development costs are much lower than those of Linux or open source developments.
The important take away from Table I is that although most Linux and open source software is free – that is free of acquisition costs – it can’t be really free if the actual cost of development is greater than the cost of a commercial RTOS.
What Table I doesn’t take into consideration is whether the design was compromised (by removing features and capabilities) in order to meet a target window of opportunity. In such a design might appear to be cost effective but fail to reach its intended capabilities.
The 2016 survey and associated data address this possibility. EMF calls this “Design Outcomes” in which developers indicate how close their final design outcome compares with their pre-design expectation. Developers are given choices from within 10% to within 50% with 10% intervals and a final data point “not within 50%”.
Clearly “within 10%” represents the best possible design outcome and shows that feature and capability requirements were met.
Table II presents Design Outcome comparisons.
In an increasingly competitive embedded environment where cost containment and time to market are keys to success, it is important to inform developers and financial managers to the realities that extensive and comprehensive surveys can reveal. The use of Linux and open source software can be very effective – but there are reasons why developing with them is more expensive. This is a topic for a future discussion.
The purpose of this paper is an attempt to put forward a rational discussion regarding the use of commercial and so-called “free” software. The data presented in Tables I and II are based on data derived from a broad user base. Certainly, we may find that for certain verticals and for specific projects open source outcomes may prove to be less costly. The 2016 EMF database includes 32 different design projects reported by developers. In addition, we can break out developments into 4 distinct levels of complexity as well according to whether or not cadres of developers are working on IoT applications. We did not pursue these possibilities.
For example, when looking specifically at automotive/transportation applications, open source developments have been found to have the same or slightly better costs of development than those using a commercial RTOS.
What we have presented here is data regarding costs of development between commercial and free RTOSes based on year-over-year data derived from over 2600 survey respondents. Let’s base future discussions on these or other statistically accurate information – not on anecdotal stories.
About the author:
Jerry Krasner, Ph.D., MBA is VP and Chief Analyst at Embedded Market Forecasters, a Division of American Technology International, Inc. Before moving to the analyst side of the marketplace he was the co-founder of 4 medical device companies, two of which were taken public.