RTC Magazine

Flying a “swarm” of unmanned aerial vehicles (UAVs) will place demands on both the hardware and software aloft. Indeed it appears the performance envelope for the UAV mission today is established not by airframe, sensor array or propulsion, but by available computing power. The Windows CE operating system, with both real-time and managed code environments, may provide the best path forward to enable the swarm concept of UAV operation.

In the swarm configuration, dozens or even hundreds of UAVs must patrol in unison, and report as a unit to the warfighters on the ground. Likely, these “robots” will work with even more numerous “sta-bots” or sensor platforms recording vibration, RF, sound, images, etc. from many sites. Using today’s sensors, engines and materials, UAVs and sensor platforms can be mass-produced and can reach the cost level of disposable items, like munitions.

What is still needed is enough compute power in both hardware and software to make the swarm largely self-tending and able to work as a unit, under only general guidance from the ground controllers. The swarm should perform as George Bernard Shaw described the Roman military, “And that Legion! It fights like a creature with a thousand arms, one mind and no religion.” -Cleopatra

To deserve such praise, the swarm needs better computers and smarter software than what we are flying today. The systems must be able to assimilate real-time information from attitude, environment and visual sensors and analyze the data—in real time—within a strategic and tactical context established by broad instructions carried throughout the swarm. The swarm moves far past the current UAV notion of “eyes over the next hill”. Available COTS hardware and software components are here today to do this.

Hardware and Software

The current generation of cell phones has mandated the development of powerful 32-bit RISC systems with onboard image, acoustic, RF signal, DSP and parallel processing. If more compute power is needed, the technology to embed special-purpose “soft core” FPGAs, purpose-built CODECs or ASICs for image or spectrum analysis is well developed.

Interestingly, it seems the most expeditious way to tap all this silicon power is via a commercial OS like Windows CE. Windows CE offers several advantages for UAV and sensor-net applications. First, its real-time performance is remarkably good. Windows CE shows a high level of deterministic response, even under high load, down to time windows measured in tens of microseconds. Second, it maintains comprehensive power management, in line with the best of class RISC CPUs. Third, it contains extensive image management software. Fourth, in this same real-time environment, it can host an object-oriented application environment perfect for coordination of swarms of sensor platforms or UAVs.

Real Time

UAVs require millisecond-level real-time determinism. MEMS-based inertial sensors can quickly indicate disturbances to the UAV’s flight path. Efficient dynamic models have been developed to keep the craft on the straight and level—provided the model execution is regular and in phase with servo actuators and sensor systems. Real-time demands will increase; exciting options exist for future integration of optical, perhaps binocular optical subsystems integrated with existing attitude and direction control.

This will allow low-cost UAVs to seek and destroy the fast-moving/low-tech targets such as Rocket Propelled Grenades (RPGs), mortar shells and short-range rockets that have become one of our principle threats. Figure 1 shows the input/output response of a CE system under heavy dynamic load, measured by Maarten Struys and Michel Verhagen of PTS Software (2003). This shows that in the best case, Windows CE latency is 14.0 µseconds, in the worst case the latency is 54.4 µseconds. Jitter—the most important measure of determinism—is only 40.4 µseconds.

Low Power

Low power consumption and power management are important to most UAV and remote sensor applications. Even if power is plentiful aboard some UAVs, low power consumption allows the computer to be enclosed in an environmentally sealed container and run at elevated temperatures if needed. But for many applications, such as electric aircraft, “leave behind” sensor nets and deep-sea submersibles, low power consumption itself is a critical virtue. Windows CE offers extensive power management capabilities, matched to RISC CPUs such as the Intel PXA 27,0 which supports as many as five modes of reduced power operation (including a sleep at .18mW and a deep sleep at .09mW). Reducing long-term power demand to the level of battery leakage is quite possible, and several systems are now deployed for unattended operation for a year or more.

This level of power conservation is obtained only by integrated design throughout the system. The board itself must be partitioned for selective power management. Software drivers need to be power-managed. Power supply design must be efficient and allow for frequent power-up/power-down. Finally, the OS needs power-aware network APIs and similar extensions. The power-thrifty system is engineered to spend most time in “sleep” or some lower power mode, ready to move into full speed computation as dictated by external conditions. Applied Data Systems has further engineered these systems to support an in-built 8-bit CPU with very low power demands that can put the entire system into a “coma” mode drawing only micro amps but ready to return to action with sensor input, serial line input or clock request (Figure 2).

Imaging

Most UAVs are really remote imaging platforms. To this end, it is useful to have good compression for either still or motion pictures so they can be stored or sent back to a ground station. Windows CE is fortunately well supported with various CODECs.

Perhaps the real future of 32-bit systems in the air and in sensor nets will come with on-line image analysis based on artificial intelligence. The unit cost of a UAV with a camera payload could easily be brought down below $2,500. A sta-bot sensor platform might cost well below $1,000. At this cost, a swarm of a 1,000 UAVs and sta-bots could be economically sent aloft and otherwise deployed, provided the images could be analyzed autonomously within the swarm and sent on to ground controllers only as needed.

Current RISC CPUs such as the Intel PXA270 include a image register in the CPU, which can be accessed and manipulated perhaps ten times faster than images in memory, allowing development of powerful new image analysis algorithms. Making the most of this capability will require a new class of multi-point, collaborative image analysis software, perhaps following the early work by the MIT Media Lab and their “eye-Society” project. But, the hardware and OS to do this are available now.

The swarm will demand intelligent analysis of images at the point of capture much more than it will require high-resolution image compression and transmission. The swarm has the ability to move in close and transmit only the key images for human analysis. By flying “out of the sun” a 12-inch UAV could get very close to a potential target without detection.

Collaborative Mission Logic

The final requirement for the next-generation 32-bit UAV computer is that it must host a collaborative, multi-processor, distributed application to keep the swarm operating properly. Much work to develop this has been undertaken by the University of Pennsylvania GRASP Lab with their ROCI project, and by the MIT CSAIL Lab and James McLurkin’s large swarm of tiny Robots.

Creating collaborative systems is a challenging task. By its nature, the swarm must be a self-organizing intelligence able to act on broad direction, then differentiate, prioritize and delegate on its own (Table 1). The swarm should self-manage issues like the loss of a single UAV to hostile fire, mechanical failure or lack of fuel. There will simply be too many aircraft in the air for a person to make a decision like “this one is low on fuel, and is not doing anything important right now, so let’s bring it back”

Likewise, if the broad direction from the ground is to monitor a convoy to and from particular locations, the swarm itself should decide which UAVs to place on point and flank based on fuel reserves, surveillance equipment available, maximum/minimum speed and so forth. Basically, decisions that can be reduced to unambiguous rules should be, and those rules should then be executed by the AI distributed within the swarm.

This is a tough, but far from intractable problem. The GRASP Lab has shown that such an environment can be easily flown in a distributed UAV swarm using environments like Microsoft’s .NET. While a difficult technical problem, this task is not terribly computer-intensive. Rule-based evaluation is relatively easy and by its nature highly distributed.

For example, assume that the human controllers broadcast a mission request: search for a tall man in Arab dress in these mountains. Inside each UAV, variables are calculated and compared to limits to determine action. The UAV could calculate fuel level and average consumption to indicate a maximum range. It then knows if it is not able to satisfy a certain mission. After various UAVs signal their availability for the mission, the ready ones might arbitrate to see which one is mission leader (the one with the best high altitude camera?) The leader would then pick the UAV squad it needs for the mission.

Further, this parsing of rules would tend to be event-driven, meaning rule-based AI processing would happen at a frequency relating to changes in mission status—seconds or minutes—which is a very long time with respect to computer processing. With a thousand UAVs in the swarm, each with a hundred million instructions per second to devote to the AI of a mission

status program, the swarm would constitute a Terra-IPS level parallel processing supercomputer. A next-generation system with a 10,000-member swarm and 1 gig RISC CPUs could constitute a 100 Terra-IPS system, more powerful than most supercomputers in use today.

When we can equip UAVs with low-cost CPUs managing real time, image and sensor data and mission status programs, we see that the future effectiveness of a swarm will largely be based on the quality of its rule-based AI, and indeed might incorporate some “genetic” programming to autonomously improve its rule set. Indeed, programs written in C# and .NET incorporate the ability to actually write C# code and then run it—an AI feature not seen since LISP.

Decoupling this level of logic from the individual device and surrendering it to the network marks an interesting breakpoint in technology and suggests perhaps the name of “Skynet” to describe the resultant system. If we consider the ramifications of virus and Trojan attacks, or unintentional design “features” of the battlefield AI, we want to be very sure that the swarm sticks to Bernard Shaw’s specification and does not adopt a “religion” or other manifestation of self-consciousness. At the very least, it would be wise for a while to constrain such autonomous systems to reconnaissance and not direct combat.

Applied Data Systems
Columbia, MD.
(301) 490-4007.
[www.applieddata.net].