Not your Father’s RTOS

Sidebars: Eliminating electronic amnesia Microsoft in embedded applications

Embedded real-time operating systems keep planes, trains, and automobiles—as well as factory floor equipment—running on time. At the heart of an embedded RTOS is determinism, the absolute guarantee that when the clock ticks or an interrupt arrives, the system will respond appropriately.

But today that’s not enough. Safety, security, and communications rank among other requirements. Consider the experience of industrial control supplier Siemens Energy and Automation (SEA) of Norcross, GA. The company wanted to deploy a PC on its customer’s factory floor. According to Eric Kaczor, product marketing manager for engineering software products at SEA, engineers quickly learned that a standard office PC wouldn’t work. To avoid crashes and freezes, they decided to go with an RTOS.

However, the need to preserve office PC software applications played a role in RTOS selection. “When we chose the real-time operating system, we were looking for an environment that we could easily port our office applications to,” says Kaczor.

More functions than a PLC

Today an RTOS has to handle connectivity of various types, meet safety certifications, satisfy security requirements and, in essence, look like an office personal computer. RTOS vendors have responded with added features and capabilities, which puts pressure on memory and other constraints. Fortunately, it’s possible to slim down an RTOS considerably, ensuring that it and any applications fit in the space available. A look at the current RTOS landscape shows how it’s changing and what to look for in industrial, automation and control applications.

For its part, SEA went with the RTX RTOS from Ardence, a Citrix company, of Waltham, MA, in its Simatic Microbox 420 industrial PC. RTOS capabilities extend beyond what’s available to the casual viewer. “They look at this, and they think it’s a PLC…, not realizing that it actually has more functionality than a PLC,” says Kaczor.

Part of that extra capability is connectivity. Paul Chen, VxWorks product line manager at RTOS vendor Wind River Systems of Alameda, CA, notes that linking up with the outside world is a large requirement for state-of-the-art embedded real-time operating systems. This includes such user-expected technologies as USB, Ethernet, and wireless. End-users also expect such standards such as next-generation Internet (IPv6), the various 802.x wireless flavors, MIPv4 and MIPv6 for mobile applications, along with IPsec and HTTPS for security.

Customers guide RTOS vendors. “If the real-time operating system software wasn’t providing these technologies, our customers would be encoding those layers themselves,” says Chen.

The danger is that end-user or original equipment manufacturer (OEM) add-ons could impact the function of the software scheduler—the all-important part of an RTOS that makes it deterministic. Since they know the code, RTOS suppliers can add the capabilities without removing the real-time part of their product.

The same is true for safety and security certifications. The former shows up in airborne systems, industrial applications, and medical software under regulations that begin with three-letter acronyms: FAA DO-178B, IEC 61508, and FDA 510(k).

The added connectivity makes the job of an embedded RTOS more difficult, especially when it comes to security. One version of security is the type used in the military, which in the past has kept different security levels on different systems. Today there’s an effort to bring all levels onto one piece of hardware, which means the hardware/software combination has to keep “top secret” information top secret.

The other version of security, though, is the one familiar to anyone with an office computer, where programs can blunder into each other’s memory space, and outside forces can attack.

Chen notes that hardware improvements, the third big driver behind the-state-of-the-art in embedded RTOS, offers some help in preventing such problems. For one thing, semiconductor manufacturers are adding processing elements to their chips so that dedicated functions can be off-loaded from software.

Some functions include encryption for security and pattern matching for network virus detection. “Dedicated hardware is usually faster than software, so RTOS software needs to be able to support and leverage the various hardware engines,” says Chen.

The development of flash memory chips with enough capacity has enabled industrial systems to abandon short lived rotating hard disks and instead opt for solid state storage. Courtesy of Siemens Energy and Automation.

He adds that multicore processors are now becoming available for embedded applications. By splitting a processor into homogeneous units or cores, microprocessor manufacturers can run each at a lower clock rate, thereby cutting total power consumption while achieving better performance. Thus, an embedded RTOS could benefit from such hardware—if the software supports multicore processors.

Robert Day, vice president of marketing for Lynuxworks of San Jose, CA, notes that modern microprocessor cores offer dedicated memory sections, providing brick-wall partitioning that keeps everything separate. The concept can be extended by using a small RTOS as a hyper supervisor, allowing an RTOS and its applications to run in one part of the memory while non-real-time applications run in another.

Dan Mender, director of business development for RTOS supplier Green Hills Software Inc. of Santa Barbara, CA, notes that the use of such a separation kernel makes for a secure system and pays other dividends. He says, “The same principles that isolate and contain attacks on one area of the system can be used to create systems that are safe from inadvertent programming errors.”

Bare necessities

Aside from being able to support and take advantage of such hardware innovations, an RTOS has to meet connectivity, security, and safety demands. However, Day is quick to point to other functions of the software that are absolute necessities. In particular, there has to be some form of communication so that an application with multiple tasks running at once doesn’t get in the way of others.

“Most applications that have multiple tasks also require some form of inter-task communication, such as queues, semaphores and Mutexs, with the latter being important to applications that cannot afford to have a priority inversion issue that allows lower priority tasks to block out vital system functions,” says Day.

For industrial automation applications, Day adds that the RTOS should permit rate-monotonic scheduling and also time and space partitioning. The first supplies a ticking clock so that time windows are evenly spaced while the second ensures that critical tasks get a constant time window.

There are also some communication protocols and considerations specific to automation applications and devices. These may or may not be required because other standards, like IPv4 and IPv6, may be enough. However, having these protocols could certainly be helpful. The list includes the Controller Area Network (CAN), OPC, the Distributed Common Object Model, or DCOM, industrial WLAN for wireless communications, Profinet or other industrial Ethernet protocol, and Web services powered by XML and SOAP (which forms the foundation layer of the Web services stack, providing a basic messaging framework that more abstract layers can build on). There are also new technologies and standards, such as ZigBee in the wireless arena, that could be part of the RTOS bag of tricks.

Mark Hamilton, a field application engineer at middleware vendor Real-Time Innovations, Inc. (RTI) of Santa Clara, CA, has had extensive experience designing, developing and deploying real-time systems for the intelligence and military communities. While an extensive list of protocols and other features are nice and perhaps necessary, he cautions that there’s a limit as to how much can be piled onto an RTOS.

As more functions become part of the system, there will come a point at which the scheduler simply can’t run the software threads. In that case, RTOS determinism will suffer. Then the system may offer the functionality of a desktop computer and behave like one.

Not surprisingly he points out that middleware can help avoid such problems by managing the communication between software applications. This capability can allow the RTOS to unload some burdens via its connectivity and not bump up against scheduler constraints. Middleware even can add value by supplying some ready-made solutions to common problems.

“Middleware may provide the ability to move data very efficiently between CPUs that are a part of your system in a way where you don’t have to now worry about writing a bunch of networking code,” says Hamilton.

Such off-loading and partitioning may be necessary for another reason beyond scheduler constraints. An embedded RTOS typically operates in an environment where memory is constrained and storage restricted. At the same time, as devices add new features, applications tend to get bigger. The result is that the memory footprint of the RTOS must remain small while supporting growing device functionality.

RTOS vendors have taken several approaches to meet these competing demands. One method puts the final decision in the hands of developers and end-users. The ultimate RTOS configuration is determined by adding or removing modules during build time, allowing fine-grained control over the trade-off between capability and footprint. Potential savings with this approach can be substantial.

For example, during development and testing, users may elect to have diagnostic information points placed within the code. When development is done, these diagnostics can be removed, sometimes saving double-digit percentages in footprint size.

A corollary is that the modules themselves have to be small. That ensures that the final build and the RTOS itself will be compact. However, another solution may be to shrink the application space through a careful choice of programming language and compilers. That can have a dramatic impact on the application size. National Instruments LabView software, for instance, has options to develop, integrate, re-use existing code then port RTOS programming to a variety of targets.

As the interplay between RTOS and application space indicates, whatever steps are taken to fit within the constraints will be a function of the integration between the RTOS, applications, software tools, and hardware. Todd Brian, product marketing manager for the Nucleus Kernals product line at electronic design automation supplier Mentor Graphics Corp. of Wilsonville, OR, notes that no RTOS vendor is an island. All work within an eco-system and that fact is crucial to getting the best performance from an embedded RTOS. “A vendor needs partners that have integrated with the vendor’s OS. That way, they are not burdened by the integration process,” Brian says.

Such integration benefits RTOS suppliers and end-users, and is something to look for in an embedded RTOS and its vendor. “The complexity of device design is growing so quickly that delivering software is not enough. That software has to be integrated so the customer is not left with that burden,” Brian says.

Eliminating electronic amnesia

For systems running an embedded real-time operating system (RTOS), nonvolatile memory holds the operating system, running applications, and collected data. Memory choices abound, and candidates include battery-backed-up SRAM (serial random access memory), flash memory of various forms, and serial EEPROM (electrically erasable and programmable read-only memory).

So what’s best?

“The answer to the memory question is ‘some combination of the above,’” says Wind River’s Chen.

As for why no one size fits all, consider the static and runtime requirements of a situation. The RTOS and any applications, which could total hundreds of megabytes, may be accessed during system startup and largely untouched afterwards. Process data, on the other hand, may be only a handful of bytes but collected and stored continuously.

One approach simply replaces the spinning hard drive found in a PC with one constructed out of flash memory chips. Solid-state drives are small, environmentally rugged, and consume little power. They can be replaced as needed and even can take some of the load off the processor running the RTOS by handling read/write housekeeping.

Solid state drives have also become more affordable. Gary Drossel, vice president of planning at solid-state drive maker SiliconSystems of Aliso Viejo, CA, recalls when solid-state disks cost 40 times what similarly sized rotating ones did. That’s no longer the case. “We’re starting to see maybe a two- to three-times price premium,” he says.

Not all solid-state drives are the same, with some offering error correction and wear-leveling. The latter extends disk life by ensuring bits don’t wear out prematurely through overuse. Some devices have the ability to write protect specific areas, thus keeping the RTOS and applications pristine.

But solid-state drives share something with the rotating variety. There’s a minimum size, which currently can run in the tens of megabytes.

For smaller storage needs, users can go with flash memory, battery backed up RAM, or serial EEPROM. Several factors play a role in the choice, such as the amount of data involved and how often data is written.

Typically the RTOS and applications will be stored in flash, so the question becomes what to do with the process data. If there’s little process data involved, then serial EEPROM may be the answer, notes Martin Bowman, senior application engineer in the memory products division of Microchip Technology of Chandler, AZ. That’s particularly true for industrial applications, where long equipment lifetimes are a must.

“One of the advantages of using serial EEPROM technology is the actual number of erase/write cycles,” says Bowman.

While bit errors start to appear in flash memory after 10,000 such cycles, serial EEPROM can handle a million. The downside is that serial EEPROM is slower and isn’t as cost effective for larger storage as flash.

A final solution would be to store information remotely via an Ethernet or wireless connection, using battery-backed-up SRAM for those times when the connection is out. There’s no bit wear problem, and writing and reading data is fast. But there is the issue of the battery, which has to be monitored and maintained. Otherwise, there’s a chance for electronic amnesia to occur and data to be lost.

Microsoft in embedded applications

The standard version of Microsoft Windows cannot easily be made to understand priorities inherent in manufacturing automation and process control. Microsoft has been addressing this problem and has developed Windows versions offering various levels of size, scalability, and functionality that are aimed specifically at embedded applications. These include Microsoft Windows Embedded CE, Windows XP Embedded, and Windows Embedded for Point of Service.

For more, see

Do you have experience and expertise with the topics mentioned in this content? You should consider contributing to our CFE Media editorial team and getting the recognition you and your company deserve. Click here to start this process.