Six debugging techniques for embedded system development

Learning and propagating various debugging techniques reduces bug count and increases code quality. Here are six techniques that can go a long way in assisting the debugging process. Reset software coding expectations and get better results for embedded systems development.

April 2, 2015

Solid design practices (such as low coupling, well-defined interfaces) will go a long way to help reduce bug counts. After that, systems must be, or, depending on attitude, get to be debugged. We tend to be irritated and even surprised by the existence of bugs. It doesn’t have to be that way. It’s just another facet of problem solving. With the right tools and techniques, along with resetting our expectations, the debugging process can actually be fun.

Learning and propagating various debugging techniques reduces bug count during development and increases code quality. Try having your more experienced engineers create a short course on techniques and having the younger engineers learn from them.

Although not an exhaustive list, these six techniques can go a long way in assisting the debugging process:

1. Simplify complex data: Substitute complex analog inputs with digital, synthetic, known, and repeatable data. If a counting pattern does the trick to start, use a counting pattern. For algorithms with coefficients (such as filters), replace the production coefficients with a simpler set of coefficients (such as all zeros except one full-scale coefficient). See Figure 1.

2. Divide and conquer: Home in on the location of the problem as much as possible before trying to understand it. This can save significant time both in simulation and in the lab on hardware in the long run. To elaborate, let’s say you’ve got a chained set of processing-intensive signal processing algorithms; let’s call them A, B, C, D. When you stimulate A and observe D, you notice a problem. If you can narrow down that the input to C looks fine, but the output of C is bad, you’ve just homed in on the bug and the problem is likely within C. Now you can strip away dealing with the additional complexities of A, B, and D.

This concept also works from a bottom-up perspective. Verify that the lower level functions work first, then work up through the layers of abstraction. For example, perhaps you’ve got an analog data logger algorithm. Start by verifying the low level A/D interface, physical memory interface, and physical communication bus interface. Then move up a layer to verify functions like data handlers and routers. Finally, verify the top level of the data logger.

3. Slow the process down: Slow down clocks and data rates. Perhaps some timing requirement isn’t being met. Maybe one algorithm isn’t synchronized with another. Perhaps one of the algorithms that needs to be data-driven was accidentally coded to be clock-driven. Or maybe one device is trying to push data too quickly to another.

4. Only change one variable at a time: When there are multiple inputs, it can be valuable to isolate them and only change one input at a time, observing the system response. The trick here is to identify all the inputs that induce a change in the output of the system. This is related to fault isolation and activation.

5. Create off-line models: These models should match as closely as possible the real-time algorithm, for bit-level comparison. This is most useful for non-intuitive transformations (such as matrix operations) where it’s too challenging to understand what the output should look like relative to the input. If the real-time algorithm is fixed-point and the off-line algorithm is floating-point, there may be small discrepancies that accumulate and cause problems.

6. Start from a known-good state: Slowly tune the input until something breaks, and the bug is observable. This can be helpful for scenarios where overflow may be occurring. For example, if bit wrapping is suspected, slowly increase the amplitude at the input until the problem is observed. Similarly, if there are buffers in use, start by feeding a small amount of data into the algorithm and slowly increasing until the suspected condition occurs. Alternatively, maybe a control loop that breaks with large transients may appear fine with smaller deviations around a steady-state condition.

After looking at the problem alone for a while, a second set of eyes can certainly help as well, so don’t be afraid to have an outsider take a peek.

– David LaVine is a business development engineer at Viewpoint Systems, with knowledge and experience in the domains of systems engineering, electrical engineering, and digital engineering. Edited by Joy Chang, digital project manager, Control Engineering, jchang@cfemedia.com.

Key concepts:

• Learning and propagating various debugging techniques is crucial.
• Try having more experienced engineers create a short course on techniques and having younger engineers learn from them.
• With the right tools and techniques, along with resetting expectations, the debugging process can be fun.

Consider this

What’s your standard debugging process? Do you have the right tools and techniques to solve the problems?