Framework helps streamline the software-testing process

System helps software developers test large embedded systems faster and more thoroughly.
By Control Engineering Staff July 24, 2008

Alameda, CA — “Quality is a corner office issue and is directly related to brand and revenue impact,” says Amit Ronen, vice president and general manager of device management at embedded-system development tools provider Wind River . “By introducing Test Management, Wind River is significantly reducing the complexity of the quality assurance process, which brings with it significant cost savings and time-to-market opportunities.”
Test Management is a test framework that resides on the enterprise-level data servers carrying latest-builds of software products released by individual code-module programmers. Its purpose is to help software-test managers plan comprehensive test suites, evaluate test coverage, and manage the bug-fixing process.
Authoritative estimates peg software at 80-99% of the value of an embedded product. Of that, according to an informal survey of Wind River customers, software testing accounts for 30-50% of cost. Even with this huge software-testing investment, however, bugs still appear in embedded controls for products reaching prototype functional test, and even production units reaching the field. Ronen reports that developers find 80% of those bugs appear in sections of code that later analysis shows were never tested.
Some code modules are allowed to escape testing deliberately for extraneous reasons, such as cost or time-to-market, but a significant portion escape testing inadvertently. Software test managers believed that code to have been tested, when later analysis showed that existing test suites — believed to have been comprehensive — simply didn’t cover it.
Individual programmers still are responsible for ensuring that their modules perform according to specifications, according to Amit Ronen, vice president and general manager of device management at Wind River, while software test departments at the enterprise level ensure quality of completed distributions. They address software quality issues that only appear when modules must work together, as well as providing a second set of eyes to spot intra-module bugs escaping programmers’ scrutiny.
One of the software’s important features, Ronen points out, is the ability to insert patches into software systems without having to do a complete rebuild. Thus, he says, the test department can isolate a bug, send it back to the originating programmer for repair, then insert the corrected module into the existing build as a patch.
Other significant features include:
• Test planning facilities, which enable test teams to create manual and automated test cases, and assign each test to a designated engineer.
• A test execution framework, which automatically executes multiple test suites on selected target devices.
• Automatic collection of test and fault data, giving developers the tools to quickly reproduce defects.
• A diagnostics interface that enables test and development engineers to shorten the defect resolution process and reduce risk of project delays.
• Auto-generated “dynamic” code coverage and performance measurement, which is collected without special test builds or additional test harnesses.
• Virtual Lab Management, which enables the project team to manage boards, provision new software builds, and share access to boards.
• Integration into Wind River Workbench Eclipse2-based development tool suite.
• Support for devices running VxWorks and the company’s Linux run-time platforms.
Test Management is available today in an early-access version, with additional functionality planned for release later in third-quarter 2008.
C.G. Masi , senior editor
Control Engineering Machine Control eNewsletter
Register here and scroll down to select your choice of eNewsletters free.