Using test cases as a cost and risk control tool

Bugs going unseen until production end up costing even more time and money. Testing should be viewed as a cost and risk control tool upfront during project planning.

06/28/2016


We have all been on or seen software development projects that did not end up getting executed as planned for various reasons such as scope changes, excessive costs, or scheduling issues. Seeing how software projects are actually executed, especially when the aforementioned scenarios are in play, has highlighted how important it is to structure the project up-front.

It is unfortunate, but not uncommon, to see testing as being one of the first areas cut or cut back when a project is under cost or schedule pressure. Not only does that strategy not have the intended effect, it almost always does the opposite. Testing less or later in the process doesn't change the quality of the solution. All it does is delay issue detection.

Finding the same issue early in the development process verses at the end, i.e. in unit testing vs production operation, is exponentially cheaper and is almost always requires less time to fix. To highlight this point, consider the scenario where a bug is found in a validated production environment:

  • The issue must be identified, reported, and potentially worked around. 
  • The issue must be reproduced, triaged, and assigned a priority.
  • The issue will likely be reproduced again by the developer and fixed.
  • The fix will need to be deployed and tested in a test environment.
  • The fix will need to be deployed and tested in a validation environment.
  • The fix will need to be deployed to the production environment.

The steps listed above don't include approvals, code reviews, status reporting, change management, etc. However, this still illustrates that more time and resources are required to find and fix a bug that makes it to production instead of doing this during unit testing during initial development. While this is a worst-case scenario, it still illustrates the overall point: identifying bugs later in the process is expensive. And this is before the costs associated with potentially impacting the product, customer, or company brand are considered.

Bugs going unseen until production end up costing even more time and money. Testing should be viewed as a cost and risk control tool upfront during project planning. Courtesy: Maverick TechnologiesThe farther from development a bug is found, the more expensive it is to fix. For this reason, testing should be viewed as a cost and risk control tool upfront during project planning. Adjusting the project plan to create test cases as part of, or in parallel with, the functional and technical design is very effective because:

  1. It ensures that proper test case creation is done up front. In doing so, it prevents well-intentioned decision makers from being able to reduce or cut test case creation from scope later in the project if/when the project is under pressure. We are structuring the project to protect us from ourselves.
  2. The process of creating test cases often emphasizes how the final solution will look to the end user. Working through the process of creating the test cases and reviewing them with stakeholders and end users prior to development, provides an opportunity for miss-communication and gaps to be identified before coding begins which reduces the risk of re-work.
  3. The test cases provide developers additional context that augments and clarifies the designs. This helps ensure that developers clearly understand what the solution should look like which reduces the likelihood that bugs will be introduced in the first place. Avoiding bugs entirely is a huge time and cost savings. Especially when compared to the minimal time investment of creating test cases; test cases that would have been likely created later in the project anyways.
  4. Developers now have a robust test script to follow when performing their unit testing that should closely match the final validation test scripts. The cheapest place to find and fix a bug is during unit testing so giving developers additional tools that increase the up-front detection rate is incredibly valuable.

While making such a small shift to require test case creation prior to development may seem trivial on the surface, in practice it can have a profound positive impact on the project.

What are some other testing related tips and tricks that you have found to help detect bugs earlier in the project and reduce overall costs?

This post was written by Jason Sampers. Jason is an application architect at Maverick Technologies, a leading automation solutions provider offering industrial automation, strategic manufacturing, and enterprise integration services for the process industries. Maverick delivers expertise and consulting in a wide variety of areas including industrial automation controls, distributed control systems, manufacturing execution systems, operational strategy, business process optimization, and more.

Maverick Technologies is a CSIA member as of 6/28/2016.



The Engineers' Choice Awards highlight some of the best new control, instrumentation and automation products as chosen by Control Engineering subscribers. Vote now (if qualified)!
The System Integrator Giants program lists the top 100 system integrators among companies listed in CFE Media's Global System Integrator Database.
Each year, a panel of Control Engineering and Plant Engineering editors and industry expert judges select the System Integrator of the Year Award winners in three categories.
This eGuide illustrates solutions, applications and benefits of machine vision systems.
Learn how to increase device reliability in harsh environments and decrease unplanned system downtime.
This eGuide contains a series of articles and videos that considers theoretical and practical; immediate needs and a look into the future.
HMI effectiveness; Distributed I/O; Engineers' Choice Award finalists; System Integrator advice; Inside Machines
Women in engineering; Engineering Leaders Under 40; PID benefits and drawbacks; Ladder logic; Cloud computing
Robotic integration and cloud connections; SCADA and cybersecurity; Motor efficiency standards; Open- and closed-loop control; Augmented reality
Programmable logic controllers (PLCs) represent the logic (decision) part of the control loop of sense, decide, and actuate. As we know, PLCs aren’t the only option for making decisions in a control loop, but they are likely why you’re here.
This digital report explains how motion control advances and solutions can help with machine control, automated control on assembly lines, integration of robotics and automation, and machine safety.
This article collection contains several articles on how advancements in vision system designs, computing power, algorithms, optics, and communications are making machine vision more cost effective than ever before.

Find and connect with the most suitable service provider for your unique application. Start searching the Global System Integrator Database Now!

Control room technology innovation; Practical approaches to corrosion protection; Pipeline regulator revises quality programs
Cloud, mobility, and remote operations; SCADA and contextual mobility; Custom UPS empowering a secure pipeline
Infrastructure for natural gas expansion; Artificial lift methods; Disruptive technology and fugitive gas emissions
Automation Engineer; Wood Group
System Integrator; Cross Integrated Systems Group
Jose S. Vasquez, Jr.
Fire & Life Safety Engineer; Technip USA Inc.
This course focuses on climate analysis, appropriateness of cooling system selection, and combining cooling systems.
This course will help identify and reveal electrical hazards and identify the solutions to implementing and maintaining a safe work environment.
This course explains how maintaining power and communication systems through emergency power-generation systems is critical.
The Engineers' Choice Awards highlight some of the best new control, instrumentation and automation products as chosen by Control Engineering subscribers. Vote now (if qualified)!
The System Integrator Giants program lists the top 100 system integrators among companies listed in CFE Media's Global System Integrator Database.
Each year, a panel of Control Engineering and Plant Engineering editors and industry expert judges select the System Integrator of the Year Award winners in three categories.
This eGuide illustrates solutions, applications and benefits of machine vision systems.
Learn how to increase device reliability in harsh environments and decrease unplanned system downtime.
This eGuide contains a series of articles and videos that considers theoretical and practical; immediate needs and a look into the future.
HMI effectiveness; Distributed I/O; Engineers' Choice Award finalists; System Integrator advice; Inside Machines
Women in engineering; Engineering Leaders Under 40; PID benefits and drawbacks; Ladder logic; Cloud computing
Robotic integration and cloud connections; SCADA and cybersecurity; Motor efficiency standards; Open- and closed-loop control; Augmented reality
Programmable logic controllers (PLCs) represent the logic (decision) part of the control loop of sense, decide, and actuate. As we know, PLCs aren’t the only option for making decisions in a control loop, but they are likely why you’re here.
This digital report explains how motion control advances and solutions can help with machine control, automated control on assembly lines, integration of robotics and automation, and machine safety.
This article collection contains several articles on how advancements in vision system designs, computing power, algorithms, optics, and communications are making machine vision more cost effective than ever before.

Find and connect with the most suitable service provider for your unique application. Start searching the Global System Integrator Database Now!

Control room technology innovation; Practical approaches to corrosion protection; Pipeline regulator revises quality programs
Cloud, mobility, and remote operations; SCADA and contextual mobility; Custom UPS empowering a secure pipeline
Infrastructure for natural gas expansion; Artificial lift methods; Disruptive technology and fugitive gas emissions
Automation Engineer; Wood Group
System Integrator; Cross Integrated Systems Group
Jose S. Vasquez, Jr.
Fire & Life Safety Engineer; Technip USA Inc.
This course focuses on climate analysis, appropriateness of cooling system selection, and combining cooling systems.
This course will help identify and reveal electrical hazards and identify the solutions to implementing and maintaining a safe work environment.
This course explains how maintaining power and communication systems through emergency power-generation systems is critical.
The Engineers' Choice Awards highlight some of the best new control, instrumentation and automation products as chosen by Control Engineering subscribers. Vote now (if qualified)!
The System Integrator Giants program lists the top 100 system integrators among companies listed in CFE Media's Global System Integrator Database.
Each year, a panel of Control Engineering and Plant Engineering editors and industry expert judges select the System Integrator of the Year Award winners in three categories.
This eGuide illustrates solutions, applications and benefits of machine vision systems.
Learn how to increase device reliability in harsh environments and decrease unplanned system downtime.
This eGuide contains a series of articles and videos that considers theoretical and practical; immediate needs and a look into the future.
HMI effectiveness; Distributed I/O; Engineers' Choice Award finalists; System Integrator advice; Inside Machines
Women in engineering; Engineering Leaders Under 40; PID benefits and drawbacks; Ladder logic; Cloud computing
Robotic integration and cloud connections; SCADA and cybersecurity; Motor efficiency standards; Open- and closed-loop control; Augmented reality
Programmable logic controllers (PLCs) represent the logic (decision) part of the control loop of sense, decide, and actuate. As we know, PLCs aren’t the only option for making decisions in a control loop, but they are likely why you’re here.
This digital report explains how motion control advances and solutions can help with machine control, automated control on assembly lines, integration of robotics and automation, and machine safety.
This article collection contains several articles on how advancements in vision system designs, computing power, algorithms, optics, and communications are making machine vision more cost effective than ever before.

Find and connect with the most suitable service provider for your unique application. Start searching the Global System Integrator Database Now!

Control room technology innovation; Practical approaches to corrosion protection; Pipeline regulator revises quality programs
Cloud, mobility, and remote operations; SCADA and contextual mobility; Custom UPS empowering a secure pipeline
Infrastructure for natural gas expansion; Artificial lift methods; Disruptive technology and fugitive gas emissions
Automation Engineer; Wood Group
System Integrator; Cross Integrated Systems Group
Jose S. Vasquez, Jr.
Fire & Life Safety Engineer; Technip USA Inc.
This course focuses on climate analysis, appropriateness of cooling system selection, and combining cooling systems.
This course will help identify and reveal electrical hazards and identify the solutions to implementing and maintaining a safe work environment.
This course explains how maintaining power and communication systems through emergency power-generation systems is critical.
click me