Using test cases as a cost and risk control tool

Bugs going unseen until production end up costing even more time and money. Testing should be viewed as a cost and risk control tool upfront during project planning.


We have all been on or seen software development projects that did not end up getting executed as planned for various reasons such as scope changes, excessive costs, or scheduling issues. Seeing how software projects are actually executed, especially when the aforementioned scenarios are in play, has highlighted how important it is to structure the project up-front.

It is unfortunate, but not uncommon, to see testing as being one of the first areas cut or cut back when a project is under cost or schedule pressure. Not only does that strategy not have the intended effect, it almost always does the opposite. Testing less or later in the process doesn't change the quality of the solution. All it does is delay issue detection.

Finding the same issue early in the development process verses at the end, i.e. in unit testing vs production operation, is exponentially cheaper and is almost always requires less time to fix. To highlight this point, consider the scenario where a bug is found in a validated production environment:

  • The issue must be identified, reported, and potentially worked around. 
  • The issue must be reproduced, triaged, and assigned a priority.
  • The issue will likely be reproduced again by the developer and fixed.
  • The fix will need to be deployed and tested in a test environment.
  • The fix will need to be deployed and tested in a validation environment.
  • The fix will need to be deployed to the production environment.

The steps listed above don't include approvals, code reviews, status reporting, change management, etc. However, this still illustrates that more time and resources are required to find and fix a bug that makes it to production instead of doing this during unit testing during initial development. While this is a worst-case scenario, it still illustrates the overall point: identifying bugs later in the process is expensive. And this is before the costs associated with potentially impacting the product, customer, or company brand are considered.

Bugs going unseen until production end up costing even more time and money. Testing should be viewed as a cost and risk control tool upfront during project planning. Courtesy: Maverick TechnologiesThe farther from development a bug is found, the more expensive it is to fix. For this reason, testing should be viewed as a cost and risk control tool upfront during project planning. Adjusting the project plan to create test cases as part of, or in parallel with, the functional and technical design is very effective because:

  1. It ensures that proper test case creation is done up front. In doing so, it prevents well-intentioned decision makers from being able to reduce or cut test case creation from scope later in the project if/when the project is under pressure. We are structuring the project to protect us from ourselves.
  2. The process of creating test cases often emphasizes how the final solution will look to the end user. Working through the process of creating the test cases and reviewing them with stakeholders and end users prior to development, provides an opportunity for miss-communication and gaps to be identified before coding begins which reduces the risk of re-work.
  3. The test cases provide developers additional context that augments and clarifies the designs. This helps ensure that developers clearly understand what the solution should look like which reduces the likelihood that bugs will be introduced in the first place. Avoiding bugs entirely is a huge time and cost savings. Especially when compared to the minimal time investment of creating test cases; test cases that would have been likely created later in the project anyways.
  4. Developers now have a robust test script to follow when performing their unit testing that should closely match the final validation test scripts. The cheapest place to find and fix a bug is during unit testing so giving developers additional tools that increase the up-front detection rate is incredibly valuable.

While making such a small shift to require test case creation prior to development may seem trivial on the surface, in practice it can have a profound positive impact on the project.

What are some other testing related tips and tricks that you have found to help detect bugs earlier in the project and reduce overall costs?

This post was written by Jason Sampers. Jason is an application architect at Maverick Technologies, a leading automation solutions provider offering industrial automation, strategic manufacturing, and enterprise integration services for the process industries. Maverick delivers expertise and consulting in a wide variety of areas including industrial automation controls, distributed control systems, manufacturing execution systems, operational strategy, business process optimization, and more.

Maverick Technologies is a CSIA member as of 6/28/2016.

No comments
The Engineers' Choice Awards highlight some of the best new control, instrumentation and automation products as chosen by...
The System Integrator Giants program lists the top 100 system integrators among companies listed in CFE Media's Global System Integrator Database.
The Engineering Leaders Under 40 program identifies and gives recognition to young engineers who...
This eGuide illustrates solutions, applications and benefits of machine vision systems.
Learn how to increase device reliability in harsh environments and decrease unplanned system downtime.
This eGuide contains a series of articles and videos that considers theoretical and practical; immediate needs and a look into the future.
Robotic safety, collaboration, standards; DCS migration tips; IT/OT convergence; 2017 Control Engineering Salary and Career Survey
Integrated mobility; Artificial intelligence; Predictive motion control; Sensors and control system inputs; Asset Management; Cybersecurity
Big Data and IIoT value; Monitoring Big Data; Robotics safety standards and programming; Learning about PID
Featured articles highlight technologies that enable the Industrial Internet of Things, IIoT-related products and strategies to get data more easily to the user.
This article collection contains several articles on how automation and controls are helping human-machine interface (HMI) hardware and software advance.
This digital report will explore several aspects of how IIoT will transform manufacturing in the coming years.

Find and connect with the most suitable service provider for your unique application. Start searching the Global System Integrator Database Now!

Mobility as the means to offshore innovation; Preventing another Deepwater Horizon; ROVs as subsea robots; SCADA and the radio spectrum
Future of oil and gas projects; Reservoir models; The importance of SCADA to oil and gas
Big Data and bigger solutions; Tablet technologies; SCADA developments
Automation Engineer; Wood Group
System Integrator; Cross Integrated Systems Group
Jose S. Vasquez, Jr.
Fire & Life Safety Engineer; Technip USA Inc.
This course focuses on climate analysis, appropriateness of cooling system selection, and combining cooling systems.
This course will help identify and reveal electrical hazards and identify the solutions to implementing and maintaining a safe work environment.
This course explains how maintaining power and communication systems through emergency power-generation systems is critical.
click me