Using test cases as a cost and risk control tool

Bugs going unseen until production end up costing even more time and money. Testing should be viewed as a cost and risk control tool upfront during project planning.

06/28/2016


We have all been on or seen software development projects that did not end up getting executed as planned for various reasons such as scope changes, excessive costs, or scheduling issues. Seeing how software projects are actually executed, especially when the aforementioned scenarios are in play, has highlighted how important it is to structure the project up-front.

It is unfortunate, but not uncommon, to see testing as being one of the first areas cut or cut back when a project is under cost or schedule pressure. Not only does that strategy not have the intended effect, it almost always does the opposite. Testing less or later in the process doesn't change the quality of the solution. All it does is delay issue detection.

Finding the same issue early in the development process verses at the end, i.e. in unit testing vs production operation, is exponentially cheaper and is almost always requires less time to fix. To highlight this point, consider the scenario where a bug is found in a validated production environment:

  • The issue must be identified, reported, and potentially worked around. 
  • The issue must be reproduced, triaged, and assigned a priority.
  • The issue will likely be reproduced again by the developer and fixed.
  • The fix will need to be deployed and tested in a test environment.
  • The fix will need to be deployed and tested in a validation environment.
  • The fix will need to be deployed to the production environment.

The steps listed above don't include approvals, code reviews, status reporting, change management, etc. However, this still illustrates that more time and resources are required to find and fix a bug that makes it to production instead of doing this during unit testing during initial development. While this is a worst-case scenario, it still illustrates the overall point: identifying bugs later in the process is expensive. And this is before the costs associated with potentially impacting the product, customer, or company brand are considered.

Bugs going unseen until production end up costing even more time and money. Testing should be viewed as a cost and risk control tool upfront during project planning. Courtesy: Maverick TechnologiesThe farther from development a bug is found, the more expensive it is to fix. For this reason, testing should be viewed as a cost and risk control tool upfront during project planning. Adjusting the project plan to create test cases as part of, or in parallel with, the functional and technical design is very effective because:

  1. It ensures that proper test case creation is done up front. In doing so, it prevents well-intentioned decision makers from being able to reduce or cut test case creation from scope later in the project if/when the project is under pressure. We are structuring the project to protect us from ourselves.
  2. The process of creating test cases often emphasizes how the final solution will look to the end user. Working through the process of creating the test cases and reviewing them with stakeholders and end users prior to development, provides an opportunity for miss-communication and gaps to be identified before coding begins which reduces the risk of re-work.
  3. The test cases provide developers additional context that augments and clarifies the designs. This helps ensure that developers clearly understand what the solution should look like which reduces the likelihood that bugs will be introduced in the first place. Avoiding bugs entirely is a huge time and cost savings. Especially when compared to the minimal time investment of creating test cases; test cases that would have been likely created later in the project anyways.
  4. Developers now have a robust test script to follow when performing their unit testing that should closely match the final validation test scripts. The cheapest place to find and fix a bug is during unit testing so giving developers additional tools that increase the up-front detection rate is incredibly valuable.

While making such a small shift to require test case creation prior to development may seem trivial on the surface, in practice it can have a profound positive impact on the project.

What are some other testing related tips and tricks that you have found to help detect bugs earlier in the project and reduce overall costs?

This post was written by Jason Sampers. Jason is an application architect at Maverick Technologies, a leading automation solutions provider offering industrial automation, strategic manufacturing, and enterprise integration services for the process industries. Maverick delivers expertise and consulting in a wide variety of areas including industrial automation controls, distributed control systems, manufacturing execution systems, operational strategy, business process optimization, and more.

Maverick Technologies is a CSIA member as of 6/28/2016.



Engineers' Choice Awards
The Engineers' Choice Awards highlight some of the best new control, instrumentation and automation products as chosen by Control Engineering subscribers.
System Integrator Giants
The System Integrator Giants program lists the top 100 system integrators among companies listed in CFE Media's Global System Integrator Database.
System Integrator of the Year
Each year, a panel of Control Engineering and Plant Engineering editors and industry expert judges select the System Integrator of the Year Award winners in three categories.
How to Maximize Factory Automation Efficiency with Low Cost Machine Vision
This eGuide illustrates solutions, applications and benefits of machine vision systems.
Wireless Reliability in Harsh Environments
Learn how to increase device reliability in harsh environments and decrease unplanned system downtime.
Human Factors and the Impact on Plant Safety
This eGuide contains a series of articles and videos that considers theoretical and practical; immediate needs and a look into the future.
May 2018
Salary and Career Survey, IT and OT convergence, robotic standards and safety, secure circuit protection
April 2018
Cybersecurity best practices, artificial intelligence, robotic additive manufacturing, embedded systems, IIoT integration, energy efficiency
March 2018
Digitalization integration, process sensors, edge computing, fog computing, condition monitoring, and motors
Edge Computing
This article collection contains several articles on how today's technologies heap benefits onto an edge-computing architecture such as faster computing, better networking, more memory, smarter analytics, cloud-based intelligence, and lower costs.
IIoT: Machines, Equipment, & Asset Management
Articles in this digital report highlight technologies that enable Industrial Internet of Things, IIoT-related products and strategies.
PLCs
Programmable logic controllers (PLCs) represent the logic (decision) part of the control loop of sense, decide, and actuate. Featured articles in this digital report compare PLCs and programmable automation controllers (PACs), industrial PCs, and robotic controllers.
SIDB

Find and connect with the most suitable service provider for your unique application. Start searching the Global System Integrator Database Now!

June 2018
Machine learning, produced water benefits, progressive cavity pumps
April 2018
ROVs, rigs, and the real time; wellsite valve manifolds; AI on a chip; analytics use for pipelines
February 2018
Focus on power systems, process safety, electrical and power systems, edge computing in the oil & gas industry
John O. Ayuk, PE, CFSE, PMP, CAP
Automation Engineer; Wood Group
Doug Baker
System Integrator; Cross Integrated Systems Group
Jose S. Vasquez, Jr.
Jose S. Vasquez, Jr.
Fire & Life Safety Engineer; Technip USA Inc.
Data Centers: Impacts of Climate and Cooling Technology
This course focuses on climate analysis, appropriateness of cooling system selection, and combining cooling systems.
Safety First: Arc Flash 101
This course will help identify and reveal electrical hazards and identify the solutions to implementing and maintaining a safe work environment.
Critical Power: Hospital Electrical Systems
This course explains how maintaining power and communication systems through emergency power-generation systems is critical.
Engineers' Choice Awards
The Engineers' Choice Awards highlight some of the best new control, instrumentation and automation products as chosen by Control Engineering subscribers.
System Integrator Giants
The System Integrator Giants program lists the top 100 system integrators among companies listed in CFE Media's Global System Integrator Database.
System Integrator of the Year
Each year, a panel of Control Engineering and Plant Engineering editors and industry expert judges select the System Integrator of the Year Award winners in three categories.
How to Maximize Factory Automation Efficiency with Low Cost Machine Vision
This eGuide illustrates solutions, applications and benefits of machine vision systems.
Wireless Reliability in Harsh Environments
Learn how to increase device reliability in harsh environments and decrease unplanned system downtime.
Human Factors and the Impact on Plant Safety
This eGuide contains a series of articles and videos that considers theoretical and practical; immediate needs and a look into the future.
May 2018
Salary and Career Survey, IT and OT convergence, robotic standards and safety, secure circuit protection
April 2018
Cybersecurity best practices, artificial intelligence, robotic additive manufacturing, embedded systems, IIoT integration, energy efficiency
March 2018
Digitalization integration, process sensors, edge computing, fog computing, condition monitoring, and motors
Edge Computing
This article collection contains several articles on how today's technologies heap benefits onto an edge-computing architecture such as faster computing, better networking, more memory, smarter analytics, cloud-based intelligence, and lower costs.
IIoT: Machines, Equipment, & Asset Management
Articles in this digital report highlight technologies that enable Industrial Internet of Things, IIoT-related products and strategies.
PLCs
Programmable logic controllers (PLCs) represent the logic (decision) part of the control loop of sense, decide, and actuate. Featured articles in this digital report compare PLCs and programmable automation controllers (PACs), industrial PCs, and robotic controllers.
SIDB

Find and connect with the most suitable service provider for your unique application. Start searching the Global System Integrator Database Now!

June 2018
Machine learning, produced water benefits, progressive cavity pumps
April 2018
ROVs, rigs, and the real time; wellsite valve manifolds; AI on a chip; analytics use for pipelines
February 2018
Focus on power systems, process safety, electrical and power systems, edge computing in the oil & gas industry
John O. Ayuk, PE, CFSE, PMP, CAP
Automation Engineer; Wood Group
Doug Baker
System Integrator; Cross Integrated Systems Group
Jose S. Vasquez, Jr.
Jose S. Vasquez, Jr.
Fire & Life Safety Engineer; Technip USA Inc.
Data Centers: Impacts of Climate and Cooling Technology
This course focuses on climate analysis, appropriateness of cooling system selection, and combining cooling systems.
Safety First: Arc Flash 101
This course will help identify and reveal electrical hazards and identify the solutions to implementing and maintaining a safe work environment.
Critical Power: Hospital Electrical Systems
This course explains how maintaining power and communication systems through emergency power-generation systems is critical.
Engineers' Choice Awards
The Engineers' Choice Awards highlight some of the best new control, instrumentation and automation products as chosen by Control Engineering subscribers.
System Integrator Giants
The System Integrator Giants program lists the top 100 system integrators among companies listed in CFE Media's Global System Integrator Database.
System Integrator of the Year
Each year, a panel of Control Engineering and Plant Engineering editors and industry expert judges select the System Integrator of the Year Award winners in three categories.
How to Maximize Factory Automation Efficiency with Low Cost Machine Vision
This eGuide illustrates solutions, applications and benefits of machine vision systems.
Wireless Reliability in Harsh Environments
Learn how to increase device reliability in harsh environments and decrease unplanned system downtime.
Human Factors and the Impact on Plant Safety
This eGuide contains a series of articles and videos that considers theoretical and practical; immediate needs and a look into the future.
May 2018
Salary and Career Survey, IT and OT convergence, robotic standards and safety, secure circuit protection
April 2018
Cybersecurity best practices, artificial intelligence, robotic additive manufacturing, embedded systems, IIoT integration, energy efficiency
March 2018
Digitalization integration, process sensors, edge computing, fog computing, condition monitoring, and motors
Edge Computing
This article collection contains several articles on how today's technologies heap benefits onto an edge-computing architecture such as faster computing, better networking, more memory, smarter analytics, cloud-based intelligence, and lower costs.
IIoT: Machines, Equipment, & Asset Management
Articles in this digital report highlight technologies that enable Industrial Internet of Things, IIoT-related products and strategies.
PLCs
Programmable logic controllers (PLCs) represent the logic (decision) part of the control loop of sense, decide, and actuate. Featured articles in this digital report compare PLCs and programmable automation controllers (PACs), industrial PCs, and robotic controllers.
SIDB

Find and connect with the most suitable service provider for your unique application. Start searching the Global System Integrator Database Now!

June 2018
Machine learning, produced water benefits, progressive cavity pumps
April 2018
ROVs, rigs, and the real time; wellsite valve manifolds; AI on a chip; analytics use for pipelines
February 2018
Focus on power systems, process safety, electrical and power systems, edge computing in the oil & gas industry
John O. Ayuk, PE, CFSE, PMP, CAP
Automation Engineer; Wood Group
Doug Baker
System Integrator; Cross Integrated Systems Group
Jose S. Vasquez, Jr.
Jose S. Vasquez, Jr.
Fire & Life Safety Engineer; Technip USA Inc.
Data Centers: Impacts of Climate and Cooling Technology
This course focuses on climate analysis, appropriateness of cooling system selection, and combining cooling systems.
Safety First: Arc Flash 101
This course will help identify and reveal electrical hazards and identify the solutions to implementing and maintaining a safe work environment.
Critical Power: Hospital Electrical Systems
This course explains how maintaining power and communication systems through emergency power-generation systems is critical.
click me