Do not ignore those recurring errors in manufacturing or processing

Error messages matter, a lot. Find them, track them, and remove them. Adopt a 4-point no-errors policy. Can you answer 4 questions about errors, warnings, and alarms in manufacturing and processing? Learn from Anthony Baker.

07/26/2013


You know the manufacturing automation or process control errors we are talking about: The nagging errors in the log that don’t seem to go away. The system is “running fine” but there’s a tag in the PLC or other controller that doesn’t exist, or a link in the application that doesn’t compile completely. The lesson here is simple, but one that lots of engineers ignore:

NO ERRORS NO WARNINGS.

No errors, no warnings, advises Callisto IntegrationAnthony Baker received a recent call from a very stressed out, very important client. The client had an human-machine interface (HMI) software system that would become unresponsive after 12-15 hours of uptime. The onsite engineers were unsure of the cause and had called Anthony as a last resort. The system was in a fragile state, and production was at risk.

This was a new site for Anthony and, despite having a deep understanding of this particular application, Anthony did what lots of engineers do and went into the application error logs, created a list of errors and one-by-one hunted down the root cause and eliminated the message. In many cases the error was a warning such as “Tagname ZZ does not exist in the PLC” or “Database connection Y has timed out.” All the errors could have been prevented with clean implementation and more thorough test plans that include a “NO ERRORS NO WARNINGS” policy.

Eliminating the errors in the logs one by one, the issue was eventually solved. As a note, the most frequent and obvious error was actually not the root cause of the outage — it was a subtle, one-time issue that cropped up in the logs and was dutifully logged by the HMI application. Unfortunately, the root-cause error was drowned out by a recurring issue that had been present in the system for years. Only by fixing this error flooding the logs could Anthony fix the real issue.

4-point no-errors policy

As a culture we as software engineers need to adopt a no errors/no warnings policy with strict adherence. More specifically:

1) Maintaining errors throughout a project is much simpler than just trying to fix it at the end.

2) How do you know you are not creating issues if you have constant errors and warnings? If the logs are 100% clear, you make a change, and then something occurs, you know you have caused a problem. If, however, the logs are full of problems, you are likely to be unaware of any issues you inject into a system with your work.

3) It might seem like more work but in the long run, but taking care of errors as you go creates a higher quality, more stable end product with less overall effort.

4) Lastly, nobody would expect a bridge builder to leave a site saying: “Well, the bridge works, but there are a few cracks here and there, but they shouldn’t matter.”

4 error-related answers you should know

If you understand the prior four points about errors and alarms, you will be able to answer the following questions quickly:

a) Where are system errors and warnings logged in my current application? [Database (DB) logs, state machine compiler (SMC), programmable logic controller (PLC) code compile warnings, etc.]

b) Do I have any errors?

c) If so, am I very clear on their meaning and have a clear plan to remove them? (Such as, I have three warnings because these duplicate coils exist, but I will remove them at the end of the day when I remove my simulation logic.)

d) If a peer were to sit down and review my work thoroughly for errors and warnings would I be proud of my work? (Here, think of the civil engineer building a bridge.)

The takeaway here is this: error messages matter. A lot. Consistently find them, track them, and remove them: NO errors, No Warnings!

- This blog aggregates expert advice from Callisto Integration, providing manufacturing consulting and systems integration. This blog provides integration advice in plant-floor controls, manufacturing execution systems (MES), and manufacturing consulting, from the factory floor through to the enterprise. Andrew Barker, P.Eng., Callisto Integration, compiled the advice. www.callistointegration.com 



No comments
The Engineers' Choice Awards highlight some of the best new control, instrumentation and automation products as chosen by...
Each year, a panel of Control Engineering editors and industry expert judges select the System Integrator of the Year Award winners.
The Engineering Leaders Under 40 program identifies and gives recognition to young engineers who...
Learn how to increase device reliability in harsh environments and decrease unplanned system downtime.
This eGuide contains a series of articles and videos that considers theoretical and practical; immediate needs and a look into the future.
Learn how to create value with re-use; gain productivity with lean automation and connectivity, and optimize panel design and construction.
Go deep: Automation tackles offshore oil challenges; Ethernet advice; Wireless robotics; Product exclusives; Digital edition exclusives
Lost in the gray scale? How to get effective HMIs; Best practices: Integrate old and new wireless systems; Smart software, networks; Service provider certifications
Fixing PID: Part 2: Tweaking controller strategy; Machine safety networks; Salary survey and career advice; Smart I/O architecture; Product exclusives
The Ask Control Engineering blog covers all aspects of automation, including motors, drives, sensors, motion control, machine control, and embedded systems.
Look at the basics of industrial wireless technologies, wireless concepts, wireless standards, and wireless best practices with Daniel E. Capano of Diversified Technical Services Inc.
Join this ongoing discussion of machine guarding topics, including solutions assessments, regulatory compliance, gap analysis...
This is a blog from the trenches – written by engineers who are implementing and upgrading control systems every day across every industry.
IMS Research, recently acquired by IHS Inc., is a leading independent supplier of market research and consultancy to the global electronics industry.

Find and connect with the most suitable service provider for your unique application. Start searching the Global System Integrator Database Now!

Case Study Database

Case Study Database

Get more exposure for your case study by uploading it to the Control Engineering case study database, where end-users can identify relevant solutions and explore what the experts are doing to effectively implement a variety of technology and productivity related projects.

These case studies provide examples of how knowledgeable solution providers have used technology, processes and people to create effective and successful implementations in real-world situations. Case studies can be completed by filling out a simple online form where you can outline the project title, abstract, and full story in 1500 words or less; upload photos, videos and a logo.

Click here to visit the Case Study Database and upload your case study.