Gauging Satisfaction

When it comes to making a decision to purchase equipment for automation and control systems, nothing is as valuable as the collective experience of your peers in dealing with vendors and their products. That's why Control Engineering created its Customer Satisfaction survey—to capture the knowledge of our readers about the products they buy and the companies with which they regularly do b...

03/01/2005


Sidebars:
Product Categories
Full database access
Weighting Process

This article contains online extra material

When it comes to making a decision to purchase equipment for automation and control systems, nothing is as valuable as the collective experience of your peers in dealing with vendors and their products. That's why Control Engineering created its Customer Satisfaction survey—to capture the knowledge of our readers about the products they buy and the companies with which they regularly do business.

By making these data available, we hope to help you in making future purchasing decisions with vendors you may not have done business with before, and also help you evaluate vendors and products you currently use. After all, it's helpful to get feedback that supports your decision to continue working with a particular vendor. And it's equally important to verify any uneasy feelings you may have about continuing a particular business relationship.

Last year's introduction of the Customer Satisfaction survey showcased reader satisfaction levels with vendors and the products they sell across 16 of the 29 categories of automation, control, and instrumentation equipment we regularly cover. This year, we address the remaining 13 categories (see 'Product categories' sidebar).

Though the same survey architecture employed last year was used again this year, we made a change to the weighting system to deliver a more accurate appraisal of reader satisfaction across vendors and products. The 'Weighting process' sidebar explains how the new system is applied. In addition, steps were taken to ensure that employees of companies under evaluation were removed from the sample to protect the credibility of results.

Respondents

Taking part were more than 2,300 Control Engineering readers who bought more than 4,300 products across the 13 categories included in this year's survey. The top five categories from which respondents purchased goods and services, based on number of units purchased , are: switches (1,032), data acquisition hardware and software (682), servo motors (369), power elements for motor control (358), and single and multi-loop controllers (338).

The largest group of respondents is employed in manufacturing industries (75%), predominantly in industrial/commercial/agricultural machinery, instrumentation/ measurement/control systems, food and beverage, automotive, and chemical; the remaining respondents are employed in non-manufacturing industries, such as systems engineering/ integration, utilities, and scientific research.

Overall satisfaction

Based on the survey's overall satisfaction report grouping—derived from general satisfaction with product/manufacturer, repair, associated software, technical support, and recalibration—the top three product categories based on total weighted score are: switches (78.66); alarm, annunciator, and message panels (76.12); and process analyzers (75.94). The lowest rated categories are servo motors (72.24); robotic system hardware and software (74.36); and paper and/or paperless recorders (74.60).

It can be assumed that the lowest ranked categories were more likely to receive such ratings based on the complexity of the products themselves, thereby lending a greater likelihood that they will encounter more problems. The lower ratings for these categories are quite relative, however. They may be the lowest of the bunch but, at worst, are no more than 3.5 points off the survey's average overall satisfaction rating of 75.65. Therefore, it should not be assumed that the lower-ranked categories indicate potentially troublesome products.

Speaking to overall product quality is the high level of satisfaction reported when respondents were asked the question: Considering everything, not just the most recent interaction, how satisfied are you with the manufacturer's product? More than 90% responded that they were either 'very satisfied' or 'somewhat satisfied' in every product category—underscoring the level of quality products available on the market today.

Product performance and ease of installation are also rated highly across the board, with all categories receiving at least a 90% satisfied rating. Much the same can be said for ease of setup/configuration, with all categories hitting at least 90% satisfaction, aside from servo motors at 88%.

Assessing value for the dollar spent, again all product categories scored well. Switches and paper and/or paperless recorders are the highest rated products in this section of the survey, with satisfaction ratings at 90% and 89%, respectively. The lowest ranked products are linear motors and related controls and servo motors, with satisfaction ratings of 77% and 81%, respectively. Again, it's necessary to point out a higher level of application complexity associated with the latter products.

When respondents were asked which services (repair, upgrades/bug fixes, technical support, and recalibration) had been required for units purchased, the clear winner is technical support—needed an average of 42% of the time. Respondents required tech support most for data acquisition (57%) and robotic systems (54%). Tech support was least in demand for switches (27%) and dc motors (30%).

Technical support

Machine Control & Discrete Sensors, Motors, Devices & Motion Control

Since technical support was the category most often cited for services needed, let's take a closer look at how support services fared across the categories.

Considering how long it took to get an initial response, providers of alarm, annunciator, and message panels rated highest overall (7.13), while dc motors performed lowest at 6.22 (though not a bad rating taking into account the average rating was 6.87). Examining the data more closely shows that providers of linear motors, torque sensors and other motion feedback devices, and single and multi-loop controllers actually respond the fastest, with same day response at 83%, 81%, and 81%, respectively. The slowest responders (next day or more than two days before responding) were dc motors (36%), step motors (33%), and robotic systems (31%).

Linear motors fared best in time to resolve a problem (7.37), while robotic systems ranked lowest at 6.60.

Instrumentation & Sensors, Process Control & Advanced Control

The pet peeve of anyone needing technical support is inability to contact someone who can help. In this section, all categories rated well, but torque sensors and other motion feedback device providers ranked highest at 7.95. Paper and/or paperless recorders and servo motors brought up the rear at 7.27 each.

Internet access to technical support information hasn't caught on quite as well across all categories yet, as can be seen by the wide array of ratings—ranging from 5.95 to 7.40 for paper and/or paperless recorders (only 65% were satisfied with Web sites for these products) and switches, respectively. Providers of linear motors got the highest satisfaction ratings for the quality of Web site technical support material (91%), but switches beat them out at the overall rating for completeness and accuracy of Web site material with a 7.40 rating compared to linear motors' 7.35.

Embedded Control, HMI, Software

One area that product manufacturers and providers are getting predominantly right is staffing technical support ranks with people knowledgeable about the product, its application, the industry, and troubleshooting.

Providers of linear motors ranked highest in the technical support product knowledge category with an 8.59 rating. The lowest ranked category is dc motors at 8.02. The slight disparity from highest to lowest highlights how well companies are doing at providing this type of service. The same can be said across other technical support categories, too, as indicated by the spread in 'industry knowledge' ratings at 8.08 and 7.55 for switches and dc motors, respectively. One of the biggest rating spreads is found in 'troubleshooting,' at 8.24 for alarm, annunciator, and message panels, and 7.39 for torque sensors and other motion feedback devices—a 0.85 difference in ratings.

Repair and recalibration

Two motor product categories bookend the repair satisfaction report grouping. Linear motors ranked highest with a total weighted score of 76.80, while servo motors scored the lowest rating of 64.21. While linear motors ranked high nearly across the board in terms of satisfaction based on ease of obtaining a replacement device, ease of returning defective device, turn-around time on solution, and overall repair satisfaction, the servo motors category was hurt most by low ratings in method of repair, problem identification, and obtaining a replacement device.

Gauging overall satisfaction, considering percentage of scores tallied for 'very satisfied' and 'somewhat satisfied,' linear motors made a clean sweep with 100% of respondents in this product category recording entries in one of these two categories. Servo motors did not, likewise, take the bottom score in this area. Step motors and torque sensors and other motion feedback devices shared a 29% response rate for 'somewhat dissatisfied' and 'very dissatisfied'—the highest level of dissatisfaction scored for repair satisfaction.

According to respondents, recalibration was most often performed to comply with site, company, or industry best practices (36%). It was also performed for compliance with quality processes (33%) and government regulations (20%). The least often cited reason for recalibration was to ensure custody transfer integrity (5%).

Two categories in the recalibration section made a clean sweep of the highest and lowest ratings across the topics of turn-around time, quality of service, documentation, overall experience, and total weighted score. Robotic systems ranked highest with an 88.19 total weighted score, while power elements for motor control pulled a 62.94. The average across all categories is 76.62.

Analysis

Control Components

If you have experienced specific problems with the types of products detailed here, you should, of course, investigate those aspects when considering a new product or doing business with a new vendor. But when looking at these products across the board, what one aspect of the data can best indicate the overall quality of a particular category?

I believe it's overall product/manufacturer satisfaction. If you examine the spread between highest and lowest ratings received in these categories, you can get an idea of the overall available quality. In other words, categories with the narrowest spread between highest and lowest product/manufacturer satisfaction ratings would indicate that product and service quality is generally high throughout the market. But categories with a wide spread between numbers indicate that buyers should be more guarded about purchases of these products.

Networks & Communication, System Integration

In this year's survey, the widest spread can be found in torque sensors and other motion feedback devices, single and multi-loop controllers, and step motors, posting a spread between highest and lowest ratings of 25.44, 17.81, and 17.23, respectively.

Categories with the narrowest gap between highest and lowest scores were power elements for motor control (7.07), robotic systems (9.2), and switches (9.49).

Experienced end-user satisfaction with a product is typically the best guideline that can be used when deciding upon a new purchase, but it's not the only advice you should heed. Your own gut feelings and experience should play just as great a part as do the opinions of others.

Overall satisfaction with tech support

Highest Lowest
Linear motors and related controlsSwitchesProcess analyzersPaper and/or paperless recordersStep motors and related controlsServo motors
91%91%90%82%80%78%
The above percentages represent the three highest and lowest ratings for the 'very satisfied' response to a question related to overall satisfaction for the technical support experienced with the product/manufacturer being rated.


Customer Satisfaction

ONLINE ONLY

Pre-assigned weights


The following tables detail the pre-assigned weight given to each closed-ended question on this year’s Customer Satisfaction survey.

Product/Manufacturer satisfaction report grouping
Questions included in this grouping: Response options: Individual response weighting: Question weighting:
Q6– Ease of installation 10 to 110741 1
4 choices
very satisfied
somewhat satisfied
somewhat dissatisfied
very dissatisfied
10 to 1
10
7
4
1
1
Q6– Ease of setup/configuration4 choices10 to 11
Q6– Quality of documentation4 choices10 to 11
Q6– Quality of factory calibration4 choices10 to 11
Q6– Product availability4 choices10 to 11
Q6– Product performance4 choices10 to 11
Q6– Price4 choices10 to 11
Q6– Value for the dollar4 choices10 to 11
Q6– Product upgrades4 choices10 to 11
Q6– Legacy product support4 choices10 to 11
Q6– Web site usefulness4 choices10 to 11
Q6– Understanding my needs4 choices10 to 11
Q7– Overall supplier satisfaction4 choices10 to 13
Q8– Likelihood to purchase again4 choices
Very likely
Likely
Unlikely
Very unlikely
10 to 1
10
7
4
1
4
Q9– Willingness to “buck” your manager4 choices
Strongly support Somewhat support Comply with mgr Support dif supplier
10 to 1
10
7
4
1
5
Q11– Services required5 choices
None of above
1 of 4 other choices
2 of 4 other choices
3 of 4 other choices
4 of 4 other choices
10 to 1
10
7.75
5.5
3.25
1
2
Sum of weights26
Output scoring
Required values to score:3 or more
Treat no answers as:Removed from decision weighting
Make sum
(index number):
10 to 100

 

Repair satifcaction report grouping
Questions included in this grouping: Response options: Individual response weighting: Question weighting:
Q12– When problem first developed
7 choices
24 or more months
6-23 months of use
2-5 months of use
8-28 days of use
2-7 days of use
First 24 hours
When first installed
10 to 1
10
8.5
7
5.5
4
2.5
1
1
Q14– Method of repair
4 choices
Sent replacement Repaired onsite
Sent replacement parts
Returned to factory
10 to 1
10
7
4
1
1
Q15– Obtaining replacement device4 choices10 to 11
Q15– Returning defective device4 choices10 to 11
Q15– Turn-around time4 choices10 to 11
Q15– Overall repair satisfaction4 choices10 to 13
Sum of weights9
Output scoring:
Required values to score:3 or more
Treat no answers as:Removed from decision weighting
Make sum (index number):10 to 100

 

Technical support satisfaction report grouping
Questions included in this grouping: Response options: Individual response weighting: Question weighting:
Q19– Initial response time6 choices
Within 2 hours
2 - 4 hours
Same day
Next day
More than two days
Never contacted
10 to 1
10
8.2
6.4
4.6
2.8
1
3
Q20– Time to resolve problem7 choices
During initial contact
Same day
Next day
2 - 5 days
6 - 10 days
More than 10 days
Never
10 to 1 10
8.5
7
5.5
4
2.5
1
2
Q21– Ease of reaching correct person
4 choices
very satisfied
somewhat satisfied
somewhat dissatisfied
very dissatisfied

10 to 1
10
7
4
1
2
Q21– Website4 choices10 to 11
Q21– Turn-around time4 choices10 to 12
Q21–Suitability of resolution4 choices10 to 14
Q21– Overall satisfaction4 choices10 to 110
Q22– Product knowledge4 choices10 to 13
Q22– App. knowledge4 choices10 to 12
Q22– Industry knowledge4 choices10 to 11
Q22– Troubleshooting4 choices10 to 13
Q22– Try everything4 choices10 to 13
Sum of weights36
Output scoring:
Required values to score:3 or more
Treat no answers as:Removed from decision weighting
Make sum (index number):10 to 100

 

Recalibration satisfaction reprot grouping
Questions included in this grouping: Response options: Individual response weighting: Question weighting:
Q25– Ease of getting return authorization4 choicesvery satisfiedsomewhat satisfiedsomewhat
dissatisfied
very dissatisfied
10 to 1
10
7
4
1
1
Q25– Turn-around time4 choices10 to 11
Q25– Quality of service4 choices10 to 13
Q25– Documentation4 choices10 to 12
Q25– Overall experience4 choices10 to 15
Sum of weights12
Output scoring:
Required values to score:3 or more
Treat no answers as:Removed from decision weighting
Make sum (index number):10 to 100

 

Software satisfaction report grouping
Questions included in this grouping: Response options: Individual response weighting:
Question weighting:
Q16– When problem first developed7 choices
24 or more months
6-23 months of use
2-5 months of use
8-28 days of use
2-7 days of use
First 24 hours
When first installed
10 to 1
10
8.5
7
5.5
4
2.5
1
1
Q17–identifying problem4 choices
verysatisfied
somewhat satisfied
somewhat dissatisfiedvery dissatisfied
10 to 1
10
7
4
1
2
Q17–temporary fix4 choices10 to 12
Q17–permanent fix4 choices10 to 12
Q17– Turn-around time4 choices10 to 12
Q17– Overall satisfaction4 choices10 to 13
Sum of weights 12
Output scoring:
Required values to score:3 or more
Treat no answers as:Removed from decision weighting
Make sum (index number):10 to 100

 

Overall satisfaction report grouping
Report groups included in this grouping: Rating options: Individual response weighting: Group weighting:
Product/Manufacturer
5 choices
Excellent
Very good
Fair
Poor
Avoid
10 to 1
10
7.75
5.5
3.25
1
8
Repair5 choices10 to 11
Technical support5 choices10 to 13
Recalibration5 choices10 to 11
Software5 choices10 to 12
Sum of weights15
Output scoring:
Required values to score:Satisfied
minimum bases in all sub-groupings; i.e. no “NA” scores.
Treat no answers as:Removed from decision weighting
Make sum (index number):10 to 100



Product Categories

Alarm, annunciator, and message panels

Data acquisition hardware and software

DC motors

Linear motors

Paper and/or paperless recorders

Power elements

Process analyzers

Robotic system hardware and software

Servo motors

Single and multi-loop controllers

Step motors and related controls

Switches

Torque sensors and other motion feedback devices

Full database access

This article highlights only a portion of the total survey results gathered. You can view the complete Customer Satisfaction database and see the data in any way you prefer—by product or by manufacturer—and drill down into the data to access the level of detail needed by going to

Weighting Process

In this year's Customer Satisfaction survey, Control Engineering and Reed Research Group created a pre-defined weighting formula that assigns weights to each possible answer in all closed-ended survey questions (weighting was not applied to verbatim responses).

Working with the preassigned weights (

1. Weighted score at bottom of question 6, 'ease of installation' = [(# very satisfied x 10) + (# somewhat satisfied x 7) + (# somewhat dissatisfied x 4) + (# very dissatisfied x 1)]/n

a) n represents total responding

b) repeat for every row in product/manufacturer sub-grouping and other sub-groupings

2. 10-100 index for product/manufacturer grouping =

a) Sum of

i) Weighted score at bottom of question 6, 'ease of installation' x [question weighting]

ii) Repeat for every row in product/manufacturer sub-grouping

iii) Weighted score at bottom of question 11 'services required' x [question weighting]

b) Sum of question weights

c) Divide a by b to get score between 1-10

d) Multiply by 10 to get index between 10-100

e) Repeat for repair and other report sub-groupings

3. 10-100 index for overall grouping =

a) Sum of

i) 10-100 index for product/manufacturer grouping x 8 [group weighting]

ii) 10-100 index for repair grouping x 1 [group weighting]

iii) 10-100 index for tech support grouping x 3 [group weighting]

iv) 10-100 index for recalibration grouping x 1 [group weighting]

v) 10-100 index for software grouping x 2 [group weighting]

b) Sum of sub-grouping weights

c) Divide a by b to get overall index between 10-100

This same formula is applied with the 'total' column to generate the weighted score for an entire product category.

Required values to score indicate minimum response bases needed to assign indices and ratings to each product/manufacturer cell. We determined that the required values to score for every sub-grouping is three respondents. When these base requirements are not met, an 'NA' is noted in the weighted score cells of the data tables. If one of the rows feeding into a sub-grouping generates an NA score, the score for that entire sub-grouping will be NA, as will the 'overall' grouping. Otherwise, artificially low indices in the sub-grouping and overall grouping will be created.

Index evaluation and labeling for each grouping:

& 60 Avoid

60-69 Poor

70 to 79 Fair

80 to 89 Very good

> 90 Excellent



The Engineers' Choice Awards highlight some of the best new control, instrumentation and automation products as chosen by Control Engineering subscribers. Vote now (if qualified)!
The System Integrator Giants program lists the top 100 system integrators among companies listed in CFE Media's Global System Integrator Database.
Each year, a panel of Control Engineering and Plant Engineering editors and industry expert judges select the System Integrator of the Year Award winners in three categories.
This eGuide illustrates solutions, applications and benefits of machine vision systems.
Learn how to increase device reliability in harsh environments and decrease unplanned system downtime.
This eGuide contains a series of articles and videos that considers theoretical and practical; immediate needs and a look into the future.
Maximize ROI with integrated control system approach; Microcontrollers vs. PLCs; Power quality; Accelerate and rewire IIoT; Traits for excellent engineers
HMI effectiveness; Distributed I/O; Engineers' Choice Award finalists; System Integrator advice; Inside Machines
Women in engineering; Engineering Leaders Under 40; PID benefits and drawbacks; Ladder logic; Cloud computing
Programmable logic controllers (PLCs) represent the logic (decision) part of the control loop of sense, decide, and actuate. As we know, PLCs aren’t the only option for making decisions in a control loop, but they are likely why you’re here.
This digital report explains how plant engineers and subject matter experts (SME) need support for time series data and its many challenges.
This article collection contains several articles on how advancements in vision system designs, computing power, algorithms, optics, and communications are making machine vision more cost effective than ever before.

Find and connect with the most suitable service provider for your unique application. Start searching the Global System Integrator Database Now!

Control room technology innovation; Practical approaches to corrosion protection; Pipeline regulator revises quality programs
Cloud, mobility, and remote operations; SCADA and contextual mobility; Custom UPS empowering a secure pipeline
Infrastructure for natural gas expansion; Artificial lift methods; Disruptive technology and fugitive gas emissions
Automation Engineer; Wood Group
System Integrator; Cross Integrated Systems Group
Jose S. Vasquez, Jr.
Fire & Life Safety Engineer; Technip USA Inc.
This course focuses on climate analysis, appropriateness of cooling system selection, and combining cooling systems.
This course will help identify and reveal electrical hazards and identify the solutions to implementing and maintaining a safe work environment.
This course explains how maintaining power and communication systems through emergency power-generation systems is critical.
The Engineers' Choice Awards highlight some of the best new control, instrumentation and automation products as chosen by Control Engineering subscribers. Vote now (if qualified)!
The System Integrator Giants program lists the top 100 system integrators among companies listed in CFE Media's Global System Integrator Database.
Each year, a panel of Control Engineering and Plant Engineering editors and industry expert judges select the System Integrator of the Year Award winners in three categories.
This eGuide illustrates solutions, applications and benefits of machine vision systems.
Learn how to increase device reliability in harsh environments and decrease unplanned system downtime.
This eGuide contains a series of articles and videos that considers theoretical and practical; immediate needs and a look into the future.
Maximize ROI with integrated control system approach; Microcontrollers vs. PLCs; Power quality; Accelerate and rewire IIoT; Traits for excellent engineers
HMI effectiveness; Distributed I/O; Engineers' Choice Award finalists; System Integrator advice; Inside Machines
Women in engineering; Engineering Leaders Under 40; PID benefits and drawbacks; Ladder logic; Cloud computing
Programmable logic controllers (PLCs) represent the logic (decision) part of the control loop of sense, decide, and actuate. As we know, PLCs aren’t the only option for making decisions in a control loop, but they are likely why you’re here.
This digital report explains how plant engineers and subject matter experts (SME) need support for time series data and its many challenges.
This article collection contains several articles on how advancements in vision system designs, computing power, algorithms, optics, and communications are making machine vision more cost effective than ever before.

Find and connect with the most suitable service provider for your unique application. Start searching the Global System Integrator Database Now!

Control room technology innovation; Practical approaches to corrosion protection; Pipeline regulator revises quality programs
Cloud, mobility, and remote operations; SCADA and contextual mobility; Custom UPS empowering a secure pipeline
Infrastructure for natural gas expansion; Artificial lift methods; Disruptive technology and fugitive gas emissions
Automation Engineer; Wood Group
System Integrator; Cross Integrated Systems Group
Jose S. Vasquez, Jr.
Fire & Life Safety Engineer; Technip USA Inc.
This course focuses on climate analysis, appropriateness of cooling system selection, and combining cooling systems.
This course will help identify and reveal electrical hazards and identify the solutions to implementing and maintaining a safe work environment.
This course explains how maintaining power and communication systems through emergency power-generation systems is critical.
The Engineers' Choice Awards highlight some of the best new control, instrumentation and automation products as chosen by Control Engineering subscribers. Vote now (if qualified)!
The System Integrator Giants program lists the top 100 system integrators among companies listed in CFE Media's Global System Integrator Database.
Each year, a panel of Control Engineering and Plant Engineering editors and industry expert judges select the System Integrator of the Year Award winners in three categories.
This eGuide illustrates solutions, applications and benefits of machine vision systems.
Learn how to increase device reliability in harsh environments and decrease unplanned system downtime.
This eGuide contains a series of articles and videos that considers theoretical and practical; immediate needs and a look into the future.
Maximize ROI with integrated control system approach; Microcontrollers vs. PLCs; Power quality; Accelerate and rewire IIoT; Traits for excellent engineers
HMI effectiveness; Distributed I/O; Engineers' Choice Award finalists; System Integrator advice; Inside Machines
Women in engineering; Engineering Leaders Under 40; PID benefits and drawbacks; Ladder logic; Cloud computing
Programmable logic controllers (PLCs) represent the logic (decision) part of the control loop of sense, decide, and actuate. As we know, PLCs aren’t the only option for making decisions in a control loop, but they are likely why you’re here.
This digital report explains how plant engineers and subject matter experts (SME) need support for time series data and its many challenges.
This article collection contains several articles on how advancements in vision system designs, computing power, algorithms, optics, and communications are making machine vision more cost effective than ever before.

Find and connect with the most suitable service provider for your unique application. Start searching the Global System Integrator Database Now!

Control room technology innovation; Practical approaches to corrosion protection; Pipeline regulator revises quality programs
Cloud, mobility, and remote operations; SCADA and contextual mobility; Custom UPS empowering a secure pipeline
Infrastructure for natural gas expansion; Artificial lift methods; Disruptive technology and fugitive gas emissions
Automation Engineer; Wood Group
System Integrator; Cross Integrated Systems Group
Jose S. Vasquez, Jr.
Fire & Life Safety Engineer; Technip USA Inc.
This course focuses on climate analysis, appropriateness of cooling system selection, and combining cooling systems.
This course will help identify and reveal electrical hazards and identify the solutions to implementing and maintaining a safe work environment.
This course explains how maintaining power and communication systems through emergency power-generation systems is critical.
click me