Gauging Satisfaction

When it comes to making a decision to purchase equipment for automation and control systems, nothing is as valuable as the collective experience of your peers in dealing with vendors and their products. That's why Control Engineering created its Customer Satisfaction survey—to capture the knowledge of our readers about the products they buy and the companies with which they regularly do b...

By David Greenfield March 1, 2005
Sidebars:
Product Categories
Full database access
Weighting Process

This article contains online extra material

When it comes to making a decision to purchase equipment for automation and control systems, nothing is as valuable as the collective experience of your peers in dealing with vendors and their products. That’s why Control Engineering created its Customer Satisfaction survey—to capture the knowledge of our readers about the products they buy and the companies with which they regularly do business.

By making these data available, we hope to help you in making future purchasing decisions with vendors you may not have done business with before, and also help you evaluate vendors and products you currently use. After all, it’s helpful to get feedback that supports your decision to continue working with a particular vendor. And it’s equally important to verify any uneasy feelings you may have about continuing a particular business relationship.

Last year’s introduction of the Customer Satisfaction survey showcased reader satisfaction levels with vendors and the products they sell across 16 of the 29 categories of automation, control, and instrumentation equipment we regularly cover. This year, we address the remaining 13 categories (see ‘Product categories’ sidebar).

Though the same survey architecture employed last year was used again this year, we made a change to the weighting system to deliver a more accurate appraisal of reader satisfaction across vendors and products. The ‘Weighting process’ sidebar explains how the new system is applied. In addition, steps were taken to ensure that employees of companies under evaluation were removed from the sample to protect the credibility of results.

Respondents

Taking part were more than 2,300 Control Engineering readers who bought more than 4,300 products across the 13 categories included in this year’s survey. The top five categories from which respondents purchased goods and services, based on number of units purchased , are: switches (1,032), data acquisition hardware and software (682), servo motors (369), power elements for motor control (358), and single and multi-loop controllers (338).

The largest group of respondents is employed in manufacturing industries (75%), predominantly in industrial/commercial/agricultural machinery, instrumentation/ measurement/control systems, food and beverage, automotive, and chemical; the remaining respondents are employed in non-manufacturing industries, such as systems engineering/ integration, utilities, and scientific research.

Overall satisfaction

Based on the survey’s overall satisfaction report grouping—derived from general satisfaction with product/manufacturer, repair, associated software, technical support, and recalibration—the top three product categories based on total weighted score are: switches (78.66); alarm, annunciator, and message panels (76.12); and process analyzers (75.94). The lowest rated categories are servo motors (72.24); robotic system hardware and software (74.36); and paper and/or paperless recorders (74.60).

It can be assumed that the lowest ranked categories were more likely to receive such ratings based on the complexity of the products themselves, thereby lending a greater likelihood that they will encounter more problems. The lower ratings for these categories are quite relative, however. They may be the lowest of the bunch but, at worst, are no more than 3.5 points off the survey’s average overall satisfaction rating of 75.65. Therefore, it should not be assumed that the lower-ranked categories indicate potentially troublesome products.

Speaking to overall product quality is the high level of satisfaction reported when respondents were asked the question: Considering everything, not just the most recent interaction, how satisfied are you with the manufacturer’s product? More than 90% responded that they were either ‘very satisfied’ or ‘somewhat satisfied’ in every product category—underscoring the level of quality products available on the market today.

Product performance and ease of installation are also rated highly across the board, with all categories receiving at least a 90% satisfied rating. Much the same can be said for ease of setup/configuration, with all categories hitting at least 90% satisfaction, aside from servo motors at 88%.

Assessing value for the dollar spent, again all product categories scored well. Switches and paper and/or paperless recorders are the highest rated products in this section of the survey, with satisfaction ratings at 90% and 89%, respectively. The lowest ranked products are linear motors and related controls and servo motors, with satisfaction ratings of 77% and 81%, respectively. Again, it’s necessary to point out a higher level of application complexity associated with the latter products.

When respondents were asked which services (repair, upgrades/bug fixes, technical support, and recalibration) had been required for units purchased, the clear winner is technical support—needed an average of 42% of the time. Respondents required tech support most for data acquisition (57%) and robotic systems (54%). Tech support was least in demand for switches (27%) and dc motors (30%).

Technical support

Since technical support was the category most often cited for services needed, let’s take a closer look at how support services fared across the categories.

Considering how long it took to get an initial response, providers of alarm, annunciator, and message panels rated highest overall (7.13), while dc motors performed lowest at 6.22 (though not a bad rating taking into account the average rating was 6.87). Examining the data more closely shows that providers of linear motors, torque sensors and other motion feedback devices, and single and multi-loop controllers actually respond the fastest, with same day response at 83%, 81%, and 81%, respectively. The slowest responders (next day or more than two days before responding) were dc motors (36%), step motors (33%), and robotic systems (31%).

Linear motors fared best in time to resolve a problem (7.37), while robotic systems ranked lowest at 6.60.

The pet peeve of anyone needing technical support is inability to contact someone who can help. In this section, all categories rated well, but torque sensors and other motion feedback device providers ranked highest at 7.95. Paper and/or paperless recorders and servo motors brought up the rear at 7.27 each.

Internet access to technical support information hasn’t caught on quite as well across all categories yet, as can be seen by the wide array of ratings—ranging from 5.95 to 7.40 for paper and/or paperless recorders (only 65% were satisfied with Web sites for these products) and switches, respectively. Providers of linear motors got the highest satisfaction ratings for the quality of Web site technical support material (91%), but switches beat them out at the overall rating for completeness and accuracy of Web site material with a 7.40 rating compared to linear motors’ 7.35.

One area that product manufacturers and providers are getting predominantly right is staffing technical support ranks with people knowledgeable about the product, its application, the industry, and troubleshooting.

Providers of linear motors ranked highest in the technical support product knowledge category with an 8.59 rating. The lowest ranked category is dc motors at 8.02. The slight disparity from highest to lowest highlights how well companies are doing at providing this type of service. The same can be said across other technical support categories, too, as indicated by the spread in ‘industry knowledge’ ratings at 8.08 and 7.55 for switches and dc motors, respectively. One of the biggest rating spreads is found in ‘troubleshooting,’ at 8.24 for alarm, annunciator, and message panels, and 7.39 for torque sensors and other motion feedback devices—a 0.85 difference in ratings.

Repair and recalibration

Two motor product categories bookend the repair satisfaction report grouping. Linear motors ranked highest with a total weighted score of 76.80, while servo motors scored the lowest rating of 64.21. While linear motors ranked high nearly across the board in terms of satisfaction based on ease of obtaining a replacement device, ease of returning defective device, turn-around time on solution, and overall repair satisfaction, the servo motors category was hurt most by low ratings in method of repair, problem identification, and obtaining a replacement device.

Gauging overall satisfaction, considering percentage of scores tallied for ‘very satisfied’ and ‘somewhat satisfied,’ linear motors made a clean sweep with 100% of respondents in this product category recording entries in one of these two categories. Servo motors did not, likewise, take the bottom score in this area. Step motors and torque sensors and other motion feedback devices shared a 29% response rate for ‘somewhat dissatisfied’ and ‘very dissatisfied’—the highest level of dissatisfaction scored for repair satisfaction.

According to respondents, recalibration was most often performed to comply with site, company, or industry best practices (36%). It was also performed for compliance with quality processes (33%) and government regulations (20%). The least often cited reason for recalibration was to ensure custody transfer integrity (5%).

Two categories in the recalibration section made a clean sweep of the highest and lowest ratings across the topics of turn-around time, quality of service, documentation, overall experience, and total weighted score. Robotic systems ranked highest with an 88.19 total weighted score, while power elements for motor control pulled a 62.94. The average across all categories is 76.62.

Analysis

If you have experienced specific problems with the types of products detailed here, you should, of course, investigate those aspects when considering a new product or doing business with a new vendor. But when looking at these products across the board, what one aspect of the data can best indicate the overall quality of a particular category?

I believe it’s overall product/manufacturer satisfaction. If you examine the spread between highest and lowest ratings received in these categories, you can get an idea of the overall available quality. In other words, categories with the narrowest spread between highest and lowest product/manufacturer satisfaction ratings would indicate that product and service quality is generally high throughout the market. But categories with a wide spread between numbers indicate that buyers should be more guarded about purchases of these products.

In this year’s survey, the widest spread can be found in torque sensors and other motion feedback devices, single and multi-loop controllers, and step motors, posting a spread between highest and lowest ratings of 25.44, 17.81, and 17.23, respectively.

Categories with the narrowest gap between highest and lowest scores were power elements for motor control (7.07), robotic systems (9.2), and switches (9.49).

Experienced end-user satisfaction with a product is typically the best guideline that can be used when deciding upon a new purchase, but it’s not the only advice you should heed. Your own gut feelings and experience should play just as great a part as do the opinions of others.

Overall satisfaction with tech support

Highest Lowest
Linear motors and related controls Switches Process analyzers Paper and/or paperless recorders Step motors and related controls Servo motors
91% 91% 90% 82% 80% 78%
The above percentages represent the three highest and lowest ratings for the ‘very satisfied’ response to a question related to overall satisfaction for the technical support experienced with the product/manufacturer being rated.

Customer Satisfaction

ONLINE ONLY

Pre-assigned weights


The following tables detail the pre-assigned weight given to each closed-ended question on this year’s Customer Satisfaction survey.

Product/Manufacturer satisfaction report grouping
Questions included in this grouping: Response options: Individual response weighting: Question weighting:
Q6– Ease of installation 10 to 110741 1 4 choices
very satisfied
somewhat satisfied
somewhat dissatisfied
very dissatisfied
10 to 1
10
7
4
1
1
Q6– Ease of setup/configuration 4 choices 10 to 1 1
Q6– Quality of documentation 4 choices 10 to 1 1
Q6– Quality of factory calibration 4 choices 10 to 1 1
Q6– Product availability 4 choices 10 to 1 1
Q6– Product performance 4 choices 10 to 1 1
Q6– Price 4 choices 10 to 1 1
Q6– Value for the dollar 4 choices 10 to 1 1
Q6– Product upgrades 4 choices 10 to 1 1
Q6– Legacy product support 4 choices 10 to 1 1
Q6– Web site usefulness 4 choices 10 to 1 1
Q6– Understanding my needs 4 choices 10 to 1 1
Q7– Overall supplier satisfaction 4 choices 10 to 1 3
Q8– Likelihood to purchase again 4 choices
Very likely
Likely
Unlikely
Very unlikely
10 to 1
10
7
4
1
4
Q9– Willingness to “buck” your manager 4 choices
Strongly support Somewhat support Comply with mgr Support dif supplier
10 to 1
10
7
4
1
5
Q11– Services required 5 choices
None of above
1 of 4 other choices
2 of 4 other choices
3 of 4 other choices
4 of 4 other choices
10 to 1
10
7.75
5.5
3.25
1
2
Sum of weights 26
Output scoring
Required values to score: 3 or more
Treat no answers as: Removed from decision weighting
Make sum
(index number):
10 to 100

Repair satifcaction report grouping
Questions included in this grouping: Response options: Individual response weighting: Question weighting:
Q12– When problem first developed 7 choices
24 or more months
6-23 months of use
2-5 months of use
8-28 days of use
2-7 days of use
First 24 hours
When first installed
10 to 1
10
8.5
7
5.5
4
2.5
1
1
Q14– Method of repair 4 choices
Sent replacement Repaired onsite
Sent replacement parts
Returned to factory
10 to 1
10
7
4
1
1
Q15– Obtaining replacement device 4 choices 10 to 1 1
Q15– Returning defective device 4 choices 10 to 1 1
Q15– Turn-around time 4 choices 10 to 1 1
Q15– Overall repair satisfaction 4 choices 10 to 1 3
Sum of weights 9
Output scoring:
Required values to score: 3 or more
Treat no answers as: Removed from decision weighting
Make sum (index number): 10 to 100

Technical support satisfaction report grouping
Questions included in this grouping: Response options: Individual response weighting: Question weighting:
Q19– Initial response time 6 choices
Within 2 hours
2 – 4 hours
Same day
Next day
More than two days
Never contacted
10 to 1
10
8.2
6.4
4.6
2.8
1
3
Q20– Time to resolve problem 7 choices
During initial contact
Same day
Next day
2 – 5 days
6 – 10 days
More than 10 days
Never
10 to 1 10
8.5
7
5.5
4
2.5
1
2
Q21– Ease of reaching correct person 4 choices
very satisfied
somewhat satisfied
somewhat dissatisfied
very dissatisfied
10 to 1
10
7
4
1
2
Q21– Website 4 choices 10 to 1 1
Q21– Turn-around time 4 choices 10 to 1 2
Q21–Suitability of resolution 4 choices 10 to 1 4
Q21– Overall satisfaction 4 choices 10 to 1 10
Q22– Product knowledge 4 choices 10 to 1 3
Q22– App. knowledge 4 choices 10 to 1 2
Q22– Industry knowledge 4 choices 10 to 1 1
Q22– Troubleshooting 4 choices 10 to 1 3
Q22– Try everything 4 choices 10 to 1 3
Sum of weights 36
Output scoring:
Required values to score: 3 or more
Treat no answers as: Removed from decision weighting
Make sum (index number): 10 to 100

Recalibration satisfaction reprot grouping
Questions included in this grouping: Response options: Individual response weighting: Question weighting:
Q25– Ease of getting return authorization 4 choicesvery satisfiedsomewhat satisfiedsomewhat
dissatisfied
very dissatisfied
10 to 1
10
7
4
1
1
Q25– Turn-around time 4 choices 10 to 1 1
Q25– Quality of service 4 choices 10 to 1 3
Q25– Documentation 4 choices 10 to 1 2
Q25– Overall experience 4 choices 10 to 1 5
Sum of weights 12
Output scoring:
Required values to score: 3 or more
Treat no answers as: Removed from decision weighting
Make sum (index number): 10 to 100

Software satisfaction report grouping
Questions included in this grouping: Response options: Individual response weighting: Question weighting:
Q16– When problem first developed 7 choices
24 or more months
6-23 months of use
2-5 months of use
8-28 days of use
2-7 days of use
First 24 hours
When first installed
10 to 1
10
8.5
7
5.5
4
2.5
1
1
Q17–identifying problem 4 choices
verysatisfied
somewhat satisfied
somewhat dissatisfiedvery dissatisfied
10 to 1
10
7
4
1
2
Q17–temporary fix 4 choices 10 to 1 2
Q17–permanent fix 4 choices 10 to 1 2
Q17– Turn-around time 4 choices 10 to 1 2
Q17– Overall satisfaction 4 choices 10 to 1 3
Sum of weights 12
Output scoring:
Required values to score: 3 or more
Treat no answers as: Removed from decision weighting
Make sum (index number): 10 to 100

Overall satisfaction report grouping
Report groups included in this grouping: Rating options: Individual response weighting: Group weighting:
Product/Manufacturer 5 choices
Excellent
Very good
Fair
Poor
Avoid
10 to 1
10
7.75
5.5
3.25
1
8
Repair 5 choices 10 to 1 1
Technical support 5 choices 10 to 1 3
Recalibration 5 choices 10 to 1 1
Software 5 choices 10 to 1 2
Sum of weights 15
Output scoring:
Required values to score: Satisfied
minimum bases in all sub-groupings; i.e. no “NA” scores.
Treat no answers as: Removed from decision weighting
Make sum (index number): 10 to 100

Product Categories

Alarm, annunciator, and message panels

Data acquisition hardware and software

DC motors

Linear motors

Paper and/or paperless recorders

Power elements

Process analyzers

Robotic system hardware and software

Servo motors

Single and multi-loop controllers

Step motors and related controls

Switches

Torque sensors and other motion feedback devices

Full database access

This article highlights only a portion of the total survey results gathered. You can view the complete Customer Satisfaction database and see the data in any way you prefer—by product or by manufacturer—and drill down into the data to access the level of detail needed by going to

Weighting Process

In this year’s Customer Satisfaction survey, Control Engineering and Reed Research Group created a pre-defined weighting formula that assigns weights to each possible answer in all closed-ended survey questions (weighting was not applied to verbatim responses).

Working with the preassigned weights (

1. Weighted score at bottom of question 6, ‘ease of installation’ = [(# very satisfied x 10) + (# somewhat satisfied x 7) + (# somewhat dissatisfied x 4) + (# very dissatisfied x 1)]/n

a) n represents total responding

b) repeat for every row in product/manufacturer sub-grouping and other sub-groupings

2. 10-100 index for product/manufacturer grouping =

a) Sum of

i) Weighted score at bottom of question 6, ‘ease of installation’ x [question weighting]

ii) Repeat for every row in product/manufacturer sub-grouping

iii) Weighted score at bottom of question 11 ‘services required’ x [question weighting]

b) Sum of question weights

c) Divide a by b to get score between 1-10

d) Multiply by 10 to get index between 10-100

e) Repeat for repair and other report sub-groupings

3. 10-100 index for overall grouping =

a) Sum of

i) 10-100 index for product/manufacturer grouping x 8 [group weighting]

ii) 10-100 index for repair grouping x 1 [group weighting]

iii) 10-100 index for tech support grouping x 3 [group weighting]

iv) 10-100 index for recalibration grouping x 1 [group weighting]

v) 10-100 index for software grouping x 2 [group weighting]

b) Sum of sub-grouping weights

c) Divide a by b to get overall index between 10-100

This same formula is applied with the ‘total’ column to generate the weighted score for an entire product category.

Required values to score indicate minimum response bases needed to assign indices and ratings to each product/manufacturer cell. We determined that the required values to score for every sub-grouping is three respondents. When these base requirements are not met, an ‘NA’ is noted in the weighted score cells of the data tables. If one of the rows feeding into a sub-grouping generates an NA score, the score for that entire sub-grouping will be NA, as will the ‘overall’ grouping. Otherwise, artificially low indices in the sub-grouping and overall grouping will be created.

Index evaluation and labeling for each grouping:

& 60 Avoid

60-69 Poor

70 to 79 Fair

80 to 89 Very good

> 90 Excellent