Gauging Satisfaction

When it comes to making a decision to purchase equipment for automation and control systems, nothing is as valuable as the collective experience of your peers in dealing with vendors and their products. That's why Control Engineering created its Customer Satisfaction survey—to capture the knowledge of our readers about the products they buy and the companies with which they regularly do b...

03/01/2005


Sidebars:
Product Categories
Full database access
Weighting Process

This article contains online extra material

When it comes to making a decision to purchase equipment for automation and control systems, nothing is as valuable as the collective experience of your peers in dealing with vendors and their products. That's why Control Engineering created its Customer Satisfaction survey—to capture the knowledge of our readers about the products they buy and the companies with which they regularly do business.

By making these data available, we hope to help you in making future purchasing decisions with vendors you may not have done business with before, and also help you evaluate vendors and products you currently use. After all, it's helpful to get feedback that supports your decision to continue working with a particular vendor. And it's equally important to verify any uneasy feelings you may have about continuing a particular business relationship.

Last year's introduction of the Customer Satisfaction survey showcased reader satisfaction levels with vendors and the products they sell across 16 of the 29 categories of automation, control, and instrumentation equipment we regularly cover. This year, we address the remaining 13 categories (see 'Product categories' sidebar).

Though the same survey architecture employed last year was used again this year, we made a change to the weighting system to deliver a more accurate appraisal of reader satisfaction across vendors and products. The 'Weighting process' sidebar explains how the new system is applied. In addition, steps were taken to ensure that employees of companies under evaluation were removed from the sample to protect the credibility of results.

Respondents

Taking part were more than 2,300 Control Engineering readers who bought more than 4,300 products across the 13 categories included in this year's survey. The top five categories from which respondents purchased goods and services, based on number of units purchased , are: switches (1,032), data acquisition hardware and software (682), servo motors (369), power elements for motor control (358), and single and multi-loop controllers (338).

The largest group of respondents is employed in manufacturing industries (75%), predominantly in industrial/commercial/agricultural machinery, instrumentation/ measurement/control systems, food and beverage, automotive, and chemical; the remaining respondents are employed in non-manufacturing industries, such as systems engineering/ integration, utilities, and scientific research.

Overall satisfaction

Based on the survey's overall satisfaction report grouping—derived from general satisfaction with product/manufacturer, repair, associated software, technical support, and recalibration—the top three product categories based on total weighted score are: switches (78.66); alarm, annunciator, and message panels (76.12); and process analyzers (75.94). The lowest rated categories are servo motors (72.24); robotic system hardware and software (74.36); and paper and/or paperless recorders (74.60).

It can be assumed that the lowest ranked categories were more likely to receive such ratings based on the complexity of the products themselves, thereby lending a greater likelihood that they will encounter more problems. The lower ratings for these categories are quite relative, however. They may be the lowest of the bunch but, at worst, are no more than 3.5 points off the survey's average overall satisfaction rating of 75.65. Therefore, it should not be assumed that the lower-ranked categories indicate potentially troublesome products.

Speaking to overall product quality is the high level of satisfaction reported when respondents were asked the question: Considering everything, not just the most recent interaction, how satisfied are you with the manufacturer's product? More than 90% responded that they were either 'very satisfied' or 'somewhat satisfied' in every product category—underscoring the level of quality products available on the market today.

Product performance and ease of installation are also rated highly across the board, with all categories receiving at least a 90% satisfied rating. Much the same can be said for ease of setup/configuration, with all categories hitting at least 90% satisfaction, aside from servo motors at 88%.

Assessing value for the dollar spent, again all product categories scored well. Switches and paper and/or paperless recorders are the highest rated products in this section of the survey, with satisfaction ratings at 90% and 89%, respectively. The lowest ranked products are linear motors and related controls and servo motors, with satisfaction ratings of 77% and 81%, respectively. Again, it's necessary to point out a higher level of application complexity associated with the latter products.

When respondents were asked which services (repair, upgrades/bug fixes, technical support, and recalibration) had been required for units purchased, the clear winner is technical support—needed an average of 42% of the time. Respondents required tech support most for data acquisition (57%) and robotic systems (54%). Tech support was least in demand for switches (27%) and dc motors (30%).

Technical support

Machine Control & Discrete Sensors, Motors, Devices & Motion Control

Since technical support was the category most often cited for services needed, let's take a closer look at how support services fared across the categories.

Considering how long it took to get an initial response, providers of alarm, annunciator, and message panels rated highest overall (7.13), while dc motors performed lowest at 6.22 (though not a bad rating taking into account the average rating was 6.87). Examining the data more closely shows that providers of linear motors, torque sensors and other motion feedback devices, and single and multi-loop controllers actually respond the fastest, with same day response at 83%, 81%, and 81%, respectively. The slowest responders (next day or more than two days before responding) were dc motors (36%), step motors (33%), and robotic systems (31%).

Linear motors fared best in time to resolve a problem (7.37), while robotic systems ranked lowest at 6.60.

Instrumentation & Sensors, Process Control & Advanced Control

The pet peeve of anyone needing technical support is inability to contact someone who can help. In this section, all categories rated well, but torque sensors and other motion feedback device providers ranked highest at 7.95. Paper and/or paperless recorders and servo motors brought up the rear at 7.27 each.

Internet access to technical support information hasn't caught on quite as well across all categories yet, as can be seen by the wide array of ratings—ranging from 5.95 to 7.40 for paper and/or paperless recorders (only 65% were satisfied with Web sites for these products) and switches, respectively. Providers of linear motors got the highest satisfaction ratings for the quality of Web site technical support material (91%), but switches beat them out at the overall rating for completeness and accuracy of Web site material with a 7.40 rating compared to linear motors' 7.35.

Embedded Control, HMI, Software

One area that product manufacturers and providers are getting predominantly right is staffing technical support ranks with people knowledgeable about the product, its application, the industry, and troubleshooting.

Providers of linear motors ranked highest in the technical support product knowledge category with an 8.59 rating. The lowest ranked category is dc motors at 8.02. The slight disparity from highest to lowest highlights how well companies are doing at providing this type of service. The same can be said across other technical support categories, too, as indicated by the spread in 'industry knowledge' ratings at 8.08 and 7.55 for switches and dc motors, respectively. One of the biggest rating spreads is found in 'troubleshooting,' at 8.24 for alarm, annunciator, and message panels, and 7.39 for torque sensors and other motion feedback devices—a 0.85 difference in ratings.

Repair and recalibration

Two motor product categories bookend the repair satisfaction report grouping. Linear motors ranked highest with a total weighted score of 76.80, while servo motors scored the lowest rating of 64.21. While linear motors ranked high nearly across the board in terms of satisfaction based on ease of obtaining a replacement device, ease of returning defective device, turn-around time on solution, and overall repair satisfaction, the servo motors category was hurt most by low ratings in method of repair, problem identification, and obtaining a replacement device.

Gauging overall satisfaction, considering percentage of scores tallied for 'very satisfied' and 'somewhat satisfied,' linear motors made a clean sweep with 100% of respondents in this product category recording entries in one of these two categories. Servo motors did not, likewise, take the bottom score in this area. Step motors and torque sensors and other motion feedback devices shared a 29% response rate for 'somewhat dissatisfied' and 'very dissatisfied'—the highest level of dissatisfaction scored for repair satisfaction.

According to respondents, recalibration was most often performed to comply with site, company, or industry best practices (36%). It was also performed for compliance with quality processes (33%) and government regulations (20%). The least often cited reason for recalibration was to ensure custody transfer integrity (5%).

Two categories in the recalibration section made a clean sweep of the highest and lowest ratings across the topics of turn-around time, quality of service, documentation, overall experience, and total weighted score. Robotic systems ranked highest with an 88.19 total weighted score, while power elements for motor control pulled a 62.94. The average across all categories is 76.62.

Analysis

Control Components

If you have experienced specific problems with the types of products detailed here, you should, of course, investigate those aspects when considering a new product or doing business with a new vendor. But when looking at these products across the board, what one aspect of the data can best indicate the overall quality of a particular category?

I believe it's overall product/manufacturer satisfaction. If you examine the spread between highest and lowest ratings received in these categories, you can get an idea of the overall available quality. In other words, categories with the narrowest spread between highest and lowest product/manufacturer satisfaction ratings would indicate that product and service quality is generally high throughout the market. But categories with a wide spread between numbers indicate that buyers should be more guarded about purchases of these products.

Networks & Communication, System Integration

In this year's survey, the widest spread can be found in torque sensors and other motion feedback devices, single and multi-loop controllers, and step motors, posting a spread between highest and lowest ratings of 25.44, 17.81, and 17.23, respectively.

Categories with the narrowest gap between highest and lowest scores were power elements for motor control (7.07), robotic systems (9.2), and switches (9.49).

Experienced end-user satisfaction with a product is typically the best guideline that can be used when deciding upon a new purchase, but it's not the only advice you should heed. Your own gut feelings and experience should play just as great a part as do the opinions of others.

Overall satisfaction with tech support

Highest

Lowest

Linear motors and related controls

Switches

Process analyzers

Paper and/or paperless recorders

Step motors and related controls

Servo motors

91%

91%

90%

82%

80%

78%

The above percentages represent the three highest and lowest ratings for the 'very satisfied' response to a question related to overall satisfaction for the technical support experienced with the product/manufacturer being rated.


Customer Satisfaction

ONLINE ONLY

Pre-assigned weights


The following tables detail the pre-assigned weight given to each closed-ended question on this year’s Customer Satisfaction survey.

Product/Manufacturer satisfaction report grouping

Questions included in this grouping:

Response options:

Individual response weighting:

Question weighting:

Q6– Ease of installation 10 to 110741 1


4 choices
very satisfied
somewhat satisfied
somewhat dissatisfied
very dissatisfied

10 to 1
10
7
4
1

1

Q6– Ease of setup/configuration

4 choices

10 to 1

1

Q6– Quality of documentation

4 choices

10 to 1

1

Q6– Quality of factory calibration

4 choices

10 to 1

1

Q6– Product availability

4 choices

10 to 1

1

Q6– Product performance

4 choices

10 to 1

1

Q6– Price

4 choices

10 to 1

1

Q6– Value for the dollar

4 choices

10 to 1

1

Q6– Product upgrades

4 choices

10 to 1

1

Q6– Legacy product support

4 choices

10 to 1

1

Q6– Web site usefulness

4 choices

10 to 1

1

Q6– Understanding my needs

4 choices

10 to 1

1

Q7– Overall supplier satisfaction

4 choices

10 to 1

3

Q8– Likelihood to purchase again

4 choices
Very likely
Likely
Unlikely
Very unlikely

10 to 1
10
7
4
1

4

Q9– Willingness to “buck” your manager

4 choices
Strongly support Somewhat support Comply with mgr Support dif supplier

10 to 1
10
7
4
1

5

Q11– Services required

5 choices
None of above
1 of 4 other choices
2 of 4 other choices
3 of 4 other choices
4 of 4 other choices

10 to 1
10
7.75
5.5
3.25
1

2

Sum of weights

26

Output scoring

Required values to score:

3 or more

Treat no answers as:

Removed from decision weighting

Make sum
(index number):

10 to 100

 

Repair satifcaction report grouping

Questions included in this grouping:

Response options:

Individual response weighting:

Question weighting:

Q12– When problem first developed


7 choices
24 or more months
6-23 months of use
2-5 months of use
8-28 days of use
2-7 days of use
First 24 hours
When first installed

10 to 1
10
8.5
7
5.5
4
2.5
1

1

Q14– Method of repair


4 choices
Sent replacement Repaired onsite
Sent replacement parts
Returned to factory

10 to 1
10
7
4
1

1

Q15– Obtaining replacement device

4 choices

10 to 1

1

Q15– Returning defective device

4 choices

10 to 1

1

Q15– Turn-around time

4 choices

10 to 1

1

Q15– Overall repair satisfaction

4 choices

10 to 1

3

Sum of weights

9

Output scoring:

Required values to score:

3 or more

Treat no answers as:

Removed from decision weighting

Make sum (index number):

10 to 100

 

Technical support satisfaction report grouping

Questions included in this grouping:

Response options:

Individual response weighting:

Question weighting:

Q19– Initial response time

6 choices
Within 2 hours
2 - 4 hours
Same day
Next day
More than two days
Never contacted

10 to 1
10
8.2
6.4
4.6
2.8
1

3

Q20– Time to resolve problem

7 choices
During initial contact
Same day
Next day
2 - 5 days
6 - 10 days
More than 10 days
Never

10 to 1 10
8.5
7
5.5
4
2.5
1

2

Q21– Ease of reaching correct person


4 choices
very satisfied
somewhat satisfied
somewhat dissatisfied
very dissatisfied


10 to 1
10
7
4
1

2

Q21– Website

4 choices

10 to 1

1

Q21– Turn-around time

4 choices

10 to 1

2

Q21–Suitability of resolution

4 choices

10 to 1

4

Q21– Overall satisfaction

4 choices

10 to 1

10

Q22– Product knowledge

4 choices

10 to 1

3

Q22– App. knowledge

4 choices

10 to 1

2

Q22– Industry knowledge

4 choices

10 to 1

1

Q22– Troubleshooting

4 choices

10 to 1

3

Q22– Try everything

4 choices

10 to 1

3

Sum of weights

36

Output scoring:

Required values to score:

3 or more

Treat no answers as:

Removed from decision weighting

Make sum (index number):

10 to 100

 

Recalibration satisfaction reprot grouping

Questions included in this grouping:

Response options:

Individual response weighting:

Question weighting:

Q25– Ease of getting return authorization

4 choicesvery satisfiedsomewhat satisfiedsomewhat
dissatisfied
very dissatisfied

10 to 1
10
7
4
1

1

Q25– Turn-around time

4 choices

10 to 1

1

Q25– Quality of service

4 choices

10 to 1

3

Q25– Documentation

4 choices

10 to 1

2

Q25– Overall experience

4 choices

10 to 1

5

Sum of weights

12

Output scoring:

Required values to score:

3 or more

Treat no answers as:

Removed from decision weighting

Make sum (index number):

10 to 100

 

Software satisfaction report grouping

Questions included in this grouping:

Response options:

Individual response weighting:


Question weighting:

Q16– When problem first developed

7 choices
24 or more months
6-23 months of use
2-5 months of use
8-28 days of use
2-7 days of use
First 24 hours
When first installed

10 to 1
10
8.5
7
5.5
4
2.5
1

1

Q17–identifying problem

4 choices
verysatisfied
somewhat satisfied
somewhat dissatisfiedvery dissatisfied

10 to 1
10
7
4
1

2

Q17–temporary fix

4 choices

10 to 1

2

Q17–permanent fix

4 choices

10 to 1

2

Q17– Turn-around time

4 choices

10 to 1

2

Q17– Overall satisfaction

4 choices

10 to 1

3

Sum of weights 12

Output scoring:

Required values to score:

3 or more

Treat no answers as:

Removed from decision weighting

Make sum (index number):

10 to 100

 

Overall satisfaction report grouping

Report groups included in this grouping:

Rating options:

Individual response weighting:

Group weighting:

Product/Manufacturer


5 choices
Excellent
Very good
Fair
Poor
Avoid

10 to 1
10
7.75
5.5
3.25
1

8

Repair

5 choices

10 to 1

1

Technical support

5 choices

10 to 1

3

Recalibration

5 choices

10 to 1

1

Software

5 choices

10 to 1

2

Sum of weights

15

Output scoring:

Required values to score:

Satisfied
minimum bases in all sub-groupings; i.e. no “NA” scores.

Treat no answers as:

Removed from decision weighting

Make sum (index number):

10 to 100



Product Categories

Alarm, annunciator, and message panels

Data acquisition hardware and software

DC motors

Linear motors

Paper and/or paperless recorders

Power elements

Process analyzers

Robotic system hardware and software

Servo motors

Single and multi-loop controllers

Step motors and related controls

Switches

Torque sensors and other motion feedback devices

Full database access

This article highlights only a portion of the total survey results gathered. You can view the complete Customer Satisfaction database and see the data in any way you prefer—by product or by manufacturer—and drill down into the data to access the level of detail needed by going to

Weighting Process

In this year's Customer Satisfaction survey, Control Engineering and Reed Research Group created a pre-defined weighting formula that assigns weights to each possible answer in all closed-ended survey questions (weighting was not applied to verbatim responses).

Working with the preassigned weights (

1. Weighted score at bottom of question 6, 'ease of installation' = [(# very satisfied x 10) + (# somewhat satisfied x 7) + (# somewhat dissatisfied x 4) + (# very dissatisfied x 1)]/n

a) n represents total responding

b) repeat for every row in product/manufacturer sub-grouping and other sub-groupings

2. 10-100 index for product/manufacturer grouping =

a) Sum of

i) Weighted score at bottom of question 6, 'ease of installation' x [question weighting]

ii) Repeat for every row in product/manufacturer sub-grouping

iii) Weighted score at bottom of question 11 'services required' x [question weighting]

b) Sum of question weights

c) Divide a by b to get score between 1-10

d) Multiply by 10 to get index between 10-100

e) Repeat for repair and other report sub-groupings

3. 10-100 index for overall grouping =

a) Sum of

i) 10-100 index for product/manufacturer grouping x 8 [group weighting]

ii) 10-100 index for repair grouping x 1 [group weighting]

iii) 10-100 index for tech support grouping x 3 [group weighting]

iv) 10-100 index for recalibration grouping x 1 [group weighting]

v) 10-100 index for software grouping x 2 [group weighting]

b) Sum of sub-grouping weights

c) Divide a by b to get overall index between 10-100

This same formula is applied with the 'total' column to generate the weighted score for an entire product category.

Required values to score indicate minimum response bases needed to assign indices and ratings to each product/manufacturer cell. We determined that the required values to score for every sub-grouping is three respondents. When these base requirements are not met, an 'NA' is noted in the weighted score cells of the data tables. If one of the rows feeding into a sub-grouping generates an NA score, the score for that entire sub-grouping will be NA, as will the 'overall' grouping. Otherwise, artificially low indices in the sub-grouping and overall grouping will be created.

Index evaluation and labeling for each grouping:

& 60 Avoid

60-69 Poor

70 to 79 Fair

80 to 89 Very good

> 90 Excellent



No comments
The Engineers' Choice Awards highlight some of the best new control, instrumentation and automation products as chosen by...
The System Integrator Giants program lists the top 100 system integrators among companies listed in CFE Media's Global System Integrator Database.
The Engineering Leaders Under 40 program identifies and gives recognition to young engineers who...
This eGuide illustrates solutions, applications and benefits of machine vision systems.
Learn how to increase device reliability in harsh environments and decrease unplanned system downtime.
This eGuide contains a series of articles and videos that considers theoretical and practical; immediate needs and a look into the future.
Sensor-to-cloud interoperability; PID and digital control efficiency; Alarm management system design; Automotive industry advances
Make Big Data and Industrial Internet of Things work for you, 2017 Engineers' Choice Finalists, Avoid control design pitfalls, Managing IIoT processes
Engineering Leaders Under 40; System integration improving packaging operation; Process sensing; PID velocity; Cybersecurity and functional safety
This article collection contains several articles on the Industrial Internet of Things (IIoT) and how it is transforming manufacturing.

Find and connect with the most suitable service provider for your unique application. Start searching the Global System Integrator Database Now!

SCADA at the junction, Managing risk through maintenance, Moving at the speed of data
Flexible offshore fire protection; Big Data's impact on operations; Bridging the skills gap; Identifying security risks
The digital oilfield: Utilizing Big Data can yield big savings; Virtualization a real solution; Tracking SIS performance
click me