Well, you shouldn’t be doing that!

Many manufacturers outsource IT infrastructure support, including management of networks, servers, email, and desktop help. Valid economic reasons exist for outsourcing these standard services. In a business environment, not much added value is gained from performing them internally. However, big problems occur when manufacturing infrastructure is ignored in service outsourcing.

By Dennis Brandl, BR&L Consulting February 1, 2006

Many manufacturers outsource IT infrastructure support, including management of networks, servers, email, and desktop help. Valid economic reasons exist for outsourcing these standard services.

In a business environment, not much added value is gained from performing them internally. However, big problems occur when manufacturing infrastructure is ignored in service outsourcing. Often, IT infrastructure service contracts ignore manufacturing’s special requirements. Contacts are negotiated at IT management’s highest levels, frequently without manufacturing IT involvement. Most negotiations are secret, since service agreements may result in layoffs or sale of a large part of the IT department to the service company.

When manufacturing’s needs are not included in the agreement, it can be a production disaster. Several large companies, which have recently outsourced their IT infrastructure, are experiencing deteriorating infrastructure reliability because the service agreements did not differentiate between business and manufacturing infrastructures.

One company had unexpected plant-system shutdowns every Monday morning. The IT service organization had re-architected the network, placing manufacturing and business on the same network segments. Monday morning, with business systems starting up, service- and ID-requests flooded the network, overflowing the manufacturing processors’ network buffers, shutting them down. Frequent network storms—torrents of traffic—also invalidated manufacturing timing constraints, slowing down displays and data collection. Automation servers were being automatically updated with service packs/patches, without considering automation applications impact. When the IT service group was asked for help, the standard response was, “Well, you shouldn’t be doing that!”

In another case, service contacts were written in terms of hours of availability, prompting one engineer’s joke that, “The 24/7 guarantee means that the system is guaranteed to be down 24 minutes 7 days a week.” And other engineers were buying their own factory-floor-hardened switches and routers under the guise of control equipment, because all “IT equipment” had to be purchased by the service company, and factory-hardened equipment was not on its list.

Because of IT outsourcing’s increasing popularity, it’s important to have a well-documented manufacturing IT-implementation. Manufacturing management may not be involved in IT-infrastructure outsourcing negotiations. However, if records exist for uptime specifications, response time requirements, and acceptable factory-hardened IT-hardware, then documentation can be provided to the negotiating team for inclusion in the contract.

First differentiate business and manufacturing infrastructures. Most businesses use blue Ethernet cables, so consider using orange Ethernet cables for all manufacturing networks. Define and implement rules for manufacturing hardware. For example “if a piece of equipment has a manufacturing network connection—orange Ethernet cable—then it is managed using the “manufacturing infrastructure rules.” Another rule may be “If a server, switch, or router has a manufacturing network connection—orange Ethernet cable—then it must be on the approved manufacturing-equipment list.”

Define your manufacturing-infrastructure internal-service contract with up- and response-time requirements in minutes, not hours. For example, specify 10-minute problem-response times and no more than five minutes of weekly scheduled downtime. Write your local agreements as if they will be used as part of an outsourcing agreement—there is a good chance that they may be in the future. Finally, if there is an outsourcing situation, work very hard to get the manufacturing infrastructure covered by a separate agreement or addendum.

It’s unacceptable to hear, “Well, you shouldn’t be doing that!” from an infrastructure service provider.