Industrial Ethernet essentials: What you need to know
Unlike the old comfortable office and home Ethernet, Ethernet of the industrial kind is a new breed, requiring specialty knowledge and practices. If you are installing or operating an industrial Ethernet network, here are some essentials you need to know about cabling, signal quality, ground loops, switches, and traffic.
Like all networks, your industrial Ethernet is only as good as its cabling. Unlike an office or home environment, industrial applications are often electrically noisy places. In addition to high electromagnetic interference (EMI), these environments are subject as a class to temperature ranges, dust, humidity and a host of other factors not normally found in a home or office.
So, what’s the right choice of cable? In your office, commercially rated cable like Category 5 is good for up to 10 MB, and Category 5e is good for up to 100 MB. The ANSI/TIA-1005 standard states that Category 6 or better cabling should be used for hosts or devices that are exposed to an industrial environment. Category 6 cable is good for up to 1 GB at 100 meters and 10 GB at 55 meters. Category 6e cable can do up to 10 GB at 100 meters.
Compared to Category 5 and 5e cables, Category 6 cable is generally less susceptible to cross talk and external EMI noise. Cable manufacturers make industrial Ethernet cable that is also less susceptible to physical deterioration in the harsher industrial environments. When installing Category 6 cable, make sure that the RJ45 ends and jacks are also rated for Category 6, or you will be defeating the purpose of your Category 6 cable. For the best results, wherever possible use premade patch cables for short runs, with factory installed connectors. For long runs you will need to install jacks.
Cables, shielding, ground loops
To shield or not to shield, that is the question. The answer is sometimes yes, but if you do, you better do it properly. Improperly installed shielded cable can create more problems than it solves.
Shielded Ethernet cable may perform better in high EMI environments if run outside of conduit. The key to the use of a shielded cable is in proper grounding. One ground reference is essential. Multiple ground connections can cause what is referred to as ground loops, where the difference in voltage potential at the ground connections can induce noise on the cable.
A ground loop can wreak havoc on your network. To get this right, use a grounded RJ45 connector on only one end of the cable. On the other end use a nonconductive RJ45 connector to eliminate the possibility of ground loops.
If your Ethernet cable must cross power lines, always have it do so at right angles. Separate parallel Ethernet and power cables by at least 8 to 12 in., with more distance for higher voltages and longer parallel runs. If the Ethernet cable is in a metal pathway or conduit, each section of the pathway or conduit must be bonded to the adjacent section such that it has electrical continuity along its entire path.
In general, route Ethernet cables away from equipment that generates EMI. This includes things like motors, motor control equipment, lighting, and power conductors. Within panels, separate Ethernet cables from conductors by at least 2 in. When routing away from EMI sources within a panel, follow the recommended bend radius for the cable.
Bait and switches
If you are fishing for problems on an industrial Ethernet network, hubs are your bait of choice. To put it simply, never, ever use a hub in an industrial Ethernet environment. Hubs are nothing more than multiport repeaters. Eliminating use of hubs leaves the choice between managed and nonmanaged (or unmanaged) switches. While managed switches are generally preferable, they are also more expensive than nonmanaged switches. Let’s take a closer look at how a switch operates, and compare the managed and unmanaged varieties.
Every device on your network has a unique identifier, referred to as a media access control (MAC) address. This is the key to the much more discriminating behavior of a switch compared to a hub. When a switch first powers up, it initially behaves like a hub broadcasting all traffic everywhere. As devices pass information between ports on a switch, it watches this traffic and figures out which MAC address is associated with which port. It places this information in a MAC address table. Once it figures out the MAC address of a device connected to a particular port, it will watch for information intended for that MAC address, and transmit such information only to the port associated with that address.
An industrial Ethernet network carries three types of traffic. Unicast traffic routes from one point to another point. Multicast traffic routes from one point to many points. Broadcast traffic routes from one point to all points.
Once a switch has built its MAC address table, managed and unmanaged switches treat unicast and broadcast traffic identically. Generally, you should keep broadcast traffic under 100 broadcasts per second, at a bandwidth of 100 Mb. A little bit of broadcasting is an integral part of any network. Examples of items that may initiate broadcasts are devices like print servers, announcing themselves periodically to the network, which allows for a lot of nifty features for the convenience of the network users.
Snooping: Not just nosy
One of the primary differences between managed and unmanaged switches is how they treat multicast traffic. Multicast traffic typically comes from smart devices on plant floor process networks, in a connection-oriented producer-/consumer-based technology. In this context a connection is simply a relationship between two or more nodes across a network.
EtherNet/IP (an Ethernet industrial protocol managed by ODVA) is an application layer communication protocol that uses this technology. This protocol, used by many automation vendors, is based on ODVA’s Common Industrial Protocol (CIP). Industrial products you might find in a multicast group include flowmeters, variable-speed drives, and scales. Each of these items produces process data and consumes configuration data.
A device needs to be a member of a multicast group to receive group data. All members of the group receive data. You do not need to be a member of a group to send data to the group. The main problem with multicast traffic in a producer/consumer model is that traffic grows exponentially with the number of hosts. This is where the managed switch comes in.
A managed switch has the ability to turn on Internet Group Management Protocol (IGMP) Snooping. Here’s how it works. When enabled, IGMP Snooping sends out broadcast traffic to determine the members of any multicast groups. Using this information, combined with the MAC address table, allows a managed switch to route multicast traffic only to those ports associated with members of a multicast group. A nonmanaged switch treats multicast data the same as broadcast data and sends it everywhere.
If your network is using producer/consumer technology, or otherwise has multicast traffic, a managed switch is an absolute must, and worth the premium you will pay for it.
Mirrored ports, troubleshooting
There are other reasons to consider a managed switch. This class of switches usually provides error logs, control of individual port speeds, duplex settings, and the ability to mirror ports. These extra capabilities allow more precise control of network behavior and can be an invaluable aide in troubleshooting issues that will certainly occur on the network at some point.
Speaking of troubleshooting, when network performance issues occur, the first suspect often is the switch. While physical failures can and do happen with any piece of equipment, the switch rarely is the core of most network performance problems. Switches tend to be the lowest latency points in a system, typically operating 10 to 50 times faster than all other network components.
While there is excellent software to help troubleshoot network performance issues, most of it can only see broadcast and multicast traffic. That’s fair enough, because many performance issues are caused by unrestrained multicast traffic or excessive broadcast traffic. If you need to examine unicast traffic for any reason, port mirroring is the only way to do so.
So, are there times when it is OK to use a nonmanaged switch? Yes, provided there is no multicast traffic on the network. On very small, simple networks with a few devices, many people will still use nonmanaged switches. Sometimes they take half-steps and combine the two, having a few remote devices on a nonmanaged switch, which then feeds into a managed switch.
As a general practice for networks of more than a few nodes, if cost is not a primary concern, go with a managed switch, often a much better choice in hindsight.
Analyzers and monitors
Once the network is assembled, you will need a way to maintain and monitor it. It is so easy to create unintended cross-connections that expose a plant floor process network to unintended broadcast traffic, sometimes just a matter of connecting port “A” on switch 1 to port “B” on switch 2. While Joe from finance may be quite happy that he can directly transfer production data from the control system into his spreadsheet, doing so may allow Sheila’s new application in accounting to broadcast traffic that is slowing your plant floor graphic terminals to a crawl.
There are a couple of types of software that you can use to help manage things when eventual trouble occurs. To take a low-level look at a particular port, Wireshark (formerly known as Ethereal) is a popular and free open source packet analyzer that runs on most common platforms. There are a host of other similar products, open source and commercial, with various features sets and benefits commonly available for packet analysis. As a class, they provide packet level data captures giving visibility into the nature of the traffic on a particular network node.
Sometimes a broad view of your network is required. Hosts of network traffic monitoring products are available from places such as HP Network Management (formerly known as HP OpenView), SolarWinds, Network Vision, and many more suppliers. Each gives an excellent view of network traffic overall. With products of this sort, it is possible to quickly see which devices might be generating excessive traffic or broadcast storms, which are requesting large files, which nodes might be having connection problems, or which are becoming sluggish in response. As a class they also help identify and document all of the devices on a network.
An industrial network can be a beautiful thing. If done correctly, it can facilitate the flow of information between the plant floor and the boardroom. It is part of your core infrastructure backbone that makes so many of your organization’s efficiency and information technology (IT) initiatives possible. Following the essentials outlined here will position an industrial network for peak performance for years to come.
– David McCarthy is president and chief executive officer, TriCore Inc., www.tricore.com. Edited by Mark T. Hoske, content manager CFE Media, Control Engineering and Plant Engineering, firstname.lastname@example.org.