100x the systems with the Industrial Internet of Things
Engineering and IT Insight: Does your manufacturing IT infrastructure have tools to handle 100 times the number of current connections and manage tens of thousands of new smart network connected devices, as industry moves to the Industrial Internet of Things (I2oT) and distributed control, with every device in a facility connected to a plant-wide Ethernet network? What got you here won’t get you there.
Is your manufacturing IT infrastructure ready to handle 100 times the number of current connections? Do you have the tools in place to manage tens of thousands of new smart network connected devices? This is the situation you may be facing as industry moves to the Industrial Internet of Things (I2oT), distributed control, and every device in a facility connected to a plant-wide Ethernet network.
These devices will be simple and complex with wired and wireless connections, ranging from sensors with a few pieces of information to control values with complete diagnostic and advanced control information to analyzers and vision systems with potentially massive amounts of information. Heed these six considerations as we interconnect with I2oT.
1. Why? 100x = interconnected value
With Moore’s Law driving down connection costs and Metcalfe’s Law driving up the value for each connected device, there will be a large economic incentive to put every device on an IP network. A typical factory may have a few dozen PCs and printers, but it will also have thousands of sensors and actuators so that each will become a separate managed IP device. A chemical plant or refinery may have hundreds of PCs and printers, but tens of thousands of sensors and control devices. The scale and scope of an internal I2oT network will dwarf the size of your company’s current network. While any company’s I2oT network may be small compared to the giant commercial systems, such as Amazon, Google, IBM, and Microsoft, it will be 100 times larger than what currently is internally managed. Manufacturing IT will need new tools, management systems, and support systems to handle plant-wide networks with tens of thousands of devices because "What got you here, won’t get you there."
2. Use IPv6
There are steps to take to prepare for this tsunami of devices. One of the first things to do is to move all devices to the IPv6 Ethernet standard. This is the new standard for Ethernet addressing, replacing the original IPv4 standard that has run out of Ethernet addresses.
Without a move to IPv6, you will usually be limited to 255 devices on a segment and no global address space. This will significantly complicate device management and information sharing.
3. Managed switches
Next, ensure that you are using managed switches and not unmanaged switches or routers. Managed switches give the ability to manage control congestion on networks and network security. Thousands of devices on an unmanaged network, even if they communicate only a few times a second, will overload the network. Even with IPv6 you will need to segment your network, and these segments should map to your operational hierarchy of work centers and work units.
These are ISA 95 definitions, where work centers correspond to process cells, production lines, packaging lines, and warehouses. Work centers are made up of work units. Most communication is within work units, or across work units in a work center, usually with limited traffic across work centers or up to higher-level systems.
4. Alerts, notifications
Another step to take is to start adding performance monitoring to all network segments. If possible, set up alert and notification limits on traffic rates, with automatic text messaging to support staff. It’s also a great idea to track network traffic rates, collision rates, and average message delays in your data historian for analysis and troubleshooting. If you are not already using network management tools in your current system, then start using them to take snapshots of your network configuration, to determine average and peak network traffic, and to fine-tune your managed switch settings. Too many companies still rely on manually updated spreadsheets to maintain IP address lists and network segment specifications, and make only guesses at their actual network loading. [subhead]
5. Cyber security, network security
Security with thousands of networked devices will also be important. With the move to IPv6 and every device having a globally unique address, you will need to protect your network from outside access using managed switches and firewalls. Having multiple firewalls will be a good idea. While they will require management, they can prevent infections from bringing down an entire network. The processors in smart devices will be based on off-the-shelf hardware usually running a standard operating system, so they must be managed and protected the same way that PCs and printers are today. One important point to remember is to change the default passwords on devices that can be remotely configured.
6. Policies and patching
Many of these new networked devices will be hardcoded, but a substantial fraction will be patchable and updateable. The patchable devices will really be full-fledged computers and can theoretically be compromised the same as any laptop or desktop device. With networks containing thousands of devices, updating and patching these devices must be centrally managed and controlled. You cannot let each device handle its own update policy. It will be vital to maintain a local repository of patches and updates so that you can decide when to patch. You will also need a complete and up-to-date inventory of all network devices, which includes the device owner, product supplier, make, model, current software version, and current patch level. Even hardcoded devices should be in your inventory because when a hardcoded device is replaced, the new device may have different characteristics, network traffic patterns, and vulnerabilities. You should have patch management procedures in place to regularly check for the availability of patches, to download the patches to your local repository, and to perform an assessment of which patches are needed.
Managing the I2oT, the Industrial Internet of Things, will require that automation engineers become even more familiar and comfortable with information technologies. Your ability to configure a control system with distributed intelligence, to monitor and manage the network traffic, and to protect control network segments will be critical for implementing successful projects in the coming Industrial Internet of Things.
– Dennis Brandl is president of BR&L Consulting in Cary, N.C., www.brlconsulting.com. His firm focuses on manufacturing IT. Edited by Mark T. Hoske, content manager, CFE Media, Control Engineering and Plant Engineering, firstname.lastname@example.org.
This posted version contains more information than the print / digital edition issue of Control Engineering and links to related articles and concepts.
See related stories from Brandl below.