What does time-sensitive networking and real-time Ethernet data mean for the future of industrial systems?

Avnu Alliance members explain how real-time communications enabled by Time Sensitive Networking (TSN) will strengthen the future of industrial systems.
By Todd Walter, Ludwig Leurs, Anil Kumar, Paul Brooks, Bogdan Tenea, Paul Didier January 14, 2017

Figure 1: Multiple applications can use Avnu Common Network Services. Image courtesy: Avnu Alliance Industrial GroupThe Industrial Internet of Things (IIoT) brings the opportunity to create smart systems that share data between devices across the enterprise and in the cloud. As interest in IIoT and Industrie 4.0 continues to grow, more designers, engineers, and end-users are looking to Time Sensitive Networking (TSN)—a set of standards under revision from the Institute of Electrical and Electronics Engineers (IEEE)—to solve the need to process raw data in a timely fashion, reduce latency, increase robustness, and provide a foundation for more advanced manufacturing and product models where data can be more flexible and shared between layers of the control system. (The TSN Work Group, part of the IEEE 802.1 Working Group [Ethernet], evolved from the Audio Video Bridging [AVB] Work Group.)

To support these new capabilities of an IIoT infrastructure and to connect network devices, there is a need for an interoperable system that allows multiple manufacturers, protocols, and organizations to share the same TSN network. To prepare for this, Avnu Alliance selects and certifies a set of underlying mechanisms to meet a given market’s requirements, creating a common infrastructure that others can build on to enable convergence between operational technology (OT) and information technology (IT). This convergence increases connectivity to industrial devices and enables a faster path to new business opportunities including Big Data analytics and smart, connected systems and machines.

Avnu Alliance is an industry partnershipthat fosters and develops an ecosystem of manufacturers providing interoperable devices for industrial networked systems. The industrial segment within Avnu includes member companies across the entire ecosystem and supply chain including IT vendors, silicon providers and industrial suppliers all working together to create an interoperable ecosystem of low-latency, time-synchronized, highly reliable networked devices. Avnu creates certification programs to ensure interoperability of networked devices for multiple markets and applications from machine control, health care, smart grid and robotics to power generation and process control, as well as automotive for the future of autonomous driving. Each of these areas employ embedded processing to perform closed-loop autonomous control, have high availability requirements, and are highly software defined.

Control Engineering (CE) caught up with several Avnu Alliance members to ask how the IIoT is affecting industrial networks and what TSN means for the future of industrial systems. Todd Walter is chief marketing manager, National Instruments and Avnu Industrial Segment chair; Ludwig Leurs is director of Ethernet convergence at Bosch Rexroth; Anil Kumar is principal systems engineer of the Internet of Things group at Intel; Paul Brooks is business development manager at Rockwell Automation; Bogdan Tenea is product specialist at Ixia; and Paul Didier is industry solutions architect at Cisco.

CE: How has the evolution of Ethernet changed the approach to designing industrial control systems?

Ludwig Leurs: Standard Ethernet has evolved to enable next generation control systems. With this, recent work by IEEE 802 LAN/MAN Standards Committee (LMSC), the Internet Engineering Task Force (IETF), and other standards groups has extended the ability to operate time-sensitive systems over standard Ethernet networks, supporting diverse applications and markets including professional audio/video, automotive, and industrial. These standards, driven primarily by the IEEE 802.1 TSN task group, define new mechanisms for creating distributed, synchronized, real-time systems using standard Ethernet technologies that will allow convergence of low-latency control traffic and standard Ethernet traffic on the same network.

TSN supports real-time control and synchronization, for example, between motion applications and robots, over one Ethernet network. TSN can at the same time support other common traffic found in manufacturing applications, driving convergence between IT and operational technologies. TSN is not a disruptive technology, it enables existing industrial real-time control applications to coexist and interoperate, such as the new OPC UA publisher/subscriber model currently being finalized in the OPC Foundation. As Avnu has shown success in audio/video applications, it will be helpful being able to reuse this technology in modern machines using video cameras in control loops.

As TSN supporting network infrastructures become more prevalent, we will see that many of today’s modified Ethernet networks can move to TSN-based networks using their OT-based application layers on standard 802.1 Ethernet.

CE: With networks becoming increasingly congested with IIoT data, how does TSN ensure that data gets communicated to make IIoT viable to help users?

Anil Kumar: TSN promises a world of smarter, hyper-connected devices, and infrastructure. Manufacturing machines, transportation systems, and the electrical grid will be outfitted with embedded sensing, processing, control and analysis capabilities. Though much of the data collected by industrial sensors and control systems in an IIoT application are not time-sensitive, there will also be a great deal of mission-critical, time-sensitive data that must be transferred and shared within strict bounds of latency and reliability.

Figure 2: This image shows a time-sensitive networking (TSN) Testbed at IoT World Congress in October 2016. Image courtesy: Avnu Alliance Industrial GroupRequirements of TSN such as a common clock for transmission scheduling, latency provisions, reserved bandwidth and redundancy make it so that TSN ensures precise time synchronization to support synchronization of multiple data streams. Large data sets from machine vision, 3-D scanning, and power analysis can put a strain on network bandwidth. TSN will support full-duplex standard Ethernet with higher bandwidth options such as 1 Gb, 10 Gb, and even the 400 Gb version in IEEE 802.3. It also offers top-tier IT security provisions and interoperability with scalability that can grow into large systems. TSN can also integrate with existing brownfield applications and standard IT traffic and is future-proof, as it is constantly evolving.

CE: How will the new generation of Ethernet with TSN features improve over previous generations of the standard?

Paul Brooks: TSN’s value is derived from simplifying convergence and increased connectivity, unlocking the critical data needed to achieve the IIoT promise of improved operations driven by Big Data analytics. Designers can take advantage of the advancements in processing, communications, software, and system design originating from the consumer world with a standard that evolves while providing precise timing.

For industrial markets specifically, these standards support the ability to simplify development and deployment of distributed, synchronized, control systems commonly found in a variety of industrial applications including machine control, factory automation, power generation and distribution, oil and gas exploration, etc. Many of today’s generation of control applications are deployed using nonstandard network interfaces and infrastructure. Limitations to Ethernet have caused a fragmentation and lead to the creation of variants of Ethernet that include real-time performance to meet the needs of measurement and control applications.

These variants often modify hardware and infrastructure components of the Ethernet stack to provide guarantees on behavior. These modifications provide the latency and bandwidth reservations needed for control. However, because they required modification of hardware they are no longer interoperable with standard Ethernet, and they are much harder to access on the network, if at all. Additionally, in some cases the modifications prevent the systems from evolving along with Ethernet standards, and they become limited in bandwidth compared to standard Ethernet.

Unlike the proprietary modified variants of Ethernet, TSN and related efforts, including IETF projects such as DetNet [deterministic networking], provide mechanisms to expand solutions to these applications using standard Ethernet technologies in a manner that enables convergence between OT and IT data on a common physical network. TSN ensures the predictable delivery of time-critical traffic and has the potential to integrate with existing applications as well as standard IT traffic to strengthen interoperability. Other advantages include simplified development and faster time to market.

CE: How are Avnu Alliance and its members driving this next-generation standard?

Todd Walter: Avnu Alliance is a nonprofit member run organization promoting, educating, and enabling companies to create devices that provide reliable, synchronized deterministic networks. Our focus is the creation of an open, interoperable ecosystem through interoperability verification and testing across broad markets. As the capabilities to Ethernet are updated with new standards, Avnu Alliance provides the community to discuss, collaborate, certify, and promote the continually evolving network capabilities that enable the convergence between OT and IT.

Avnu Alliance is made up of key technology and equipment providers including leading silicon suppliers such as Intel and Analog Devices; network infrastructure vendors such as Cisco and Belden Hirshmann; end-station suppliers such as National Instruments, Rockwell Automation, and Bosch Rexroth; and test equipment providers such as Ixia and Sprient. By covering all levels of the value chain and markets including industrial, automotive, and professional AV , we are able to effectively create a common, global foundation across all markets.

To promote the shared network and accelerate products to market, Avnu facilitates a common technical platform through a number of services including open-source software, hardware reference designs, test plans, and certification services to develop and verify the correct operation and implementation of TSN-enabled products.

Avnu Alliance also coordinates with other organizations such as IEEE, IETF, ODVA, OPC-UA and Industrial Internet Consortium (IIC) with the goal of creating this common foundation allowing multiple protocols and manufacturers to coexist on the same TSN network. Avnu also has made it a priority to educate other groups on TSN, which has led to the development of liaisons, where various groups can benefit from shared knowledge, which then contributes to a broad and growing membership.

Avnu has collaborated with the IIC to develop the world’s first TSN testbed to showcase the value to the standards and the ecosystem of manufacturing applications, including the ability for IIoT to incorporate high-performance and latency-sensitive applications. We are hosting the testbed at National Instruments.

CE: Have there been any recent advancements in TSN technology that you can share?

Bogdan Tenea: It was actually just back in September, at the IEEE Automotive Ethernet & IP Technology Day, that a couple of Avnu members demonstrated for the first time a new IEEE TSN standard called preemption. There, the test equipment from Ixia proved interoperability and measured the performance improvements of prototype devices supporting this new technology.

Preemption (IEEE 802.1Qbu/802.3Qbr) [Standard for Local and Metropolitan Area Networks – Media Access Control Bridges and Virtual Bridged Local Area Networks- Amendment: Frame Preemption] is a new addition to the Ethernet standard from the IEEE that allows a high-priority frame to interrupt a lowpriority frame in transmission, minimizing latencies in the high-priority traffic. In industrial automation control system (IACS) applications, preemption can further convergence of multiple networks of differing technologies into one Ethernet and IP infrastructure, enabling self-organizing plant operations and order controlled production. By highly decreasing the impact of lower-priority traffic on important traffic, both types of traffic can be mixed on the same link.

Avnu has a very important role in speeding time to market—Avnu enables member companies to organize NDA-plugfests to experiment with cutting-edge technology, without worrying about confidentiality. To this sense, the public demo I mentioned was possible because we had the chance to organize a plugfest, under the Avnu umbrella, to test preemption just a few weeks before.

CE: What needs to happen with network infrastructure to support future demands from industrial systems?

Paul Didier: With TSN, we have established a key building block for industrial vendors to now build IoT-ready devices and applications that use a converged network. The standards still are evolving here. We still have to finish standards on: resiliency, no packet-loss over multi-pathed networks; the ability to police traffic to further protect the TSN traffic; pre-emption, the ability to break up large packets and more.

But beyond the TSN work, we are working on numerous other challenges directly related to industrial systems. At the physical layer, coming up with more Ethernet variants: lighter copper cabling (such as single twisted pair), longer/shorter distances (10s to 1000s of meters), and more PoE options, like lower power for intrinsically safe environments. This enables deployment in more harsh, industrial environments. In the IETF DetNet working group, we are extending those TSN "deterministic" capabilities over larger, scalable networks to handle much larger systems. We we also will bring TSN into standard, open wireless networks.

All this dovetails well with key initiatives in general networking, including the ongoing efforts to bring higher bandwidths to more devices. But I would like to especially point out the push to automate network configuration and management often referred to as Software-Defined Networks. This ability to programmatically configure and manage the network will take the complexity away from the industrial ecosystem that requires simplicity and deterministic behavior. At Cisco, we are developing more than switches to support TSN.

CE: What can we expect to come from Avnu and TSN in 2017?

Walter: As the latest generation of projects finish and become formal parts of the standard, we are seeing increased interest from nearly all vendors. The first TSN industrial products were available to select customers in 2016 including controllers from National Instruments with Intel TSN silicon and drivers as well as Cisco TSN switches. In 2017, we will see more products from multiple Avnu members. The fact that this is based on the open Ethernet standards means that technology adoption benefits from the investment of all the companies and brands supporting it and is not limited by one company’s business plan or future decisions on how they are going to develop and support the technology. Manufacturers within the industrial market benefit from Avnu’s collective multi-industry experience and networking expertise to expedite advanced networking implementations ultimately leading to a faster path to market.

As an example, Avnu has recently completed a best practices document entitled "Theory of Operation for TSN-enabled Industrial Systems," describing the system architecture and requirements for an industrial model that enables multiple industry groups, vendors, and protocols to share a TSN network. This document introduces the fundamental mechanisms needed for a system architecture to build on, including time synchronization, quality of service using scheduled transmission, and network configuration. It walks through the requirements for several industrial use cases including how to enable and integrate non-TSN technologies where needed. Avnu Alliance members have created this document to help designers and engineers in the industry understand the real-world application context and build a TSN network that is configured for multiple vendor and industry groups.

As the standards and networks continue to evolve, so does Avnu’s work to define and certify the standard foundation. This eventually will include mechanisms such as frame preemption, redundancy, ingress policing, and security. In addition, Avnu’s defined foundation will continue to support additional capabilities, including support for multiple IEEE 1588 profiles, guidelines for scaling to very large network architectures, and aggregation/composition of multiple networks into one TSN-enabled network domain.

Avnu certification also will evolve with the standards and with interoperability as a baseline promise, backwards and forwards compatibility is assured. This means that system designers and engineers won’t architect a network that is outdated before it is even installed.

Avnu Alliance also has an Industrial Advisory Council where manufacturers and end users can learn more about the standards and get involved with shaping the future of industrial networking. There is no fee to join, and the group provides a perspective on the evolutions happening in standard Ethernet while giving the opportunity to network with others in the area of converged, time-synchronized communications.

Edited by Emily Guenther, associate content manager, Control Engineering, CFE Media, eguenther@cfemedia.com.


Key Concepts

  • Avnu Alliance’s role in educating manufacturers and end users
  • The future of industrial networking
  • Standard Ethernet and TSN technology.

Consider this

Would having interoperable TSN industrial networking devices ease system integration?

ONLINE extra

See related URLs.




Also see the following articles linked below.