NETWORK INFRASTRUCTURE is one of the most vital yet undervalued business assets. Lose the network and your company potentially loses phones, email, Internet, access to business systems or control/visibility of the manufacturing process. Many businesses strive to provide optimum versions of the devices connected to the network, such as computers, phones, and machines, yet attempt to economize on the network infrastructure that supports these devices.

This article discusses essential knowledge and tactics in current, near future, and distant future time domains that guide network plans. It also suggests resources and strategies to assist in the design, implement, operation, and maintain phases.

Current industrial networks

Today industrial networks are a composite of Ethernet protocols and what industry experts term “legacy protocols.” Legacy protocols are a telling term because like other legacies, we must live with them for a while.

Legacy protocols age and become more difficult to support over time. This issue is further exacerbated by the aging workforce megatrend. A large portion of the support staff for legacy protocols has reached retirement age.

Forward-thinking organizations instituted plans to retain this outbound knowledge. Other companies meet the need by engaging professional services organizations backed by major automation manufacturers.

Infrastructure investment versus longevity.

Network refresh rates

Industrial networks have added expectation versus their enterprise counterparts. Not only does the business require them to operate at peak levels, industrial networks have the longest refresh rate of any business network.

Where data centers are refreshed every three to five years, industrial networks are refreshed every 12 to 15 years. Further, the supporting physical infrastructure is often in place for 20 plus years. A major capital expenditure is required to install and commission a new network.

ROCE expectations are extremely high for all businesses. Industrial network refresh rates are accelerating over time, as companies work to balance investment performance and network performance. However, we must still anticipate longer than desirable refresh rates to make planning effective.

Network planning and management

The PPDIOO Process is a design and management methodology that spans the entire network lifecycle.

Like other business assets, a rigorous process governing the network ensures efficacy and availability as time goes by. There is not a particular methodology that is superior to others.

Realistically, the best run business networks result from a collection of elements intertwined with the governing process. Justification for creating the process is simple.

• Lowers the total cost of network ownership
• Improves business agility
• Helps the business respond quickly and effectively
• Increases availability

PPDIOO Process

The PPDIOO Process is a design and management methodology that spans the entire network lifecycle.

Prepare: Business agility is a result of good preparation. This phase is used to consider the broad vision, requirements and technologies you can employ to make your business more competitive.

Plan: Successful technology deployment must have an accurate assessment of the current state network, its security posture, and the business readiness to support the chosen solution.

Design: A detailed design reduces risk, avoids delays, and controls the total cost of network deployments.

Implement: Here the business works to integrate devices and new capabilities in accordance with the design phase without compromising network availability or performance.
Operate: The business proactively monitors the network to improve service quality, reduce disruptions, and mitigate outages while maintaining high availability, reliability, and security.

Optimize: Best-in-class businesses never stop looking for a competitive edge. So, continuous improvement is a mainstay of any network lifecycle.

Network Documentation: Does comprehensive, up-to-date documentation exist for your network? Most companies do not have the required documentation, but all of them should consider it an absolute necessity. Accurate documentation and identification substantially shortens the time to recover from a network issue. Many methods to generate this documentation exist, ranging from a summer intern project to engaging a professional services organization to assess and document the network.

Network Assessment: Professional services organizations, backed by a major automation manufacturer, perform cabling and network infrastructure assessments. Many use software that “crawls” unobtrusively through the network, discovering and visualizing the network footprint.

Legacy Protocols: During the assessment process, pay attention to legacy protocols, i.e., Fieldbus. If legacy protocols are present, plans to migrate them to a modern technology must be at the forefront. Legacy protocols migrate out of the network as it ages because they become difficult and expensive to support, even if their performance is adequate. Their replacement is infrequently a “rip and replace” proposition.

Network Longevity: Consider the age of the existing network and physical infrastructure. Are any of the active components, network switches, servers, programmable controllers, drivers, or other end devices, approaching “end of sale” or “end of support” from the manufacturer? Aged active components have support costs that grow exponentially after a certain age, so they need to retire before your business is frantically searching for good used replacements to get manufacturing up and running.

Evaluate the age of transmission media and its condition. Consider wire speed as well. Category 5e is adequate for 10Mb/s and 100Mb/s traffic but is insufficient long term. Pay attention to connections and the cable. Jacket materials are commonly thermoplastic which ages over time, particularly in challenging environmental conditions like UV exposure, temperature extremes, and chemical exposure. The same exposures age the metallic portion of connectors. Along cable routes, look for sharp bends and areas where cables appear to have been struck or deformed.

With multi-pair copper Ethernet media, these physical deformations displace pairs in the cable, damaging its performance. With fiber optic media, there can be microfractures from physical deformation that attenuate or, in severe cases, interrupt signal flow.

Protocol distribution in industrial networks.

Cabled infrastructure

As the network grows the cabled infrastructure must evolve. There are two cabling topologies used for industrial networks, point-to-point and structured cabling. Older industrial networks employ a point-to-point structured cabling topology where each connected asset has a home run cable to the control room or data center. Engineers chose this solution in the belief that connection points were vulnerable and would cause reliability issues.

More connections meant more risk. In the very early days of Ethernet connectors there may have been some credence to that conclusion. Connector and media design as well as manufacturing processes are significantly more robust today. So, in modern networks, this argument is no longer valid. Also, the flexibility and network resiliency gained from a structured cabling topology far outweighs the point-to-point reliability argument.

Enterprise networks were once point-to-point cabling. They quickly evolved to structured cabling for several reasons, notably:

• Structured cabling provides the needed flexibility to accommodate Moves, Adds, and Changes (MACs)
• Structured cabling can adapt network topology and configuration to business needs without pulling new cabling and resultant disruption of business activities
• Structured cabling topologies enhance network reliability and recovery speed from outages

Industrial networks are on this evolutionary path because the value proposition for structured cabling networks is so strong. Elimination of downtime is the strongest argument for structured cabling topologies in industrial networks. Industrial network downtime is easily monetized in lost production dollars. As such, a business can readily justify adoption of this topology.

Structured cabling enables patching or otherwise re-directing network traffic to rapidly address infrastructure-related outages. It allows outages due to fault in the horizontal cabling to be immediately addressed by patching to a different horizontal link. After the outage is resolved, the patching infrastructure permits technicians to quickly attach diagnostic instruments to the failed link. The link can be returned to normal service with minimal disruption in network operation.

Structured Cabling

When installing structured cabling, it’s good to have a few guidelines in mind to ensure maximum viability from the new installation.

• Require the cable installer to connect a network analyzer to each link installed, including spares
• Measured link performance becomes one of the job completion deliverables
• Doing so establishes that the link delivers expected transmission performance, not just electrical continuity
• Further, if there are problems with a link in the future, baseline performance data exists in your files
• Premium cable manufacturers extend a generous warranty in exchange for fidelity to their offering using a certified installer; It is a worthwhile investigation when selecting materials and installers

Network management software

Another important topic for the current day network discussion is a 3-letter acronym, NMS. NMS stands for Network Management Software. It is an emergent category of software for industrial networks. Just as the name implies, it is purpose-built software used to manage networks. There are Enterprise NMS solutions and have been for a while. Due to the unique properties of industrial networks, these tools are not suited for the job. When selecting this category of software, make certain the NMS solution you consider is purpose built for industrial networks.

The clarion call for this type of software rises from the increasing sophistication and complexity of business networks. The ability to quickly ascertain what’s connected, the health and workload on the network, misconfiguration of devices and of course, failing devices or connections, is key to effective operations with minimum downtime.

Industrial networks have two main NMS use cases. A consultant or system integrator may be working with the business uses NMS software located on their computer to discover and visualize the network. This application can be used to assess and document the current state network in anticipation of service activities that the consultant undertakes on behalf of the business.

Or an NMS solution is installed permanently in the network, typically on a server in the DMZ so the entire industrial network is visible and monitored. This application acknowledges the dynamic nature of the network and acts as a watchdog sniffing out problems; visualizes the network so a common understanding of status is provided for varying worker experience levels and dependent on the NMS solution; and provides a portal for secure remote access when needed.

The first use case focuses on the needs of network maintenance. An expert uses the NMS tool to discover and visualize the network. This step generates a baseline documentation package for the network.

Typically, businesses retain this expert to perform maintenance, usually to upgrade or expand to the network. Up-to-date documentation for the network is a welcome latent outcome of the exercise. While the expert’s NMS package is connected to the network, performance and health metrics can be seen, helping the expert spot deficiencies that must be corrected. However, these values are a “snapshot” in that the NMS solution does not remain connected to the network.

The second use case addresses a greater portion of the network lifecycle. In this use case, the NMS solution resides in the network, typically on a server in the DMZ or the Manufacturing Zone. Residence in the network allows the NMS software to act as a dashboard, allowing network users to see network health and performance.

Further, more members of the workforce interact with the network nowadays, all with varying levels of network knowledge. These workers need information out of the network to ascertain if there is a network-related problem slowing down production. A production planner can use that information to make better decisions but may not have the needed skills to access the information.

Living in the network NMS software tracks traffic and bandwidth, suggesting future improvements to the network. And of course, it is generating and maintaining an up-to-date view of the devices and connections in the network, solving the accurate documentation dilemma discussed earlier.

Finally, best-in-class NMS packages facilitate secure remote access to the network so you can enlist the help of experts without the time and cost involved with travel. In addition, the resident NMS software approach lessens the reliance on outside experts for diagnostics and network management assistance.

Reference architectures

Reference architectures are a considerable asset to the present and future states of business networks. Reference architectures streamline the deployment of standardized networking technologies and convergence of manufacturing and business networks into a cohesive whole. In short, reference architectures provide confidence and the necessary background to design, deploy, and operate a robust, reliable network.

Reference architectures provide valuable common ground to enhance the collaboration of OT (control engineers, manufacturing IT, etc.) and corporate IT staff. This common ground removes obstacles and speeds the combined team toward achieving business goals. Historically there has been some discord between IT and OT realms due to the proprietary and obscure nature of industrial networks, especially legacy protocols.

In the realm of reference architectures for industrial networks, the zenith is the Converged Plantwide Ethernet (CPwE), reference architecture. CPwE is a collection of industrial reference architectures that are use case driven and supported by rigorous testing and validation. The use cases selected represent important business needs and are garnered by exhaustive voice of the customer research.

The reference architectures are presented in a published document titled, “Converged Plant wide Ethernet Design and Implementation Guide.” The document content is dynamic; new architectures are proposed for inclusion based on VoC research. All networks are assembled, commissioned, and tested prior to publication.

Validation and performance data is published for each architecture, along with a hardware bill of materials, firmware versions used, and any software included in the test setup. CPwE remains ever green; as new devices become available and older devices go end of sale/end of support, the core reference architectures, (e.g., resiliency) are refreshed with new testing/validation and published performance data.

Some businesses use CPwE architectures as a springboard to create architectures that suit very specific needs. However, because of their CPwE basis, the architectures are built on a solid foundation.

Network building blocks

Another practice that has risen to prominence are pre-populated and pre-configured network enclosures. The solution allows companies to rapidly deploy or expand without bearing the time penalty and expense of a bespoke enclosure. These solution elements follow the functions of the 3-tier architecture of Converged Plant wide Ethernet – Access Layer, Distribution Layer and Core Layer – with appropriate solutions for each network layer.

Enclosure designs are validated electrically and thermally to eliminate risk during installation and commissioning. Active component placement within the enclosure is optimized for function, thermal performance, and maintainability.

Since the network building blocks are built to a validated design, companies gain enhanced supportability through their use, avoiding the “snowflake” scenario where each bespoke enclosure is “just a bit different.” This factor is important in a local deployment but becomes vital when multiple locations across a global footprint are considered.

Worker education

Businesses need to invest in their personnel as certainly as they make other business investments. Creating productive collaboration between IT and OT staffs is of immense benefit to businesses. The progression towards more ubiquitous use of Ethernet sets the stage for this collaboration to occur.

To hasten engagement, worker training investments are necessary. Industry experts have long provided training and certification for IT staff in Ethernet-based enterprise applications. Training materials to support industrial staff in a similar realm have been sparse at best.

Two noteworthy training offerings for industrial staff are Cisco Certified Network Associate (CCNA) industrial certification offered by Cisco and Industrial IP Advantage (IIPA) Training.

The Cisco Certified Network Associate Industrial (CCNA Industrial) certification is for plant administrators, control system engineers, and traditional network engineers in the manufacturing, process control, and oil & gas industries, who will be involved with the convergence of IT and industrial networks.

This certification provides candidates with the necessary skills to successfully implement and troubleshoot the most common industry standard protocols while leveraging best practices needed for today’s connected networks. There are prerequisite certifications as a gateway to CCNA Industrial. These are Industrial Networking Specialist, CCNET, CCNA Routing and Switching or any valid CCIE certification.

Training offered by the Industrial IP Advantage is delivered online and therefore, is self-paced. The training targets control engineers, IT professionals, system integrators, and machine builders. This multi-part eLearning program combines practical guidance with reference architectures on IP addressing, network topologies, switches and routing infrastructure, physical cabling and wireless, virtualization and cloud technologies, security measures, and more.

Conclusion

Industrial network infrastructure is a valuable business asset. Investments in legacy industrial networks require a clear migration path to optimize return on assets while not missing out on performance enhancements from new technologies.

A robust, well-executed physical layer is foundational to this asset continuing to deliver value. Rapidly emerging technology advances such as the Internet of Things, Wireless Sensor Networks, Power over Ethernet, and Time Sensitive Networking can further leverage your network with a little education and planning.

Application report by Panduit