TechnologyMarch 30, 2021

How to “practice” TSN on a Linux platform

Technical system diagram

An easy way to enter the new generation Industrial Ethernet, Time Sensitive Networking (TSN) world is by connecting PCI Express TSN network card on an industrial computer. TSN network cards on Linux platforms can provide a convenient development and testing environment to verify enhanced TSN functionalities.

The global coronavirus (COVID-19) epidemic has continued to spread since Year 2019, COVID-19 lockdown had significantly affected the operating mode and working habits of manufacturing industry around the world, such as factory production drop, goods delivery limitation, travel restrictions, etc. The use of Human-Robot Collaboration (HRC) between human and machine is used to improve factory production efficiency and manufacturing quality. This marketing trend brings new opportunities to accelerate the development of smart factory industrial automation applications.

The global strong smart manufacturing demands brings the rapid growth of Industrial Ethernet market shares compared to classic fieldbuses. Industrial Ethernet market shares (52%) have overtaken fieldbuses (42%) since Year 2018 according to HMS’ Industrial Network Market Shares Annual Reports. The newest HMS Industrial Network Market Shares 2020 Report mentioned “HMS Networks annual study of the industrial network market shows that Industrial Ethernet increases its market share to 64% of new installed nodes (59% last year), while fieldbuses drop to 30% (35).” It means the Industrial Ethernet market shares are going higher and the classis fieldbuses market shares are going lower every year.

Time Sensitive Networking (TSN)

System architecture diagram.

The standard Ethernet with TCP/IP is non-deterministic communication and is not suitable for industrial automation applications. To support real-time, deterministic communications over Ethernet, the global industrial network manufacturers have developed different industrial Ethernet solutions, such as PROFINET, EtherCAT, EtherNet/IP, Modbus TCP, POWERLINK, SERCOS, CC-Link IE, etc., and these industrial Ethernet solutions have also been practiced on respective isolated industrial Ethernet networks in factory. These isolated industrial Ethernet networks can only be connected by using different industrial fieldbus gateways.

For an Industry 4.0 smart manufacturing ecosystem, the smart factory cloud server should be able to collect and analyze all ecosystem information, such as engineering design sub-system, factory manufacturing sub-system, stocks and supply chain sub-system, etc. so manufacturers can maintain the engineering design, factory manufacturing, stocks and supply chains, etc. resources based the real customer demands more efficiently.
To reach these requirements, manufacturers need converge the non-real-time IT (Information Technology) networks and real-time OT (Operation Technology) networks into a single network. It is hard for the traditional industrial Ethernet solutions, but the new generation industrial Ethernet, Time Sensitive Networking (TSN) technology can easily converge the IT and OT networks with real-time, deterministic, security communications.

Time Sensitive Networking (TSN) technology is a set of IEEE 802.1 standards under development by the Time Sensitive Networking Task Group (TSN TG) of IEEE 802.1 Working Group. TSN is an OSI model Layer 2: Data Link Layer (DLL) communication technology so it can support hard real-time, deterministic and low latency communications over standard Ethernet. Different industrial automation machines have different industrial communication languages in factory. This problem cannot be improved by TSN technology.

The Open Platform Communication Unified Architecture technology is a cross-platform machine-to-machine industrial communication protocol standard developed by the OPC Foundation based on the application layer of the OSI seven-layer model, which provides common industrial communication languages between different industrial automation machines. The combination of TSN with OPC UA, which provides the key technology to realize Smart Factory and Industrial Internet of Things (IIoT), becomes a rising star of next generation industrial communication technologies.

PCIe TSN network card solutions

To help designers to evaluate TSN functionalities on industrial computers, ASIX AXM57104 Quad Port TSN Gigabit Ethernet PCIe NIC Card offers free Board Support Package (BSP) for designers to easily implement TSN functions on industrial Linux platforms.

AXM57104 supports enhanced TSN functions included the timing and synchronization compliant to IEEE 802.1AS-Rev/AS and IEEE 1588V2, the Forwarding and Queuing of Time Sensitive Streams (FQTSS): Specifies Credit-Based Shaper (CBS) compliant to IEEE 802.1Qav, the Time-Aware Shaper (TAS) compliant to IEEE 802.1Qbv, and the Per-Stream Filtering and Policing (PSFP) compliant to IEEE 802.1Qci. AXM57104 also supports 32 synchronous I/O pins, one Pulse Per Second (PPS) output and FPGA hard-core field upgradable via In Application Programming (IAP) for TSN standards evolution.

The AXM57104 is a cost-efficient PCIe TSN Network Card solution for Industrial Internet of Things (IIoT) applications to enable TSN functions on industrial automation platforms, Fieldbus over TSN gateways and converge the non-real-time IT network and real-time OT networks (included different industrial Ethernet protocols such as PROFINET/EtherCAT/ EtherNet IP/etc.).

System timing diagram

TSN NIC system architecture

The AXM57104 BSP is developed under Ubuntu 16.04 LTE with Linux kernel 4.15.0 platform, which includes AXM57104 Linux PCIe network driver, AXM57104 TSN service and applications modules, etc. Designers can easily evaluate the supported TSN functionalities on Linux platforms by running these Linux software stacks.

The system architecture of TSN network card includes the hardware blocks and software blocks as below figure. The hardware blocks with blue color are ASIX AXM57104 FPGA controller with TSN Gigabit Switch and PCIe NIC functions. The software blocks with green color are provided by ASIX AXM57104 BSP, the software blocks with yellow color can be manually installed from Linux kernel open.

Practice TSN on Ubuntu Linux Platform

Designers can connect AXM57104 TSN network cards on Linux platforms to practice the enhanced TSN functionalities, such as gPTP Time Synchronization (IEEE 802.1AS_Rev), Time Aware Shaper (IEEE 802.1Qbv), Credit Based Shaper (IEEE 802.1Qav), etc. and run Linux VLC Media Player to verify these TSN functions.

There are three Linux platforms installed AXM57104 PCIe TSN network card to establish a TSN-enabled network. All devices in the TSN-enabled network MUST support TSN functions.

The Linux Media Server platform is configured as VLC media player server to generate the demo video streams. Need to add two visual network interfaces on AXM57104 NIC for two VLC Server video streams, one visual network interface assigns VLAN priority level 2 for VLC Server Video #1 stream; another one visual network interface assigns VLAN priority level 4 for VLC Server Video #2 stream.

The Linux Media Client platform is configured as VLC media player client to display the demo videos for verifying TSN IEEE 802.1Qbv & 802.1Qav functionalities. Need to add two visual network interfaces on AXM57104 NIC for two VLC Client video streams, one visual network interface assigns VLAN priority level 2 for VLC Client Video #1 stream; another one visual network interface assigns VLAN priority level 4 for VLC Client Video #2 stream.

The Noise Traffic Generator platform is used to generate the broadcast noise traffic. Need to add a visual network interface on AXM57104 NIC with VLAN priority level 5 for broadcast noise traffic. The priority level 5 of broadcast noise traffic is higher than VLC Video #1 and #2 streams so the VLC Video #1 and #2 streams will be stopped while the Noise Traffic Generator is generating heavy broadcast noise traffic on the same network segment. By enabling TSN IEEE 802.1Qbv or 802.1Qav functions, the VLC Video #1 or #2 streams can play smoothly even though the Noise Traffic Generator is generating heavy broadcast noise traffic in this practice.

The TSN IEEE 802.1Qbv and 802.1Qav standards are Ethernet extensions of IEEE 802.1Q standards. Therefore, the IEEE 802.1Qbv and 802.1Qav networks use 802.1Q VLAN Ethernet packets for industrial communications. The 4-byte IEEE 802.1Q VLAN header includes 16-bit TPID (0x8100), 3-bit Priority (0..7), 1-bit CFI and 12-bit VLAN ID (0..4095). The VLAN packets with higher Priority level have higher priority for data transfer. Designers need to configure proper VLAN priority values based on the pre-defined TSN network configuration.
The following are the procedures to practice TSN functions on Linux platforms by installing AXM57104 PCIe TSN Network card.

Step 1.

Load TSN PCIe NIC Linux Driver & Configure VLAN Priority Level

Install Linux kernel ptp and dsa_core modules firstly before installing AXM57104 PCIe Network Driver; restart “axm57104_service” module and then run “AXM57104_Init.sh” script to configure the network interfaces with proper VLAN priority levels based on the pre-defined VLAN network configuration.

# modprobe ptp
# modprobe dsa_core
# insmod axm57104.ko
# systemctl restart axm57104_service
# /usr/local/bin/AXM57104_Init.sh

For example, the following commands in “AXM57104_Init.sh” script will add two visual network interfaces on enp3s0 network interface, one with VLAN ID 1 and priority level 2; another one with VLAN ID 2 and priority level 4. The visual network interfaces need to be assigned with respective VLAN IDs on Linux kernel system.

# ip link add link enp3s0 name enp3s0.11 type vlan id 1 egress 0:2 1:2 2:2 3:2 4:2 5:2 6:2 7:2
# ip link add link enp3s0 name enp3s0.12 type vlan id 2 egress 0:4 1:4 2:4 3:4 4:4 5:4 6:4 7:4
# ifconfig enp3s0.11 192.168.5.100 up
# ifconfig enp3s0.12 192.168.6.100 up

System timing diagram

Step 2.

Configure gPTP Time Synchronization

The ptp4l program is a popular Linux open source for realizing IEEE 1588 Precise Time Protocol (PTP) and TSN IEEE 802.1AS-Rev Generic Precision Time Protocol (gPTP). The ptp4l program uses TSN gPTP profile to establish gPTP time synchronization by referring to the GrandMaster (GM) clock elected by the Best Master Clock Algorithm (BMCA).

To run the ptp4l command on all Linux Media Server, Linux Media Client and Network Noise Traffic platforms to establish gPTP time synchronization on the TSN-enabled network.

# cd AXM57104_BSP_v100/Application/AXM57104_Ptp4l/AXM57104_Ptp4l_v1.0.0
# ./ptp4l -f /etc/linuxptp/gPTP.conf

After establishing the gPTP time synchronization, the delta time of gPTP time synchronization between these platforms can be measured via AXM57104 PPS (Pulse Per Second) signals, the measured result is around 26 ns in this practice.

System timing diagram

Step 3.

Verify TSN IEEE 802.1Qbv and 802.1Qav Functionalities

After finishing the pre-defined TSN network configuration, designers can run Linux VLC Media Player to verify these TSN functions.

Firstly, run the “send_stream.sh” script on Linux Media Server platform to start VLC media server for sending demo video streams.

# cd AXM57104/AXM57104_BSP_v100/Application/ASIX_demo/VLC_server/
# ./send_stream.sh

And then, run “opera &” on Linux Media Client platform to start AXM57104 TSN Demo console.

# cd AXM57104/AXM57104_BSP_v100/Application/ASIX_demo/opera-12.16-1860.x86_64.linux
# ./opera &

During the “AXM57104 Negative Test”, which IEEE 802.1Qbv is enabled and 802.1Qav is disabled, all Video #1, Video #2 and Noise Traffic streams are allocated in the same time slice so both Video #1 (priority 2) and Video #2 (priority 4) will stop playing video while starting generating the noise traffic (priority 5). This test result is similar to the test scenario on Non-TSN-enabled networks, and it shows the non-deterministic communication disadvantage on Non-TSN-enabled networks.

During the “AXM57104 Time Award Shaper Test (Qbv)”, which IEEE 802.1Qbv is enabled and 802.1Qav is disabled, Video #1 stream is allocated in time slice 1, Video #2 and Noise Traffic streams are allocated in time slice 3.

To start generating noise traffic (priority 5) will stop playing Video #2 (priority 4) in the same time slice, but won’t affect Video #1 (priority 2) playing since the Video #1 is playing in different time slice based on IEEE 802.1Qbv (Time Aware Shaper) standard. This test result shows the TSN deterministic communication benefit by enabling IEEE 802.1Qbv (Time Aware Shaper) function.

During the “AXM57104 Credit Based Shaper Test (Qav)”, which both IEEE 802.1Qbv and 802.1Qav are enabled, Video #1 stream is allocated in time slice 1, Video #2 and Noise Traffic streams are allocated in time slice 3.

To start generating noise traffic (priority 5) won’t affect either Video #1 (priority 2) or Video #2 (priority 4) playing since the Video #1 is playing in different time slot (based on IEEE 802.1Qbv) and the idle slope of noise traffic is set to 5% based on IEEE 802.1Qav (Credit Based Shaper) standard for playing Video #2 smoothly.

Network interfaces with lower idle slope occupy less network bandwidth in the same time slice. This test result clearly shows the TSN deterministic communication benefit by enabling both IEEE 802.1Qbv (Time Aware Shaper) and IEEE 802.1Qav (Credit Based Shaper) functions.

FPGA Hard-Core Field Upgradable

Time Sensitive Networking (TSN) is a set of IEEE 802.1 standards under development by TSN TG. TSN TG continues to develop more advanced TSN standards for new industrial automation applications.

Therefore, the PCIe TSN network card solutions (like ASIX AXM57104) are developed based on FPGA controller to support hard-core field upgradable via In Application Programming (IAP) for TSN standards evolution.

Conclusion

The new generation industrial Ethernet, TSN technology enables real-time, deterministic communications over standard Ethernet for industrial automation applications, and can also converge the non-time-critical IT and time-critical OT networks to meet the requirements for Industry 4.0 smart manufacturing.

The combination of TSN with OPC UA, which provides the key technology to realize Smart Factory and Industrial Internet of Things (IIoT), becomes a rising star of next generation industrial communication technologies.

To enable TSN functions on existing industrial computers, ASIX AXM57104 Quad Port TSN Gigabit Ethernet PCIe NIC Card solution provides an easy way to evaluate TSN functionalities on industrial Linux platforms.

Allan Chou, Director of Marketing, ASIX Electronics Corporation.