Wireless Communication Systems Supporting IEC 61850 Applications

Authors: Andre Smit and Suraj Chanda, Siemens Industry, Inc., USA, and Terry Fix, Dominion Energy, Richmond, USA

Cellular Communication

Cellular communication has become a new and important platform that can be leveraged to transport P&C data across a provider’s cellular network.  This might sound like a foreign concept but is actually nothing new. Many DTT systems today rely on leased telephone lines form 3rd party providers. These aging public wired networks are becoming unreliable and cellular networks are a modern reliable alternative. 

The major driver to use a cellular system is the ease of deployment. There is no need to find or install tower assets. The cellular providers have already installed extensive tower assets making cellular service widely available, similar to the old copper telephone line networks we use.

Today cellular providers offer private networks to companies. These private networks provide a system where only the company’s devices are connected to a network.

Cellular private network:  These private networks are not connected to the Internet. No unsolicited traffic from the Internet will get onto such a network. A utility can create private networks for different applications: AMI network for all meters, a Direct Transfer Trip network for DTT devices, a POTT network for protection devices, a FLISR network for automation devices to name a few. (see Figure 5).

These networks can easily be leveraged to transport GOOSE messages between protection devices.  VLAN based L2TPV3 tunnels are configured between modems to communicate the IEC61850 “GOOSE” messages. The tunnel configuration settings include:

  • VLAN number
  • Session parameters
  • Local and remote ports
  • Local and remote static IP addresses

Security

L2TPV3 Tunnels with IPSec Encryption: The GOOSE PDU (Protocol Data Unit) doesn’t include an IP header in order to avoid additional latency caused during processing of the TCP/IP layer. This was done specifically to ensure extremely efficient data processing in the embedded devices such as protection and control relays. Therefore “GOOSE” messages cannot propagate over layer 3 networks without being tunneled. (A tunneling protocol encapsulates another protocol’s data and transmits them over an unsupported network through a tunnel. At the other end of the tunnel the data is then decapsulated before it goes out). The Layer 2 Tunneling Protocol (L2TPV3) is used to transmit specific VLAN tagged GOOSE messages over the cellular network. Internet Protocol Security (IPsec) is used for securing tunnel communication by authenticating and encrypting all the tunnel data.

Typically, the same GOOSE message is retransmitted with varying and increasing retransmission intervals until a change occurs in any GOOSE dataset element. L2TPV3 supports this retransmission ensuring the GOOSE message with a given state number is delivered at the tunnel end.

IPsec is one of the key security features implemented in the cellular modem to provide immunity to cyber-attacks. IPsec is configured to encrypt/decrypt any data entering/leaving the tunnels respectively form the modems. IPsec features include:

  • Data encryption
  • Modems verify pre-shared keys

DTT network architecture:  As shown in Figure 5, the entire DTT system is part of a cellular private network pool. All the cellular modems and automation controllers are part of a unique IP subnet which has layer 3 isolation from other networks.  The network parameters for the devices within the private network are managed as follows:

Cellular Modem

  • IP Addressing:
    • Static IP Addresses - Assigned for SIM cards in the plan. Modems learn these static IPs upon successful connection to the network
    • LAN IP Addresses - Assigned based on the network addressing for the automation controllers
    • Management IP Address ? Local addresses assigned in a subnet completely isolated from Static and LAN addresses
  • VLAN addressing and Port Setup:
    • LAN port which is connected to the controller is configured as VLAN edge port
    • Management port is on default VLAN and other ports are disabled for security
  • Automation Controller:
    • IP Addresses assigned on a subnet different from cellular subnet and management subnet
    • Unique VLAN assigned for each DTT system which is different from other automation systems to separate IEC61850 based GOOSE messages

Penetration Testing: A penetration test on a GOOSE based DTT application using a RX1400 cellular modem was performed to verify the security of the link.

The results showed that the private cellular network and IPsec tunnel network are largely resilient against adversarial attack. The cellular modem’s cyber security features were validated and demonstrated it can withstand the majority of threats.

The DTT system design incorporates an innovative communication approach.  The DTT system is designed to use Fiber Optic network communication with redundant Cellular communication. The DTT controller will accept the two separate sets of DTT GOOSE signals from the two communication systems. The DTT sending devices will send separate DTT GOOSE over the two communication networks.  The controller at the PCC will react on the first DTT GOOSE received from either communication network.

Redundant GOOSE Communication Systems: A communication failure at any point of the DTT system on either communication system will have little or no effect on the system. This system can be deployed in different combinations of Cellular + Fiber, Cellular + WiMAX or Cellular + Cellular from different providers.  The digital wireless cellular system proved to be more reliable than the existing copper leased line networks on a pilot project deployed in the US. (see the Figure on the first page).

Recloser with Polarized Directional Antenna: In the near term it is possible to improve the existing DTT system reliability and selectivity by the deployment of the new GOOSE based system over cellular and improve the system’s reliability and speed with the deployment of fiber in the longer term (Figure 6).

A communication based DTT system is definitely the desired and more conservative protection approach to ensure that a DG site will be disconnected if the connected feeder system experiences an abnormality that requires maintenance or repair.

The Cellular System Deployment - The ease of deployment of a cellular system in the field is a major differentiator. The communication system's site availability can easily be verified using a cellular phone. A person can physically drive to each location and check if he can exchange data on his smart phone. An LTE signal is preferable. If signals are weak, the signals can be verified using a DTT cellular modem/router with a directional antenna. We built a simulation test DTT system consisting of a PCC controller and modem located in a laboratory and a mobile substation DTT unit with modem that can send DTT signals to the PCC unit in the lab over a private cellular network.

Mobile DTT Test System: With this method the entire DTT communication system could be verified in minutes at each location. In a pilot the user verified cellular communication to 22 sites in two days over a two-state geographic area using this method.

Cellular DTT Installation at Solar Site

The Cellular System Performance - Latency can be defined as the time it takes for a source to send a packet of data to a receiver. It is typically measured in milliseconds. Lowering the peer to peer latency results in better network performance (Figure 7). There are 2 latencies to consider:

  • Cellular Modem - Modem Latency: In the cellular terms, this is also referred as User Plane (U-Plane) Latency. (LTE for UMTS) U-Plane latency is defined as one-way transmit time between a packet being available at the IP layer in one modem and the availability of this packet at the IP layer in the other modem. U-Plane latency is relevant for the performance of many applications
  • Controller - Controller Latency: This latency is one-way transmit time of a packet from source controller to destination controller. This latency includes cellular latency, tunneling and encryption latency (at both the modems)

Since the LTE deployment a few years ago, LTE networks promised the user plane latency to be approximately 1/2 of corresponding latency in existing 3G technologies. This provides a direct service advantage for highly immersive and time critical applications like FLISR, and DTT. The published latency comparison of Verizon on different networks is as shown in Table1. From the field test results, the end to end controller latency is between 120 ms to 300 ms.

This is dependent on number of subscribers connected to the cell tower, number of resources allocated to each cell server and other factors. With the advent of 5G and Verizon upgrading their network to support LTE, the cellular latencies are expected to significantly reduce in the future.

Relion advanced protection & control.
Protecting your electrical assets? today and tomorrow