Network Solutions and their Usability in Substation Applications

Author: Clemens Hoga, Siemens AG, Germany

Substation environment

In substations, IEC 61850 communications based on Ethernet networking are state of the art today. Four types of communications take place on such networks:

  • Client – Server based on TCP/IP MMS (connection oriented)
  • Basic services like NTP, SNMP, HTML (non time critical)
  • GOOSE directly on Layer 2   (multicast, repetition mechanism)
  • Sampled Values directly on Layer 2 (multicast, data stream)

In today’s substations the Process Bus Application has not yet been greatly evidenced in projects. It is expected to be in hot projects in the beginning/mid of the 2010 decade. Nevertheless, redundancy is a big topic for the IEC 61850 station bus as well. One critical parameter is the recovery time of a redundant system. This means the time between the occurrence of the N-1 failure and the moment when the network has fully recovered. This has to be considered together with the necessity that the substation application needs the network exactly at this moment (e.g. Trip that message over the network).  A short, very simplified analysis assumes the following parameters:

  • The network recovers 1 time a year; 
  • Time critical situations like CB-Trip appear in a substation 50 times a year; 
  • Recovery time is 100 ms, 
  • Linear calculation:
    365 x 24 x 60 x 60 x 10 = 315360000/50 = 6.3 x106

The Probability that the network is down during such a trip situation is 1: 6.3 Million per year.

Applications from a Substation Controller to an IED using the Client/Server services are not time critical. TCP/IP mechanisms care for repeating lost frames and the right ordering in the receive buffers. Applications between IEDs (e.g. Interlocking Signals and Trip messages) use the GOOSE service based on a multicast service. Through a repetition mode defined in IEC 61850, it is ensured that these messages do not get lost. Therefore, the communication blackout during the recovery time does not mean that the messages sent out during this period are lost. Even when the double signal change is short enough that it is missed because at the end of the recovery period the signal has the same state as at the beginning, the application is able to recognize the uncertain state by checking the GOOSE message counter which is incremented with each GOOSE repetition.

The use of sampled values is a different issue regarding application. Even if one sample is missing, the protection relay has a measuring blackout of a measuring window.

Possibilities in real Substation layouts

Ring Redundancy: Not all of the IEC 62439 redundancy mechanisms are used in IEC 61850 applications; the most common ones are reviewed.

The Issue of ring redundancy: In principle, an Ethernet System MUST NOT be configured as a real ring. Due to its network access mechanisms, no data frame is allowed to circle around the network. In case of a closed loop / closed ring all connected devices will pump frames into the system but these frames will never disappear in closed rings. Depending on the number of frames per second of the connected devices, even GBit/s systems will crash in seconds. Therefore, measures have to be taken to prevent circulating frames in these systems (loop-prevention). There are different, standardized systems available which prevent circulating frames even in a physical ring topology. The most common systems are described.

MRP (IEC 62439-2,“Hyper Ring”): MRP is the standardized version of vendor specific ring redundancy solutions like HiPER-Ring or High-Speed-Redundancy (Fig 1).

Function: One of the Ethernet Switches is a so called “Redundancy Manager”. It sends out test frames on both ring ports. In a ring without a failure, each of these test frames must appear on the opposite ring port of the Redundancy Manager. As long as these test frames appear, the redundancy manager opens the loop and the circulation of frames is prevented (Fig 2). If the ring is interrupted, either because a connection is broken or a switch in the ring is defective, no test frames appear on the receiving side of the redundancy manager. The redundancy manager closes the loop and all devices are connected to each other again. Coupling of rings is not standardized in MRP (switch vendors use proprietary dual link solutions.)

RSTP (IEEE 802.1w/802.1d-2004, IEC 62439) Ring and meshed configurations: RSTP can provide pure ring configurations as shown in MRP and meshed configurations as well. Consequently, the loop prevention principle must be different to pure ring systems. One of the Ethernet switches is the so called root bridge (Simplified: bridge = switch). This is the bridge with the highest so called “root priority”. All ports at this switch are designated ports (Fig. 3).

Ports closest to the root bridge are “root forwarding”. The path with the closest connection to the root is active. Non necessary ports are blocked for loop prevention. If the network configuration is able to, every path has a pre-configured, alternative path; blocked ports can become active when the primary path is defective. In the case of root bridge failure, the bridge with the next higher root priority takes over the root bridge function (Fig. 4). RSTP can be used in ring configurations as well. In pure rings it shows good performance of recovery speed 4-5 ms per hop, multiple meshed systems can lead to larger recovery times. This mechanism utilizes implementation of small switches in IEDs like protection relays directly. Cost effective configurations can be achieved, such as the one shown in figure 6.

The IEDs may have integrated switches which are RSTP-aware. Multiple rings are possible. One of the multiport switches is the root switch which organizes the optimal communication paths by establishing Root ports and designated ports. At the same time alternative redundant paths are foreseen, but blocked in normal operation. In the case of n-1 failure the alternate path will be activated. RSTP has settable parameters such as aging time, root priority … Using recommended, pre-configured parameters in multiple meshed configurations can end up in recovery times of 1 second but optimized parameters allow smaller re-configuration times.  

Dual homing (dual link) redundancy: In a dual homing configuration, the two interfaces in an IED and in a substation controller have two interfaces. One is active; the other is actively monitoring the backup link if it is still usable (Fig. 5a). In the case of an n-1 failure the IED checks the missing link and switches over to the reserve link. It sends out a special message in order to establish the alternative path. This establishment is reduced to the missing link only; therefore the recovery time is very fast (Fig. 5b). This type of redundancy is described in principle in IEEE 802.1d but is often implemented with some proprietary functionality.

Mixed Configurations: Dual homing and ring configurations can easily be mixed. The most typical configuration is as follows (Fig. 7). The Main Ring and the sub rings are using RSTP or MRP, the IEDs are dual connected by use of link redundancy. This kind of mixed technologies provides true n-1 redundancy with very low deterministic recovery times based on the available and proved technologies. Due to the fact that both technologies work independently, the recovery times do not add.

Relion advanced protection & control.
Protecting your electrical assets? today and tomorrow