SIARA - System Integrity & Restorative Actions

Authors: Priyanka Mohapatra, SP Energy Networks, UK and Hao Guo, Power Networks Demonstration Center, UK

Acknowledgements: Craig McTaggart, David Adam, Graeme Duncan, and Jennifer Kincaid Mackenzie, SP Energy Networks, Prasad Balasubramani, GE Grid Solutions, Kulbhushan Kulbhushan, and Andy Wilson, commtel, and Matthias Wehinger, OMICRON Electronics

Project SIARA (System Integrity and Restorative Actions) is an innovation project led by SP Energy Networks (SPEN) that explores the feasibility of deploying wide area protection and control (WAPC) in SP Transmission (SPT) and SP Distribution (SPD) network using IEC61850-90-5 routable GOOSE (R-GOOSE).  The key objective of project SIARA is to demonstrate feasibility of use of R-GOOSE over SPEN’s wide area network (WAN) and highlight the communication, time-synchronization and other infrastructure requirements for future roll-out of R-GOOSE for WAPC applications across the SPT and SPD network.

Project SIARA proposes a new architecture based on IEC61850-90-5 R-GOOSE for the upgrade of existing and for new SIP schemes as shown on page 38. The future-roll out of this technology will include multiple substations involved in a R-GOOSE based SIP/WAPC scheme. The protection devices in non-central (remote end) substations are denoted as S/S1 to S/Sn and the controller device in the central substation is denoted as C/C1. If there is fault occurring and it is detected by S/S1, the S/S1 will initiate local protection action for example tripping the associated circuit breaker (CB) and then send R-GOOSE to the central controller C/C1. The central controller C/C1 will execute the logic in response to receiving R-GOOSE containing fault related information and subsequently send R-GOOSE command such as closing CB to all related S/Sn for example S/S2.
The offsite tests presented in this article were conducted as part of the project SIARA at Power Networks and Demonstration Centre (PNDC) and aimed to validate the use of R-GOOSE message exchange in Wide Area Network (WAN) that is setup up using packet based MPLS-TP nodes.

Performance Requirement for Communication Service
The GB Energy Network Association (ENA) Technical Specification 48-6-7 “Communication services for teleprotection system” specifies values for steady state propagation latency of 6 ms and steady state asymmetrical latency of 0.4 ms for Category 1 service.
The project SIARA also specifies the latency requirement from the perspective of application. The expected timings of the scheme are summarized in Table 1. During the tests, requirements depicted in Table 1 have been used as the reference.

Testbed Configuration
The hardware-in-the-loop testbed set-up in PNDC to carry out the offsite laboratory testing to prove the objectives of project SIARA is shown in Figure 1 and consisted of real time digital simulator, IEDs with R-GOOSE messaging, PTP time synchronization, network simulation and monitoring equipment.
A transmission line and associated electrical fault and frequency events have been simulated using the simulator as shown in Figure 1 and the response time of an IED could be measured by the simulator. In addition to the trip signal, relays also issued binary output in response to loss of R-GOOSE messages and the test system could monitor the R-GOOSE communication status.

Test Results and Observations


Base Case Test:  The base case test was conducted at a power network steady state with R-GOOSE signals created by use of relay pushbuttons and the following were measured:

  • Network propagation latency
  • Asymmetrical latency
  • Occupied bandwidth

These quantities were derived by network analyzer software using the captured R-GOOSE messages as illustrated in Figure 2. A delay injector introduced delay up to 2 ms with jitter up to 500 µs between Node 2222 and 3333, and the test was repeated for different delay and jitter conditions.

Network Propagation Latency: The average latency of R-GOOSE (packet size between 204 bytes and 216 bytes) going through two XTran nodes (between Node 1111 and 2222 or between Node 2222 and 3333) was about 69 µs with jitter less than 0.6 µs when there was no delay injector connected in the network. The delay injector introduced extra 20 µs latency when it was physically connected in the network but did not inject any delay. The total average latency going through two XTran nodes (between Node 2222 and 3333) thus increased to 89 µs.

Asymmetrical Latency:  The asymmetrical latency of R-GOOSE (packet size between 204 bytes and 216 bytes) going through two XTran nodes (between Node 1111 and 2222 or between Node 2222 and 3333) was derived by comparing the average latency of both directions between two XTran nodes, for example from Node 1111 to 2222 and from Node 2222 to 1111.
When no delay injector was used in the network, the asymmetrical latency was less than 0.65 µs. When the delay injector was physically connected in the network (between Node 2222 and 3333) but did not generate any delay, the asymmetrical latency (between Node 2222 and 3333) increased to 0.9 µs. The maximum asymmetrical latency was observed when 2 ms delay with 500 µs was injected and it was 70 µs (between Node 2222 and 3333). 

Bandwidth: The maximum bandwidth occupied by a single R-GOOSE stream (packet size between 204 bytes and 216 bytes and was retransmitted 3 times within 0.1 s) for example the R-GOOSE from N60_A to C30, was less than 10 kB/s or 80 kb/s. Because C30 subscribed to R-GOOSE from both N60_A and N60_B, the maximum bandwidth seen by C30 was double and became 20 kB/s or 160 kb/s.

Event Response Time Test:  The testbed arrangement for event response time test was similar to the base case test besides that power events such as faults and frequency disturbances were introduced.  In the event response time test, the R-GOOSE messages were generated by relays’ protection operation upon power system events.
The event response times under different network conditions are as described below:

  • For fault events, the local tripping time on N60 was about 22.65 ms and the remote tripping time on C30 was 24.44 ms
  • For under-frequency event, the local tripping time on N60 was either 92.7 ms or 112.8 ms and the remote tripping time on C30 was either 94.4 ms or 114.5 ms
  • For over-frequency event, the local tripping time on N60 was either 97.8 ms or 117.8 ms and the remote tripping time on C30 was either 99.4 ms or 119.3 ms

The results indicated there might be 1 cycle difference (i.e. 20 ms) in the tripping time for frequency events. A possible reason was that the insertion point of frequency disturbance was uncertain, for example it could occur in the positive zone or negative zone of a sine wave as shown in Figure 3 and the relays might need more time to calculate the frequency if the disturbance occurred in the negative part of a sine wave.
The maximum bandwidth occupied by N60 and C30 during the power system events was less than 20 kB/s or 160 kb/s.

BER Test:  During the BER test, power system events such as faults were continuously injected so that R-GOOSE messages were generated continually. The impairment injector subsequently introduced bit error to the network and the event response time, correctness of relay response and communication path status were monitored.
BER = 10-6
In this circumstance, the local trip time on N60 and the remote trip time on C30 were 22.8 ms and 24.4 ms, respectively. The GE relays tripped correctly for all fault events and there was no communication alarm on the relays or the XTran node. The bandwidth among all relays was less than 20 kB/s or 160 kb/s.
BER = 10-5

In this scenario, all GE relays rebooted and there was alarm “SYSTEM EXCEPTION” on the relays, which meant “abnormal restart from modules being removed or inserted whilst the relay is powered-up.” However, there was no alarm on the XTran node. It could be observed from Figure 4 that the bandwidth between relays and the XTran nodes significantly increased to more than 10 MB/s or 80 Mb/s.  Figure 5 presents the captured packets by Omicron DANEO 400 connected between XTran node 1111 and GE N60_A and it is worth noting that the PIMv2 Bootstrap packets were flooding the link between XTran node 1111 and GE N60_A. The flooding of PIMv2 Bootstrap was regarded as a Denial of Service (DoS) attack by the Ethernet module of GE relays and the module rebooted.

Network Switching Test:  The testbed arrangement for network switching included a ping machine connected to the DANEO 400 and then to the XTran node 1111 and N60_A was directly connected to XTran node 1111.  The Ping machine could send ping messages to C30 and N60_B every 1 ms and a link failure was introduced between node 1111 and 2222.  he network would swap from main path (XTran node 1111-> 2222-> 3333-> 4444) to the backup path (XTran node 1111-> 4444-> 3333-> 2222) if path protection was activated. The associated network switchover time could be derived by looking at how many Ping messages were lost.
The network was also configured in a way that when the broken link recovered, the network would switch from the backup path back to the main path. This network recovery time was also measured. The delay injector was connected between node 3333 and 4444 to introduced various jitter with no power system events applied.
The network switchover time from main path to backup path was between 19 ms and 39 ms regardless of types of link failure (single or dual fiber cut) and size of delay jitter. The network switchover time from backup path to main path was between 6 ms and 14 ms regardless of types of link failure (single or dual fiber recover) and size of delay jitter.
There was no communication alarm or trip alarm captured during the network switchover.

Relay Synchronization Loss Test: In this test, the links feeding PTP signal to GE relays were removed and restored, and the stability of relays was recorded. The delay injector was used to introduce delay up to 2 ms with jitter up to 500 µs. No power system event was applied during the test.  In case of PTP Synchronization Loss there was alarm on relays showing “Bad PTP Signal.”
However, there was no trip or “RxGOOSE OFFLINE” alarms on the relays.
In case of PTP Synchronization Restoration the alarm “Bad PTP Signal” disappeared and there was no trip or “RxGOOSE OFFLINE” alarms on the relays. 

Incident Test:  In the incident test the applied incidents were removal of one of the redundant supplies on XTran node 1111 and removal of the single supply on XTran node 2222.  The latter would emulate a communication node failure event and the communication path was expected to swap from the main path to the backup path.  A Ping machine was used again to monitor the lost Ping packets and the delay injector was connected between XTran node 3333 and 4444 to insert jitter.

Removal of One of Redundant Supply on Node 1111:  There was an alarm “Error PSU1 Input” on XTran node 1111 but no Ping packet loss and no alarm on the GE relays were observed.  The network was still on the main path (XTran node 1111-> 2222 -> 3333 -> 4444).

Restoration of Redundant Supply on Node 1111:  Alarm “Error PSU1 Input” on XTran node 1111 disappeared and there was no Ping packet loss and no alarm on the GE relays.  The network was still on the main path (XTran node 1111 ->2222-> 3333-> 4444).

Node 2222 Failure:  The network switchover time from main path to backup path was between 18 and 31 ms regardless of the size of delay jitter. Alarms “RxGOOSE OFFLINE” and “COMM Path Incomplete” were observed from all GE relays because communication between N60_A and C30 and between N60_B and C30 were lost and the relays could not receive the subscribed R-GOOSE messages. 

Node 2222 Restoration: The network switchover time from backup to main path was between 3 and 29 ms regardless of the size of delay jitter. The time taken for XTran node 2222 to fully operate after supply restoration was between 103 s to 116 s. Alarms “RxGOOSE OFFLINE” and “COMM Path Incomplete” on GE relays disappeared.  

Analogue R-GOOSE Test:  In the analogue R-GOOSE test N60_A and N60_B were configured to generate analogue R-GOOSE messages containing 6 quantities including 3-phase current magnitude and phase angle, and C30 subscribed to both analogue R-GOOSE streams, as shown in Figure 6. The deadband of the current magnitude and phase angle was configured as the smallest so that the analogue R-GOOSE were updated and transmitted every 0.1 s. For each transmission, the same R-GOOSE was retransmitted for 3 times. Power system faults were applied to check if the analogue R-GOOSE could update the current magnitude quickly afterwards. There was no delay injector used during this test.

The average latency of analogue R-GOOSE (packet size between 237 bytes and 239 bytes) going through two XTran nodes (between Node 1111 and 2222 or between Node 2222 and 3333) was about 71 µs with jitter less than 0.7 µs when there was no delay injector connected in the network, as shown by the DANEO 400 analysis results in Figure 7.

The maximum bandwidth occupied by a single analogue R-GOOSE stream (packet size between 237 bytes and 239 bytes and was retransmitted 3 times within 0.1s) for example the analogue R-GOOSE from N60_A to C30, was about 10 kB/s or 80 kb/s. Because C30 subscribed to analogue R-GOOSE from both N60_A and N60_B, the maximum bandwidth seen by C30 was double and became 20 kB/s or 160 kb/s. It should be noted that when there was a power system fault, N60_A and N60_B would also transmit binary R-GOOSE in parallel to the analogue R-GOOSE. Therefore, the peak bandwidth was increased: 20 kB/s or 160 kb/s for N60_A and N60_B, and 40 kB/s or 320 kb/s for C30.
The value of the 6 quantities (3-phase current magnitude and phase angles) in analogue R-GOOSE only updated every 0.1 seconds. Assume a fault just occurred after the periodic analogue R-GOOSE packets were sent out from N60, the remote C30 would need to wait up to 0.1 seconds to know the current magnitude had changed, which might not be ideal for protection requiring instantaneous measurement.

Conclusion and Next Steps

The offsite testing of project SIARA proved that the OTN Systems MPLS-TP communication devices (XT-2210) could carry the R-GOOSE messages generated by GE C30 and N60 protection devices. These R-GOOSE messages were used for remote tripping and analogue signal exchange in a hardware in loop set-up. The R-GOOSE messages generated during the test were of size between 209 bytes and 239 bytes. When there was no other background traffic and no artificial delay, the network propagation latency and asymmetrical latency of R-GOOSE messages going through two XTran nodes was up to 70 µs and 2.5 µs. These values satisfied the requirement of ENA TS 48-6-7 where the propagation latency should not exceed 6 ms and the asymmetrical latency should not be greater than 400 µs. The bandwidth occupied by a single R-GOOSE stream (packet size between 209 bytes and 239 bytes) was about 80 kb/s. Analogue R-GOOSE was configured so that it would be continuously transmitted. Hence, if analogue R-GOOSE were transmitted from both N60_A and N60_B and a power system fault event was applied which could trigger binary R-GOOSE from both N60_A and N60_B, the maximum bandwidth among N60_A, C30 and N60_B would be 320 kb/s. 

The instantaneous over-current (IOC) protection, under-frequency protection and over-frequency protection were used on GE N60. The associated local tripping time (when there was no other background traffic and no artificial delay) were:

  • Local IOC protection for fault: about 22.65 ms
  • Local under-frequency protection for under-frequency disturbance: either 92.7 ms or 112.8 ms
  • Local over-frequency protection for over-frequency disturbance: either 97.8 ms or 117.8 ms

The analogue R-GOOSE from N60 could only update the value of analogue quantities every 0.1 s, which might not be suitable for protection scheme requiring instantaneous measurement. The tests provided valuable insight regarding bandwidth, time synchronization requirements, latency, and different protection schemes overall operational times using R-GOOSE over wide area network.
There were some limitations to the offline testing that did not take into account the cyber security requirement on the wide area network and this test needs to be performed with appropriate encryption on the network. This may or may not have implications on the latency and the overall operational times of the R-GOOSE based WPAC schemes provided adequate bandwidth requirements are considered during the deployment.

The offline testing has proved SPEN enough confidence to trial R-GOOSE based WPAC schemes on the SPT network. There are 2 proposals currently being considered for the first trial of R-GOOSE. The first step will be to deploy the offline arrangement at 3 nodes of the network and repeat the tests over the SP Transmission wide area network. Potential future deployment of the technology is on projects where multi-site communication and control is a key requirement, one such project that is currently being developed by SPEN is the Generator Export Management System (GEMS). GEMS aims to monitor and manage generation output across south west Scotland according to a System Operator led commercial constraint market by maximizing the utilization of available transmission capacity, whilst ensuring security of supply is maintained. The GEMS project is currently considering R-GOOSE as the main communication protocol, providing means for fast, reliable and secure data exchange over a dual redundant ring wide area network architecture. The final decision regarding use of R-GOOSE in the GEMS project will be dependent on a cost benefit analysis and a detailed risk assessment considering the time scales for deployment, cyber-security, the availability of suitable IEDs and communication network infrastructure.

SIARA through its offsite testing has already alleviated some of the risks and concerns and we remain positive regarding the roll-out of this technology on our network.
Our offsite testing experience shows that IEC61850-90-5 based R-GOOSE is a viable technology for future system integrity and restorative action schemes. The protection IEDs and testing tools supplier base compatible with R-GOOSE currently is limited and this could potentially affect roll-out of this technology on a wider scale. There are significant benefits of using R-GOOSE over WAN in terms of reduced number of IEDs used for protocol conversions and cyber security provided the right level of encryption is applied.
There is a huge potential for application of R-GOOSE in future utility WPAC schemes provided the right level of reliability, availability and security can be ensured over the wide area network.

Biographies

Priyanka Mohapatra works as RIIO T2 Innovation Lead at SP Energy Networks and is the technical lead for project SIARA. She is involved in the conceptualization of several innovation projects enhancing grid observability, controllability and standardization of digital technologies. Prior to joining SPEN, she worked with Siemens AG for 8 years. She started at Siemens Ltd., India as an electrical engineer before moving to Siemens AG Global R&D, Germany as a software developer and project manager for SCADA, EMS, DMS systems. She then worked with Siemens Protection Devices Ltd., UK as a product owner and senior engineer.

Hao Guo received the B.Eng. degree and Ph.D degree in Electrical and Electronic Engineering from the University of Manchester, UK. He is currently a smart grid research engineer in Power Networks Demonstration Centre, University of Strathclyde, UK. He has collaborated with manufacturers and utilities and worked on a number of smart grid projects. He is a member of IET and has good knowledge and experience in Protection, Control and Communication within power substations.