Verification and Supervision of communication networks for Utility Automation

Authors: Matthias Wehinger and Fred Steinhauser, OMICRON electronics GmbH, Austria

This already starts in the commissioning phase, where configuration errors and communication problems are ruled out and the correct transmission of all signals has to be verified. Later on, during the operation phase of a digital substation, it is crucial that problems on the communication network are detected immediately, so that the operating personnel can react on it.
The correct functioning of the communication network is an essential precondition for the optimal performance of a PAC system. Consequently, the performance of the communication network needs to be measured and assessed on its own. Depending on the communication architecture and technologies deployed, different approaches are applicable.

Verification of the IEC 61850 Communication
The description of the communication system in the standardized IEC 61850 substation configuration language (SCL) format serves as the basis for the verifications.
It is verified that the IEC 61850 server of all intelligent electronic devices (IED) are available and reachable over client/server (C/S) connection and the substation real-time network traffic (GOOSE and Sampled Values) is actually present on the communication network as it is defined in the configuration file.
A network analyzer tool can verify, proof and document that all protection and control devices are working and communicating properly.

Such verifications are mainly done in factory and site acceptance tests (FAT, SAT), and during the commissioning. In case of a malfunction, the network analyzer tool has to provide detailed information for debugging.
Figure 1 shows the verification of a system with two protection devices and two merging units..
The system verification provides the results for each IED in the system. If an IED is “checked green,” the complete IEC 61850 communication has been found as defined in the configuration files. A warning indicates that there is an issue, which can be related to the server in the IED or that not all Sampled Values streams or GOOSE messages are found on the network as expected. An error is shown if an IED or one of its services is not found during the verification process. Differences in found Sampled Values streams or GOOSE messages are visualized by showing the found values next to the defined ones.

The GOOSE shown in Figure 2 as an example has different values for the Application ID, GOOSE ID, and the Configuration revision. If the values found on the network are the correct ones, the IED configuration file has to be updated, otherwise the device need to be reconfigured accordingly.
A system verification is often an interactive working process because the devices are put in operation one by one during commissioning. The verification steps can be performed incrementally without re-executing all the checks for all devices already verified.

If devices do not perform as desired, detailed information is provided for further investigating and debugging of the problem.
If there are any other GOOSE message or Sampled Values streams found on the network they are listed as “orphans.”  If these orphans are not used in the PAC system anymore they should be eliminated from the network by reconfiguration or removing of the publisher devices, otherwise the SCL configuration files have to be updated accordingly.
After the successful verification of the complete IEC 61850 communication, it is proofed that all devices are available in the PAC system and they are communicating correctly.
The next step is to set up a supervision of the IEC 61850 communication so that any issue during the operation of the PAC system is detected immediately.

Supervision of the IEC 61850 Communication
During the normal operation of a PAC system, it is recommended to supervise the IEC 61850 network communication based on the SCL definition. This is achieved by constantly evaluating all network packets of the Sampled Values streams and GOOSE message in the system.
If Precision Time Synchronization (PTP) is used in the PAC system, it is also important to supervise the PTP communication in the network.
Figure 3 shows a possible setup with a network analyzer which is tapped into a link to supervise the network traffic.

The network analyzer detects the abnormalities in the real-time network traffic and automatically logs all events with the corresponding detailed information (e.g. lost samples, GOOSE timing problems, PTP time synchronization issues…) to a storage device. The event severity and category helps to filter and analyze the entries in the event log.
The analyzer is working autonomously and can be connected in passive TAP mode to the substation network. Thus, it can obtain all traffic on a link without the requirement to configure traffic monitoring features such as port mirroring in the Ethernet switches. Events can trigger the recording of the relevant data for in-depth investigation of the abnormalities occurred. Additionally, notifications can be sent via email to inform the operating staff about the occurrence of an event.
Figure 4 shows the supervisor event list of the network analyzer and the details of a selected GOOSE time to live expired event. In the example the network analyzer has detected that the time allowed to live of a GOOSE message has expired and the repetition of the GOOSE packet was missing. After the GOOSE message was received again, a GOOSE out of sequence event was logged into the event list which provides information about how many repetition packets or even if status changes of the GOOSE message were missing during the timeout period.

Timeouts and out of sequence events of supervised Sampled Values streams are also detected and logged by the network analyzer. If there are any malformed GOOSE or Sampled Values packets on the network, a parsing error event is logged. For configured Sampled Values streams and GOOSE messages which are not received by the network analyzer a never seen event is created in the log. A clock drift between a Sampled Values publisher and the network analyzer is also detected as an event.
Depending on the type of event it can be necessary to immediately react on it or notify the responsible personal. A flexible configuration of different actions for any kind of supervisor event type is required.
After the complete setup the permanently installed network analyzer is constantly supervising the configured IEC 61850 network communication. Beside the reaction with the defined actions it is also possible to check the event log entries via remote connection to the analyzer device.

Protecting your electrical assets? today and tomorrow