by Jan Mazan, JPembedded, Poland
This article describes one of the possible approaches to the implementation of gateway converting regular Ethernet based GOOSE messages to routable (R) version of this protocol. GOOSE which stands for Generic Object Oriented Substation Event, is one of the protocols introduced by the IEC 61850 standard released in 2004. Its primary purpose is to notify peers about different protection events generated at intelligent electronic device (IED) e.g. protection relay. In the first edition of the standard, it was specified as a protocol implemented directly on top of the link layer – Ethernet.
There are two types of actors in GOOSE communications:
Publisher sending multicast frames on the local network
Subscriber which ‘listens’ to selected types of frames generated by other nodes (publishers)
As a solution for horizontal, peer-to-peer communication, GOOSE over Ethernet seemed to be a good choice, especially considering the requirement of low latency for this type of traffic. Sending out multi-cast Ethernet frames on event driven or periodic basis involves very low overhead and satisfies timing requirement of low delay between the time at which an event occurred and generation of the message with the information about it.
One obvious limitation of this approach is that such communication is limited to the local network (LAN) and with the introduction of DER and evolution of the grid towards distributed architecture, there was a need for a routable version of GOOSE protocol. This implicated the question of the cyber security of the very sensitive information which is sent by IEDs belonging to the critical infrastructure.
In the following sections I describe the solution which does not require major updates to the existing equipment installed in the field, offers great deal of flexibility and is relatively simple to deploy.
Acronyms used
AES – Advanced Encryption Standard
AES-CBC – AES cipher block chaining
AES-GMC – AES Galois counter mode
APDU – Application protocol data unit
APPID – Application identifier
CRL – Certificate revocation list
DER – Distributed energy resource
DERMS – Distributed energy resources management system
DUT – Device under test
GDOI -Ggroup domain of interpretation
GOOSE – Generic object oriented substation event
HMAC – Hash based message authentication code
IED – Intelligent electronic device
IKE – Internet key exchange
ISAKMP – Internet security association and key management protocol
IP – Internet protocol
KDC – Key distribution center
OCSP – Online certificate status protocol
RBAC – Role based access control
R-GOOSE – Routable GOOSE
SMV – Sampled values
UDP – User datagram protocol
Solution
There are several options for sending GOOSE messages over IP networks, like tunneling of layer 2 (L2) GOOSE over UDP/IP or by implementing support for routable version of GOOSE on the IED. The approach described by this article is quite simple to implement and it does not require any changes on the side of existing GOOSE publishers or subscribers. This can be done with a gateway which on one hand is a subscriber of the ‘traditional’ L2 GOOSE messages published by the IEDs connected to the local network, but on the other hand publishes received messages as an UDP/IP multi-cast traffic.
The whole idea is depicted in Figure 1.
The main tasks of the gateway are:
- Subscribe to the selected L2 GOOSE messages using one of the parameters: GOOSE message ID or name of publisher control block
- Generate UDP multicast packet containing GOOSE APDU from the received L2 GOOSE messages
- Optionally encrypt payload of the outgoing R-GOOSE multi-cast packet
For easier understanding of the whole process Figure 2 shows how the stack of L2 GOOSE maps to the protocols of the routable side.
The frame encoding details for L2 and UDP GOOSE messages are specified in IEC 61850-8.1 and IEC62351-9 respectively.
The gateway itself can act as an IEC 61850 server that subscribes to multiple L2 GOOSE published by different IEDs within the local network. For each subscribed GOOSE, as a bare minimum, there should be defined destination UDP multicast address to which GOOSE will be sent.
Depending on the need of specific applications it is also possible to update some parameters of the published R-GOOSE like application ID number, GOOSE ID string or customize the delay between re-sent messages.
As for the configuration of the gateway, it could be done statically via xml file, in which case once the xml config file is uploaded to the gateway, subscribers and publishers are initialized and could be either enabled or disabled by the IEC 61850 client application, but the parameters of the subscriber (L2) side or publisher (routable) side, do not change until the new configuration is uploaded and instance of the gateway is restarted.
If required, it is also possible to modify during run time some parameters of the gateway:
- Subscription attributes
- Destination multicast addresses
- KDC server address
It depends on the requirements of the specific application and the customer using the gateway.
Challenges
While the overall idea might seem very simple (and actually it is), as the saying goes, the devil is in the details. Putting together all the puzzles requires deployment of the right technology, tools and skills, especially if the goal is to have a flexible, scalable and salable product. In the following paragraphs I’ll scratch the surface a bit and briefly touch on the issues related to cybersecurity and performance, however this is by no means an exhaustive coverage of the matter.
Encryption
Since GOOSE messages published as UDP/IP multicast potentially will be sent over wide area networks it is extremely important to protect the information against unauthorized access and modification. Considering the fact that a GOOSE message could contain some sensitive information about the state of the critical infrastructure, this is good enough reason not to cut corners when it comes to cybersecurity. An obvious approach is to implement well know encryption standards like AES-CBC or AES-GMC. The details of how GOOSE data units (APDU) are encrypted are outside the scope of this article and could be found in IEC 62351-9. AES is a symmetric encryption standard which provides a sufficient level of security, it is not difficult to implement in software and thus is very popular and widely available. Some hardware platforms offer built-in (hardware) acceleration of the AES implementation which significantly increases the speed of encryption/decryption operation. Since this is symmetrical algorithm, both sender and receiver of the message must share the key, and secure distribution of the encryption keys is a non-trivial task. Also, the overhead related to crystallographic operations must be considered, when designing and implementing the gateway which shall serve several GOOSE ‘mapping paths’. Both distribution of encryption keys and issues related to performance are described below.
Key Distribution
The simplest way for distributing encryption/decryption keys among IEDs is to configure them statically on the device. It does not take a rocket scientist to figure out that there are many more cons of this approach than pros. However, one obvious advantage of this method is that it is simple to implement and thus could be used for prototyping and demonstration purposes.
For real life application however, a secure and automated way of key distribution must be used. The task is not trivial because of the nature of the GOOSE communication, involving multicast groups with potentially many nodes publishing or subscribing, which means that keys must be updated for all group participants in a coordinated way. The other challenge is that the cryptographic material must be delivered in a secure way, meaning that only authorized members of the group might receive it, and even if others intercept the communication, it should be impossible for them to determine the keys or temper with them. All these issues are solved by group domain of interpretation (GDOI) protocol which is based on IKE/ISAKMP. In short, each group member establishes secure communication channel with key distribution center (KDC), over which encryption/decryption keys for given group are provided. Normally KCD would periodically update the keys by sending to all active group members encrypted multicast PUSH messages. The details of the whole process are described in IEC 62351-9 specification and number of RFCs: 6407, 8263.
Garibaldi tool is an implementation of KDC which handles routable GOOSE and routable sampled value. The product features a web page interface which makes it easy to configure and operate. It is scalable in terms of number of GOOSE / sampled value streams it can handle and implements role-based access control (RBAC).
Certificate Handling
Another area which must be addressed when implementing any solution involving cyber security is handling of certificates. Currently certificates serve different purposes like
- Authentication
- Access control (RBAC)
- Identity management
- Asymmetrical encryption for session (symmetrical) key negotiation
In the context of GDOI certificates are used when exchanging session encryption keys between the KDC and group members to authenticate each side and generating message signatures.
The most popular form of digital certificate is X.509 which contains, among others, details of the entity for which certificate was issued, public key and information about certificate issuer. Certificates are issued for a specific time frame, and must be replaced once that time expires or if they are revoked by the issuer, e.g. if the private key is compromised. The challenge for the distributed system is to configure certificates of the nodes and manage their updates when necessary (expiry or revocation). Handling of revoked certificates could be implemented with certificate revocation lists (CRL) or Online Certificate Status Protocol (OCSP). There is no silver bullet here, each approach has its advantages and selection of the right one depends on the requirements of the specific application.
Performance
Since GOOSE (and sampled value) is used for delivery of time sensitive data, implementation of L2-GOOSE / R-GOOSE must take into account additional latency introduced by:
- Routing and switching done by the IP nodes
- Encryption / decryption operations
- IP routing
Usually, the customer asking about GOOSE performance, expects that the delay of GOOSE is below 4 milisec. Unfortunately, not many of them realize what it does exactly mean and how to measure such delay.
For the sake of demonstration of performance of some IEC 61850 solution (module or communication) configuration consisting of:
- Laptop running Wireshark application and GOOSE publisher
- Device with IEC 61850 server and GOOSE subscriber/publisher could be used.
The idea is to send GOOSE messages from the laptop to IEC 61850 device under test (DUT) and setup sample ‘loopback’ application which would receive GOOSE sent by the laptop and immediately trigger generation of its ‘response’ GOOSE, e.g. by updating value of some attribute. After GOOSE sent by DUT is received by the laptop, Whireshark can provide delay between GOOSE sent by the laptop and the response which is coarse and very conservative measure of generation of GOOSE by DUT in response to some event. (Figure 3).
For L2 GOOSE / R-GOOSE gateway situation is a bit more complicated because the assumption is that it has to encrypt the message, so there must be another GOOSE subscriber which decrypts the message. For the worst-case scenario let’s assume that decryption of the GOOSE message is performed by another gateway device (DUT2) and decrypted message is distributed locally as a L2 GOOSE. The sequence diagram in Figure 4 shows two variants of end-to-end L2 / R GOOSE conversion, with and without encryption. (see Figure 4).
You have to be aware that there is also a nondeterministic factor related to the transition of R-GOOSE over IP network, related to the very nature of this communication. For the sake of simplicity, let’s assume that all the devices from Figure 4 are physically connected to a single switch. The assumption is justified since the goal is to assess the impact of the introduction of two gateways (DUT1 / DUT2) which convert between L2 / R GOOSE and implement encryption/decryption.
The tests configuration used for the evaluation was like this:
- Windows 10 Laptop running L2 GOOSE publisher and Wireshark application
- Raspberry Pi 3+ (DUT1)
- Raspberry Pi 3+ (DUT2)
- Fast Ethernet (100 MB) switch
Table 1 shows the latency statistics for different scenarios:
a) L2 GOOSE sent to DUT1 un-encrypted R-GOOSE sent back to Laptop
b) L2 GOOSE sent to DUT1 encrypted R-GOOSE sent back to Laptop
c) L2 GOOSE sent to DUT1, encrypted R-GOOSE sent to DUT2, un-encrypted L2 GOOSE sent to Laptop
The measurements in Table 1 should give a rough idea about the overhead introduced by encryption and decryption operations. We have to remember that the test was executed on raspberry-Pi 3+ device which does not offer any hardware acceleration for cryptography. However, for different architectures there are some options (e.g. AES-NI extension for x86, or ARMv8) offering significant performance improvement which should result in lower processing time.
Also, the round trip ‘Max time’ measured on laptop (e.g. 9.033 milliseconds for case (b)) is noticeably higher than ‘Max time’ measured by raspberry-PI. This can be explained by the fact that Fast Ethernet switch used for the tests did not offer possibility of configuration of VLAN and prioritize some type of traffic, and as a result occasional congestion occurred.
Applications
The following subsections describe two applications of routable GOOSE protocol.
Micro grids and distributed energy resources: Growing importance and accessibility of renewable energy sources especially at the small, local, and even individual scale, will inevitably result in the power system evolving toward the distributed model. So called micro-grids are self-sufficient energy islands offering a lot of opportunities in the form of flexible energy management, optimized cost and lower carbon footprint. On the other hand, it poses some challenges related to the integration of these islands into the whole system. The main difficulty is related to task of demand – response (or load-generation) balancing which has direct impact on system frequency. Maintaining stable frequency across the system (which in case of Europe is 50.0 Hz) is critical for the health of the electrical grid.
The task which is already quite difficult to solve is even more exacerbated by introduction of energy sources which are intermittent and unpredictable by its very nature. Amount of energy generated by solar panels and windmills could differ greatly depending on weather conditions.
Even assuming that short term weather forecasts are quite reliable and offer pretty good accuracy, the sheer amount of new devices (IEDs) increases the complexity of coordination task. To make things even more complicated, now energy can flow in both directions: from the utilities (main grid) to the micro-grid or in the opposite direction in case of the surplus of energy generated locally from the renewable sources. The idea of a micro grid is shown in Figure 5.
Nowadays DERMS must communicate not only with substations but also with micro-grids and even individual energy generation points. All of that requires robust, real time and secure communication. Since the whole system is getting logically and geographically distributed, protocols capable of delivering information over wide area networks are required. Depending on the type of the information being sent this could be routable GOOSE for delivery of binary states or routable Sampled Values for transportation of measurements which are used by synchro-phasors for monitoring the health of the grid. In both cases implementation of the routable version of the protocol would be required to integrate the micro-grids and individual IEDs with the rest of the system.
And this is what actually defines the smart grid:
1) ability to reliably exchange information in real time between the IEDs
2) make automated decisions based on this information.
One of the scenarios which requires coordinated response of the system is so called micro-grid islanding, when the power supply e.g. to the university campus, factory or military base is disrupted and the local power network must be reconfigured in real time to turn on backup generators and use energy from locally installed DERs rather than store it in the batteries or feed it back to the main grid.
Since this is sensitive information concerning critical infrastructure performance and cybersecurity are of course of paramount importance, so authentication of IEDs and encryption of the data being sent is mandatory.
Transmission Line Protection: Another area where routable GOOSE could play an important role is teleprotection. By its very definition the teleprotection is responsible for monitoring state of the power grid, detection of faults and taking appropriate actions (e.g. switching position of a circuit breaker) to isolate the faults in order to prevent its propagation, protect critical infrastructure from severe damage, minimize the cost of the outage and its impact on local communities. Implementing efficient teleprotection system requires sending information about the state of the power grid across the WAN. In case of the fault caused by the equipment failure or natural event, information about it must be sent to other substations. Since GOOSE was originally designed as L2 protocol to be used on substation LAN, sending it between substations required some additional configuration effort, e.g. setup of MPLS. With the advent of the routable GOOSE implementation of complex teleprotection schemes over wide area networks become easier.
At the application layer there is no difference between L2 GOOSE and routable GOOSE. Both L2 GOOSE and routable GOOSE use the same subscriber / publisher model with publisher sending out multi-cast messages. (Figure 6).
There is of course a question of delays introduced during transmission of UDP multicast packets over the Internet and the impact of these delays on the protection logic being implemented. A good starting point for the performance analysis of implementation of GOOSE over WAN is a paper by Fred Steinhauser titled “Did you know that GOOSE accelerates your communication aided protection scheme?“
Conclusions
I wanted to present my take on the issue of deployment of GOOSE protocol with distributed systems. Definitely it is not the only solution possible, and probably depending on the requirements of the specific use case or business cases it might not be the best one. Also, the article only covers the issue of protocol conversion and cybersecurity of communication. There are a number of other features which could be implemented on a gateway used by the power system:
- Mapping between different protocols
- Redundancy of communication channels
- Buffering and storage of periodic data.
However, I hope this is a good starting point, at least for the conversation on the matter, so any feedback is much appreciated.
Biography:
Jan Mazan is one of co-founders of JPembedded – software house based in Krakow Poland, established in 2006. He graduated computer science at Jagiellonian University and has many years of experience related to implementation of software projects for customer all over the world. Since 2014 years Jan has been leading development of JPEmbedded’s products family for smart grid communication. These include hardware-based protocol converters as well as platform independent software libraries implementing standards like IEC-61850, DNP3, ICCP, IEC60870-5-10x.
