by Amadou Louh, Rene Troost, and Joey Godefrooi Stedin, Shuang Zhang, HUAWEI, Andrej Goerbing, and Andreas Jahr, Siemens
In their quest for a stable, safe and reliable electricity grid in the near and far future, distribution grid operators will have to deal with a dramatic increase in digital assets for the purpose of monitoring, protection and control. However, this automation of the distribution part of the grid is not possible without telecommunication. From hundreds of substations in the transmission network equipped with substation automation systems, tens of thousands or even hundreds of thousands of data points will need connectivity to exchange data in the distribution network.
Where substations in transmission networks are most often traditionally connected via a wired network, distribution substations are mostly without any form of communication. Rolling out fiber optics to all those connection points would be extremely costly.
In order to meet the performance and security requirements for monitoring, control and protection of distribution networks, telecommunication solutions must support at least the 5 following important features: latency, precision timing (synchronization), service segregation, availability and resiliency in order to support best-effort and critical services. Cellular technology offers an outcome to provide those 5 features and seamlessly extend existing wired packet switched networks without having to lay dedicated communication cables, either in refurbishment or new distribution substation construction projects. There are relatively high requirements for line differential protection communication, which are challenging for mobile networks (3GPP Rel.16 TR 22.804, 5.6.6 and 3GPP Rel.18 TR 22.867, 5.4.1.):
- Continuous communication between protection relays attached to the 5G network (Figure1), which is based on IP unicast and multicast, with up to approx. 1 Mbit/s in uplink and downlink direction
- Maximum end-to-end communication latency 5 ms to 10 ms with latency asymmetry < 2 ms. Figure 2 gives a reference on how the overall permissible telecom path delay is estimated for a total fault clearance time of approximately 100 ms.
- High reliable communication with minimum corrupted and dropped packets (< 1 %).
- High available communication (service availability at least 99,9999 % or 99,95% according to IEC 60870-4, class A3) with minimum recovery time. This means in particular that 5G communication shall not be interrupted or disturbed in case of power grid failure (power supply outage) in the served area.
- Minimum impact from other, interfering communication in the 5G network.
- High precise and reliable time synchronization of protection relays via the 5G network, based on IEEE 1588v2 with the PTP profile according to IEC 61850-9-3. (see page 46).
Description of the 5G Releases
Since the inception of 5G, 3GPP has been developing technologies to address the needs of a wide range of market segments and vertical industries. As a result of this approach, the 3GPP eco-system of today have the fundamental building blocks to support the identified key groups of markets: critical communications (Ultra Reliable Low Latency Communications known as URLLC), massive Internet o Things (mIoT), enhanced mobile broadband (eMBB), and network operations.
Back in 2017, 3GPP approved a Release 15 work item on the communication service requirements for automation in vertical domains, which incorporated a number of industry use cases for factory automation, motion control, mobile robots, process automation, rail-based mass transit, smart city, smart grid distribution automation, etc. Requirement from regulation such as NERC CIP, from industry organizations such as UITP, CCBG, ZVEI, CENELEC, and the corresponding industry standards such as IEC 62351 “Security for electricity distribution systems”, IEC 62443 – Network and system security are supported by 3GPP specifications. Meanwhile, 5G has developed further to support LAN-based technologies for industries, including interconnecting sensors and actuators with closed-loop and supervisory control. Aside from meeting the stringent performance requirements in terms of high availability and low latency, 5G is made suitable to support VLAN types of services. With 5G-LAN service, industry users can create VLAN services in the 5G system, and set up, configure and maintain a defined set of User Equipment (UEs) in this VLAN as needed by specific industry applications. A UE could for example be imbedded in an IED. Thanks to the wide-area coverage of 5G, 5G-LAN service is potentially an excellent candidate for a DSO to integrate remote substations into the existing private MPLS network infrastructure.
In 2019, 3GPP approved a Release 17 work item on enhanced cyber-physical control applications for vertical industries, which ensures the 5G could provide communication services supporting enhanced time-sensitive networking (TSN) defined by IEEE 802.1 working group. This enabled the 5G and industrial Ethernet integration including time synchronization and support for different time domains – for DSO that has its own PTP grandmaster, 5G system can be simply perceived as IEEE 802.1 compliant virtual Ethernet TSN bridges to transfer the PTP clock signals to the end device in a MV substation. Within the same work item, isolation of different communication services and non-public networks (NPN) as private network slices with enhancement of security were added to the 5G features.
The pinnacle of 5G for electrical utilities is manifested by a recently completed 3GPP Release 18 work item on smart energy and infrastructure. The comprehensive work addresses service requirements on 5G system to cater the principal services of the Smart Grid: Remote Control (SCADA services), Energy Management System, Distribution Management System, Tele-protection, Smart Metering, PMUs, Distributed Generation, Distribution Automation, Demand Response, and Support for micro-grid, Distributed Energy Resources (DERs).
The 5G releases in this article are described in terms of what they can offer to support the business needs in a DSO landscape and how they can cope with the addressed challenges mentioned in earlier sections on latency, precision timing (synchronization), service segregation, availability and resiliency.
To date, below are a selection of 5G capabilities specified by 3GPP to fulfil the smart grid service requirements.
Latency: When it comes to latency, line differential protection is the most critical use case for DNOs. From the 3GPP requirements’ perspective, (3GPP TS 22.104, Service requirements for cyber-physical control applications in vertical domains) up till Release 18, the 5G capabilities expected to support differential protection are listed below:
- The 5G system shall provide a suitable means to support precise time distribution, clock synchronization communication functionalities specific to smart grid applications, e.g.: IEC 61850-9-3 profile and IEEE Std C37.238-2017
- The 5G system shall support an end-to-end latency of less than 5 ms or 10 ms, as requested by the UE initiating the communication
- NOTE 1: Whether the end-to-end latency is 5 ms or 10 ms depends on the applied voltage level
- NOTE 2: The end-to-end latency is between two UEs, including two wireless links
- The 5G system shall support communication channel symmetry in terms of end-to-end latency (latency from UE1 to UE2, and end-to-end latency from UE2 to UE1), with an asymmetry of < 2ms.
- In 2019, 3GPP approved a Release 17 work item on enhanced cyber-physical control applications for vertical industries, which ensures the 5G could provide communication services supporting enhanced time-sensitive networking (TSN) defined by IEEE 802.1 working group
Synchronization: Time synchronization has been important for smart grid. With the advent of Centralized protection and Control (CPC) and smarter applications designed to react and response to dynamic instances in the smart grid, the importance of time synchronization is growing. To fulfil this need, 5G system also supports Time Sensitive Networking (TSN). In other words, 5G system shall support the fully distributed model for configuration of time-sensitive networking that is aligned with Multiple Stream Registration Protocol (MSRP, IEEE 802.1Q), IEEE P802.1CS Link-local Registration Protocol (LRP), and IEEE P802.1Qdd Resource Allocation Protocol (RAP), and specifically 3GPP TS 22.104:
- The synchronicity budget for the 5G system within the global time domain or within a working clock domain shall not exceed 900 ns. Figure 3 is an excerpt from 3GPP system architecture specification w.r.t. how TSN is realized
- The 5G system shall be able to support arbitrary placement of sync master functionality and sync device functionality in integrated 5G / non-3GPP TSN networks
- The 5G system shall be able to support clock synchronization through the 5G network if the sync master and the sync devices are served by different UEs
- The 5G system shall provide a suitable means to support precise time distribution, clock synchronization communication functionalities specific to smart grid applications, e.g.: IEC 61850-9-3 profile and IEEE Std C37.238-2017
The clock synchronisation KPI requirements are specified in Table 1 (from 3GPP Release 18 TS 22.104).
A typical use case that addresses 5G synchronization capabilities are Synchrophasor-based WPAC applications. 3GPP Release 18 stage 1 requirements specifically capture the wide-area communication for PMUs for protection and control purposes. Specifically, the requirements are, 3GPP TS 22.104:
- The 5G system shall support at least one of the two profiles for synchrophasor communications: IEC 61850-90-5:2012, or IEEE Std C37.118.2-2011
- NOTE 7: The requirement above assumes the ability to support all main clock synchronization functionalities defined in the respective references, unless explicitly indicated. Support of IEC 61850-90-5 can entail supporting multiple profiles (e.g. A-profile, T-profile, KDC-profile and IEEE 802.1Q QoS profiles can be applicable for support of synchrophasor communications)
- The 5G system shall support delivery of the same UE-originated data in a resource-efficient manner in terms of service bit rate to UEs distributed over a large geographical area
- The 5G system shall allow a UE to request a communication service to simultaneously send data to different groups of UEs at the same time
- The 5G system shall allow different QoS policy for each group the UE communicates with
Service Segregation: 5G system enables different ways of segregation. In this article the focus will be on 5G LAN-virtual network (5G LAN-VN) service and network slicing.
5G LAN-virtual Network (5G LAN-VN) Service
Generally speaking, 5G LAN Virtual Network (5G LAN-VN) service provided by 5G system aims at enabling on-demand and customized establishment of LAN/WAN networking for industrial devices connected with 5G, and accordingly facilitating seamless migration of industrial and smart grid control and automation applications. Multiple types of data communication shall be supported, at least IP and Ethernet. The 5G network shall ensure that only member UEs of the same 5G LAN-VN are able to establish or maintain private communications among each other using 5G LAN-type service. Using the 5G LAN-VN, routing based on a private addressing scheme as preferred by the DSO is supported, too. When a DSO uses 5G LAN-VN for connecting devices, 5G shall support a mechanism to provide consistent QoE to all the member UEs in the 5G LAN-VN. In the meantime, depending on the smart grid application, the 5G network shall also enable member UEs of a 5G LAN-VN to use multicast/broadcast over a 5G LAN-type service to communicate with required latency (e.g. 180 ms), 3GPP TS 22.1043GPP TS 22.104].
Figure 4 illustrates how DSO can use 5G LAN service to realize private connection among UEs and the devices in the data center. An interesting example would be the IEDs or Distribution Automation gateways across different MV substations and/or LV cabinets could be connected wireless with each other and with the DSO’s data center, in the forms of LxVPN as needed by the existing services provided by the DSO’s MPLS infrastructure.
Figure 5 shows an example that two 5G LAN VN Groups can be created for PMUs that are used for monitoring and control purposes.
In Release 18, 5G LAN service is going to be further enhanced by extending 5G LAN-VN to cover multiple sites, enhancing 5G core network interconnection capabilities by possible coordination among different SMFs (Session Management Functions), and realizing the following capabilities, too:
- Generic Group Management: Generalize 5G VN group for other usages besides group communication, e.g., group targeting same edge server
- Group Exposure: Enhance NEF exposure framework for 5G VN to support 5G-ACIA/SA1 requirements (connectivity configuration, connectivity monitoring and group management)
- Group Communication: Enhance 5G VN communication to enable better support for multi-site campus (factory/enterprise), and integration with legacy L2 LAN, as well as new use cases (5G Smart Energy and Infrastructure, Residential 5G)
- The 5G LAN-VN shall support member UEs that are subscribed to different Public Land Mobile Networks (PLMNs), e.g. a 5G LAN-VN may span multiple countries and have member UEs that have a subscription to a PLMN in their home country.
- The 5G system shall be able to restrict private communications within a 5G LAN-VN based on UE’s location (i.e. when the UE moves out of the area it can no longer communicate on the 5G LAN-VN)
Network Slicing is one of the important capabilities of 5G. Not only security but also the control of used network slices by a DSO are enabled according to 3GPP stage 1 requirements. In short, the 5G system shall provide suitable Application Programming Interfaces (APIs) to allow third-party infrastructure (i.e. physical/virtual network entities at RAN/core level) to be used in a private slice. Furthermore, based on operator policy, the 5G network shall provide suitable APIs to allow a trusted third-party application to request appropriate QoE from the network, and/or to allow a trusted 3rd-party to create, monitor, modify, and delete network slices being used.
Some important relevant 3GPP requirements for network slicing are, 3GPP TS 22.261: Service requirements for the 5G system:
- Based on operator policy, a 5G network shall provide suitable APIs to allow a trusted third-party to create, monitor, modify, and delete network slices used for the third-party
- Based on operator policy, the 5G network shall provide suitable APIs to allow a trusted third-party application to request appropriate QoE from the network
- The 5G system shall provide suitable APIs to allow third-party infrastructure (i.e. physical/virtual network entities at RAN/core level) to be used in a private slice
Resiliency and Availability Considerations: Using 5G for more business applications, increases the relevance and importance of resiliency. A robust network is a network that delivers the required levels of network availability, reliability and resiliency. We will define network resiliency as the ability to provide and maintain an acceptable service level in case of faults, disruptions and other events affecting normal system operation. In this article resiliency is addressed on data security, privacy and application level.
The 5G network consists mainly of the 5G Core, the Next-Generation Radio Access Network (NG-RAN) and the user equipment (UE). At all these levels features and mechanisms are put in place to increase resiliency. The 5G robustness toolbox gives mobile operators the flexibility and power to activate the most appropriate mechanisms depending on the use cases and the deployment variants for different UE’s in the same network. Both standardized and vendor-specific network features and mechanisms are included.
Features at the Core Level
5G core network functions (NFs) also have the internal mechanisms to tolerate failover at software-component level. Cloud native implementations of 5G core allows the support of “fail fast, recover fast” principle to ensure internal resilience between software components.
At network level, the 5G core focuses on standard session resilience support instead of vendor-specific georedundancy solutions. The general In-Service performance requirement on network function availability for mobile broadband services is 99.999 % and the requirement for 5G-critical services (e.g. industrial manufacturing) is even up to zero tolerance of failure interruption.
The 3GPP introduces the generic network function set concept for 5G core control plane network functions to support E2E session resilience at network level. With this concept, (geo)redundancy and scalability can be provided by deploying a network function in such a way that several instances are part of a network function set. In such a set, the equivalent network functions share the same context data. In case of a failure a network function instance can then be replaced by an alternative network function instance within the same set.
Based on product implementation, the network functions around the UDR (user data repository) already support failure reselection in the UDR georedundant cluster. Furthermore, the 5G Core supports overload control, load (re-)balancing, and NAS-level congestion control to ensure that the network functions are operating under nominal capacity for providing connectivity and necessary services to the UE’s. Load and overload control over a service-based interface are the generic mechanisms for all 5G Core control plane network functions.
Features at the NG-RAN Level
As RAN centralization leads to a higher number of UEs being served by a unit that may fail, the “fail fast, recover fast” principle will apply to smaller modules than box level, as in LTE. The NG-RAN standards still contain similar Single Points of Failures (SPOFs) as LTE for the whole UE on RRC-level and pCell for the UE. A new SPOF is the UE-RAN UP on PDU session level. In addition, there are functions to support bounded latency and higher reliability, as well as unified access control. The current understanding is that vendor implementations can solve the above SPOFs, at least partly based on cloud-native principles. These features and mechanisms will cover robust UE UP and UE CP on the RAN side, robust RAN resources like cell availability, radio link and air-interface redundancy mechanisms, as well as fronthaul transport.
Regarding the requirements concerning data security and privacy for smart grid communication, 3GPP stage 1 states the following, 3GPP TS 22.261:
- The 5G system shall support data integrity protection and confidentiality methods that serve URLLC, high data rates and energy constrained devices
- The 5G system shall support a mechanism to verify the integrity of a message as well as the authenticity of the sender of the message
- The 5G system shall support encryption for URLLC services within the requested end-to-end latency
Resiliency at Application Level: Resiliency at application level will be illustrated with line differential protection. Properly engineered protection systems use redundant protection functions and redundant communication paths.
Standard protection guidelines and policies are describing different protection functions and schemes to provide this backup to realize a resilient energy system. The most common protection is overcurrent protection which function stand-alone and are therefore well suited as backup functionality.
Communication is a part of the overall chain and should match with the power autonomy policy of the utility company. Four to eight hours power autonomy is very common for DSO’s, meaning that the 5G network should at least have battery backup for the same amount of time. This is unfortunately not a part of the 5G standard but depends on the service provider.
Conclusions and Recommendations
This article is derived from a CIGRE paper Seamless extension of fiber optical IP/MPLS network with 5G technology Releases allowing Business service segregation, Precision time synchronization and Critical teleprotection services in Utility distribution networks.
This article shows that the 5 major preconditional requirements could be met by 5G. The technology offers more flexibility to mobile operators to activate appropriate mechanisms depending on the use cases, deployment variants and UE’s to increase resiliency in the 5G network. Improving the overall observability of the network as well as automation in the correction of network failures, is key to improving network reliability and resilience.
Biographies:
Amadou Louh works at Stedin as a strategic advisor in the field of communication and is chairman of the Dutch study committee D2 (Information Systems and Telecommunications). He holds two MSc’s in Telecommunication and Power Systems – High-Voltage engineering, from Delft University of Technology (TU Delft). His main area of expertise is data communication and substation automation. This mostly comprises design and maintenance specifications for IP/MPLS data communication networks and IEC 61850 substation automation systems. He is the chairman of the Dutch CIGRE study committee
Rene Troost graduated as an Electrical Engineer. In 2014, Troost joined Stedin, the DSO for the South-West area of The Netherlands, including the Port of Rotterdam. He is currently responsible for substation automation policy in Stedin. He chairs the Dutch technical working group designing a DSO/TSO-DER interface, is an active member of IEC TC57 (NEC57) and TC57 WG10 and chairs the Dutch CIGRE B5 study committee.
Joey Godefrooi is Senior Protection Engineer for medium and high voltage systems, a specialist in IEC61850 schemes. He is responsible for the configuration, standardisation and implementation of protection systems for all medium and high voltage systems within the grid operations of the Dutch DSO – Stedin. Throughout his 17-year career within Stedin he has acquired extensive and divers experience in Grid Operations. He has the ability to think out-of-the-box and has been involved in a variety of innovative programs.
Shuang Zhang is an active contributor of 3GPP, ETSI and 5G-ACIA in his role as standardization expert at HUAWEI. Since graduation as Electrical and Electronics Engineer (MSc. Delft University of Technology), he has 15 years’ experience with EU Telecom and Wireless communication industry partners, and a proven track-record of research, innovation, consultancy, standardization and applying Telecom knowledge in vertical industries such as smart grid utility.
Andrej Goerbing is System Architect Communication at product development of Siemens Smart Infrastructure / Electrification & Automation in Berlin / Germany. He is involved in the development of Siemens SIPROTEC protection relay communication and IEC TC57 WG10 standardization, in particular regarding power utility substation-internal and wide area communication and IEEE 1588 synchronization. Andrej has some thirty years of experience in development and practice of various wireline and wireless communication technologies.
Andreas Jahr graduated as Dipl. Ing. EEI (Electrics, Electronics, Information technologies) at the Friedrich-Alexander-University Erlangen-Nurnberg and started working for HV-Substations at Euro-Engineering in Erlangen. In 2013, he switched to SIEMENS SI GSW PTI as consultant for protection coordination and network calculations. In 2016, he took responsibilities as product lifecycle manager for the SIPROTEC Line differential protection devices. This year, 2022, he returned to his SIEMENS roots as consultant for protection coordination and network calculations.
