By Thomas Charton, NGET, UK, Dieter Binon, Elia, Belgium, Volker Leftoff, RTE, France, Lei Xu, NR Electric, China, and Takaya Shono, Toshiba, Japan
Starting from about 2015, an increasing number of publications reported the design or commissioning of demonstrators, pilot projects, pre-series or series of fully digital substation Protection, Automation and Control Systems (PACS) using IEC 61850 process bus (PB). This constitutes a major development, as until then, PACS were mainly using only IEC 61850 station bus. The increasing availability on the market of Intelligent Electronic Devices (IED), Stand Alone Merging Units (SAMU), Merging Units (MU), Switchgear Control Unit (SCU) and Low-Power Instrument Transformers (LPIT) connecting to the process bus sustains this trend.
There has been no shortage of reminders that we need to ramp up the pace of our energy system transformation and make progress with agreed targets in terms of decarbonization. More digitalization and a wider introduction of IEC 61850 process bus technology has always been considered one of the solutions to help deliver a more agile and flexible electricity system that can accommodate new connections and secondary asset replacement without lengthy primary system outages. It also provides the capability to use process bus data to validate and monitor the health of the protection and control system.
Despite the growing experience from installed systems, for many of us, process bus deployments are still considered new technology. Utilities are in the process of optimizing specifications and standards and embed the learning from their own and other stakeholders’ experience. Significantly different implementations of process bus depending on utilities and specific substations are observed. There can also be significant variations in the communication architecture. Some utilities are using separate station and process bus as physical systems, while in other cases the logical station and process buses run on the same physical Ethernet network. The implementations of the time synchronization, redundancy and many other implementation issues are addressed in different ways by the users of process bus.
Our industry prioritizes secure, dependable systems. We’ve progressed from feasibility studies and pilot projects to optimizing process bus deployment, technologies, and designs. Our goal is to maximize the value from these systems. In order to gain confidence and enable a realistic technical and economic evaluation of these technologies, it is necessary for the industry to obtain an overview over these projects and the experience gained and lessons learnt so far. A review of a wide range of process bus projects as part of CIGRE working group B5.69 “Experience gained and Recommendations for Implementation of Process Bus in Protection, Automation and Control Systems” has highlighted some common areas of interest and trends with regards to the future direction of travel and the challenges on the way to optimized process bus implementations. In the following, some feedback regarding these questions and areas of improvement for the next generation of digital secondary systems is summarized based on the Technical Brochure under preparation by WG B5.69. We hope designers, system integrators and supply chain stakeholders will benefit from this for their current and next generation of Protection Automation and Control Systems (PACS).
Process Bus Level Interoperability and Time Synch–still a Thorny Issue?
At the heart of standardization, we expect interoperability, which is listed as a key objective in IEC 61850. The concept has grown from the initial interoperability of the communications to also include the structure and the semantic meaning of the information as well as Substation Configuration Language (SCL). Progress with interoperability is regularly tested as part of the UCA International User Group Interoperability Tests (IoP).
Detailed test reports are available from the UCA illustrating use cases such as interoperability between Sampled Value (SV) profiles and backwards compatibility. In general, the interoperability for sampled value streams has never had any major issues. However, further challenges regarding time synchronization, holdover, behavior of (boundary) clocks during a time shift, validation of IED Capability Descriptions (ICD files) and the quality of SCL tools still require improvements. It is also worth noting that simulated SV streams should not be used in a system with IEC 61850-9-2 Ed1 SV subscribers as this feature is not adequately supported. (See Figure 1).
The time synchronization related behavior based on Precision Time Protocol (PTP) is essential to the functional behavior. PTP is generally implemented with a set of clocks, each being assigned a class and a priority. The Best Master Clock Algorithm (BMCA) picks the relevant time source based on these parameters. Implementation experience suggests that clearly defined priorities based on oscillator accuracy can avoid slow failovers where protection functions might be affected. The standard (IEC 61850-9-3) allows for up to 16 s of loss of time synch during the changeover between master clocks which may not be acceptable to utilities. Figure 1 illustrates the transition to a new master clock as defined in the standard. Scenarios where time synch could be maintained when the BMCA switches between clocks which have a small offset need to be defined and how to handle larger time jumps.
A possible improvement to the availability of protection functions could be to ensure that the holdover capability of merging units during loss of synch is sufficient to cover the slowest possible transition between master clocks triggered by the BMCA. Another factor affecting the delays in these transitions is a mismatch in PTP profiles.
Therefore, implementations need to carefully match PTP profiles to reduce temporary synch related protection function loss. Overall, further standardization of the behavior during synch status changes and holdover conditions needs to take place to guarantee interoperability under degraded conditions. Since in general, protection and control systems cannot tolerate these transients due to the risk of mis-operations, it is important to develop methods that provide corrected synchronized process bus data via software approaches.
More Links in the Chain
Beyond the interoperability between devices in the protection chain there are a few more components to consider and things to look out for when assessing the overall performance of PACS functions. Travelling through the functional chain, starting at the instrument transformers, there may be more emphasis on the validation of the characteristics, transfer function and accuracy of merging units. (Figure 2).
Requirements for digitally interfaced Instrument Transformers are defined in the IEC 61869 series of standards and their impact on the overall protection chain needs to be understood. Once measurements are converted to an SV stream, the design of the communication network including bandwidth, credible failure modes and latency will need to be considered before the actual characteristic and performance of the protection algorithm will come to bear. Some variation in the behavior during failure scenarios for time synchronization and networking such as time adjustments and lost packets have been found in some implementations.
These issues are being addressed by the GOOSE/SV supervision task force of IEC TC57 WG10, which is due to provide a Technical Report TR 61850-90-28. Finally, protection function requirements declared by the manufacturer for Current Transformers (CT) are mandatory in IEC 60255-1xx relay protection standards and need to be adapted and translated into requirements for Merging Units publishing the SV streams subscribed to by protection functions. Figure 3 shows the result of the B5.69 survey regarding the use of SAMU and LPIT in process-bus projects.
Merging Units offer the possibility of providing validated data via software-based corrections. Two main issues are: (a) error correction due to the non-ideal characteristics of instrumentation channels and (b) calibration of the MUs. For example, in the case of conventional CTs and VTs, the instrumentation channel can be modelled and provide error correction of the acquired SV. An example is shown in Figure 4 for the case of a conventional CT which experiences saturation.
Field experience indicates that calibration should address the following issues: (a) magnitude, (b) DC offset, and (c) synchronization. The technology allows software-based calibration, assuming that the errors can be measured in field conditions. One effective and accurate way to measure errors under field conditions is via dynamic state estimation. This process provides the errors in magnitude, DC offset and synchronization which can then be compensated by software-based calibration.
Network Architecture, Redundancy and Flexible Subscriptions
Most implementations of process bus PACS have opted for either Parallel Redundancy Protocol (PRP) or High availability Seamless Redundancy (HSR) or sections of both (IEC 62439-3). The survey of B5.69 has shown that a preference for PRP has been expressed by some utilities when considering extendibility and management of cyber security, but the options are vast and it is easy to get entangled in the complexity between resilience, cyber security and asset management requirements.
Network engineering and traffic segregation are also flagged up as essential to ensure that no IED ports get flooded with unnecessary messages which could lead to a denial-of-service kind of scenario.
Typically merging units and switchgear interface units will be duplicated leading to a fully duplicated set of measurements and plant status messages on the process bus, thus offering opportunities to engineer automatic failover into protection and control functions, should the default dataset no longer be available. This brings obvious benefits in terms of functional availability but has also a cost in design and testing. As design templates and automated testing can address some of the downsides, we may well see more designers adopt automatic failover.
For those utilities that don’t install VTs on their busbars, the implementation of the voltage synchronizing function for the recloser has been a challenge in particular for larger substations. The conventional design would consist of a hardwired connection of all bay VTs with auxiliary switches and relays to derive the busbar voltage for any given primary topology. Replicating this on the process bus would require subscription to all SV streams containing bay voltages that may be required to derive the running voltage on the busbar which easily pushes some products beyond their limit of SV stream subscriptions. Flexible subscriptions at the receiving end or flexible publication on the publishing end might provide further redundancy options and could save on additional VT requirements.
Centralized PACS
With the progress in computing power and real time capability of commodity hardware the transition to centralization appears to be the next step in the digital transition of our OT assets. The concept consists of hosting all PACS functions centrally on a single or a cluster of servers, interfacing the process bus which provides the data from the entire substation. The bay equipment is thus reduced to the process interface units. Overall, the potential to deliver savings through centralization was confirmed in the survey performed by B5.69. In addition to full centralization new functional integration options for bay IEDs and IEDs covering multiple bays are also under evaluation by several utilities as shown in Figure 5.
Replacement of LPIT MUs
Maintainability and efficient replacement of IEDs and merging units is essential for any utility. Where multiple IEDs subscribe to the SV stream from a MU its replacement would typically require re-validation of all affected functional chains. To avoid any updates to other IEDs and network configurations, the replacement MU needs to be more or less a clone of the replaced device. This may include matching all functional settings as well as the communications configuration.
The main takeaway from this scenario is clearly that the procedure for tests and replacement of any device needs to consider any side effect on functional chains, recalibration, the communication network and cyber security from the outset in the engineering process.
Environmental Aspects
As the environmental requirements may vary quite a bit between utilities and are likely to see more extremes, there is a common challenge to control temperature and humidity in areas where PIUs and other IEDs may be installed. The challenging working environment, in particular in outdoor cubicles is also a reminder why we need to carefully consider which functions are hosted on which devices. Considering what is at stake, getting the interface to primary plant right should be top of our priorities.
Engineering Process
Standardization and the use of templates features very heavily when we think about the benefits and opportunities for digital substation deployment. However, the work to make this a reality is still in progress. Getting the interfaces and tools standardized is a key part of this. There is a broad consensus that opting for a top-down engineering approach is the way to go. This starts with bay level functional requirements including the data and data attributes which will be part of the IEC 61850 ISD and SSD files. Based on this information (virtual) IEDs can be selected that meet all requirements as documented in their respective ICD files. This should also include hardware performance, internal logic, application profiles (such as IEC 61869-9 support for SV) and setting ranges. Recent work defining these requirements in IEC 61850-90-30 and IEC 61850-7-6 has provided more guidance and support for the implementation of a standardized engineering process.
Enabling utilities to reap the full benefit of utility specific templates may require a move to their own private data model which in turn needs to be developed and maintained. The IEC 61850-1-2 namespace specification description (NSD) is a key part of this, but care should be taken to follow the relevant rules when creating private namespaces to ensure interoperability is not jeopardized. Defining and maintaining a private namespace definition will involve the creation of NSD files to start with as well as data type templates, function templates, bay and application profiles which enable the creation of ISD and SSD files for projects. A successful implementation of these technologies will require a suitable engineering lifecycle management system to enable a smooth run of the process and its workflows, since all these templates will have to be managed and sometimes a part of the system can evolve while the rest remains unchanged.
To make this process efficient and reliable standardization and tools are required. Full integration with testing and validation processes and tools is also a key success factor to address issues as early as possible during the engineering process rather than later at the Factory Acceptance Test (FAT).
Cyber Security
When it comes to cyber security the message has been very clear. It needs to be part of everything we deliver, in particular for process bus applications where performance requirements may put some constraints on the available security measures. Many utilities are moving from a generic reliance on IEC 62351 and IEC 62443 to a more detailed specification of how to apply these standards. Process bus applications undoubtedly increase the attack surface from a number of threats such as:
- Spoofing, i.e. an attacker manages to imitate a legitimate device and sends messages which are being processed by subscribers instead. Message authentication for SV and GOOSE can be used to mitigate this type of threat
- Replay, i.e. recording and retransmission of network traffic. This type of threat can be mitigated using state machines which also help handle delayed packets for SV
- Tampering, i.e. replaying modified network traffic. This should be mitigated by implementing secure packet integrity protection
- Alteration of static data in MUs and applications
Some of the measures that lend themselves to secure GOOSE and SV are described in IEC 62351 series of standards
When implementing cyber security, it is important to cater for any legitimate operational scenarios during commissioning, fault finding, digital injection testing, replacement, patching. This may include clear documentation how to enable functionality that is normally disabled to cater for these activities.
Testing and Monitoring
Due to the numerous opportunities to build redundancy into digital substation designs and extended opportunities to track diagnostic data, it is possible to move away from a time-based maintenance regime to a condition and event focused regime. Patching and software / firmware upgrades and obsolescence management are taking a central role in secondary systems asset management. This further emphasizes the need for efficient firmware and configuration management processes and tools.
The built in test features in edition 2 of the IEC 61850 standard allow many tests to be carried out device by device or even function by function thus reducing outage time massively.
The CIGRE technical brochure 760 “Test strategy for Protection, Automation and Control (PAC) functions in a fully digital substation based on IEC 61850 applications” covers these features in detail. As mentioned above, the full integration of the testing into the engineering process is still in development. The required tests can all be standardized and derived from the Substation Configuration Description (SCD) but care should be taken to ensure that tests validate the requirements rather than self-validate the current configuration which may contain errors.
Monitoring of the communication network for performance, bandwidth usage, dropped packets, latency and other indicators will need to become an integral part of the SCADA system and there may even be synergies between the cyber security monitoring of the traffic and the monitoring of network and asset performance. The survey carried out as part of CIGRE B5.69 highlights the landscape of how new test features are adopted by utilities. Building on the lessons from initial pilots a further shift towards the digital test features can be expected. (Figure 6).
Conclusions
Looking at the experience gained from recent projects and ongoing development activities, the survey performed by WG B5.69 confirms that there is definitely a drive to make digitalization work and deliver benefits to customers and consumers. The process bus is a key enabler for further changes in functional integration, such as centralization and virtualization that will enable a more efficient and agile approach to delivering substation functions. It also provides the flexibility to handle the ever more complex behavior of our networks in the context of the rapidly changing energy landscape.
Many utilities are facing a massive increase of workload with new connections, changing flows, reduced inertia, control interaction and new infrastructure requirements. To maximize the benefits from digital technology, after a phase of pilot projects and demonstrators, the technology will need to prove its benefits as part of wider deployment.
Whilst WG B5.69 has focused here mainly on the technical challenges; it is worth noting that there are a number of other issues which might prove even bigger obstacles to overcome the future challenges the industry faces. Staff training and resource shortages are all too common within utilities and their supply chain. Up scaling our capabilities will take some time and effort but digitalization and innovation will remain our most valuable tools to achieve a successful energy system transformation.
Biographies:
Thomas Charton is a Senior Lifecycle Engineer at National Grid Electricity Transmission (NGET). He joined National Grid in 2010 as a SCADA and Communications Engineer and is now responsible for NGET’s Substation Control System, telecoms and cyber security lifecycle management. He is also leading NGET’s digital substation development and rollout. In 2022 he was appointed to the role of UK regular member for CIGRE D2 (Information Systems and Telecommunications). Thomas is a chartered electrical engineer, a member of the IET and received his M.Eng. in Electrical Engineering in 2000 followed by a Ph.D in 2005.
Dieter Binon graduated as an Electrical Engineer, and in 2009 Dieter started his career in smart grid applications at Eandis, a Belgian DSO. In 2019 Dieter joined Elia, the TSO for Belgium, where he is currently a member of the asset management department as an expert in the field of digital substations. He as experience in the field of IEC 61850 applications for over 10 years and is an active member of the CIGRE B5.69 working group and IEC 61850 WG10. He is passionate by all aspects concerning electricity.
Volker Leitloff studied Electrical Engineering at the university of Stuttgart (Germany) and received the Dr. INPG degree from the Institut National Polytechnique de Grenoble (INPG, France). At the R&D Division of Electricité de France, he worked successively on network protection and on transformers and network technology. Since 2003, he works at RTE on protection of transmission networks and substation control. He is chair of IEC TC38 and CENELEC TC38 (Instrument Transformers), convenor of IEC TC95 WG2 (Digital I/O of protection IED,) and actual or past member of several IEC and CIGRE working groups.
Lei XU received his bachelor’s and master’s degree in electrical engineering in 1998 and 2001 from Southeast University and NARI respectively. He joined NR Electric in 2001 as a relay R&D engineer, from 2003 he developed process level equipment including LPIT, SAMU, CSD and BIED. From 2007 he worked as the project leader and now the principal engineer in the Department of Digital Substation. From 2012, He started to participate in the activity of IEC and Cigre, He is the member of TC17(MT2/MT9), TC38(WG37/WG47/WG48/MT58/MT59), TC95(MT2/WG2) and Cigre A3.35, A3.49, B3.39, B5/D2.67, B5.69, B5.83 and the convener of B5.59. He also works as the liaison of TC38-TC17, TC38-Cigre B3, and Cigre B5-TC17.
Takaya Shono joined Toshiba Corporation in 1995. Since then, he has been engaged in the development of IEDs and network solutions for power system protection and control system. Especially, he was engaged in the development of IED, WAMPAC, line differential protection over Ethernet / IP network, etc. Since 2009, he has been engaged in the development of Digital Substation Automation System based on IEC 61850 and the process bus. Now he is working on a DX (Digital Transformation) solution based on the full digitalization of the substation.
Julien Saunier is an active CIGRE B5.69 WG member who graduated as Electrotechnical & Automation engineer in 2008 and has over 10 years of experience in System Engineering and Technical Leadership. He moved to Digital Substation Solutions product management role participating in innovation projects including digital substation projects and successfully implemented the concept in multiple regions/countries. In 2022, Julien joined Hitachi Energy – High Voltage Product Business Unit leading the digital strategy of Power & Industry Components Global Product Group.
Rene Troost graduated as an Electrical Engineer. In 2014, Troost joined Stedin, the DSO for the South-West area of The Netherlands, including the Port of Rotterdam. He is currently responsible for substation automation policy in Stedin. He chairs the Dutch technical working group designing a DSO/TSO-DER interface, is an active member of IEC TC57 (NEC57) and TC57 WG10 and chairs the Dutch CIGRE B5 study committee.
