by Christian Brauner, OMICRON electronics GmbH, Austria and Eugenio Carvalheira, OMICRON electronics Corp., USA
Testing the protection element settings of IEDs and protection schemes are well established practices when testing a Protection, Automation and Control (PAC) system. Tools and methods are available to support standardized and automated protection testing routines. Test plans can be created for specific relay types and schemes to be reused during distinct phases of a project, like Factory Acceptance Tests (FAT), commissioning, Site Acceptance Tests (SAT) and maintenance.
Contrary, testing SAS–the Substation Automation Systems, which involves many automation, control and SCADA functionalities, is usually performed manually. When looking at the time spent during commissioning, for example, testing the automation and communication system is currently more time consuming than testing the protection functions. Automation systems have become increasingly complex and the efforts for testing communication, interlocking logic, and proper operation of all signals transmitted to SCADA systems have grown dramatically.
In substations, all connection interfaces between IEDs and primary equipment need to be checked as part of the FAT and SAT. For hardwired interfaces, as an example, this is usually performed one-by-one in a manual process of “green marking” all interfaces on printed functional and wiring diagrams.
For testing the implemented logic functions such as command interlockings, many physical inputs need to be forced at the same time, and the logic verified by executing the related control operation. For testing the SCADA signaling, an end-to-end check is performed by stimulating the signals directly at the equipment level in the switchyard or by forcing them at the IED’s input terminals. Additional documentation is typically required, like a spreadsheet with Remote Terminal Unit (RTU) signal and mappings list.
This process, which is ideally performed during FAT before delivery and installation at site, requires several weeks for a typical substation and involves several experienced control and SCADA engineers.
The following hardware, software and technical competencies shall be available for system testing in the factory:
- Ideally the entire SAS with all bay IEDs, networking equipment, gateways, HMIs, etc.
- A switchgear simulator hardwired to the IEDs (can be simple switches and LED indicators up to sophisticated PLC based simulators)
- Control center simulator supporting the used SCADA protocol (e.g. IEC 60870-5-104, DNP3)
- Network testing and IED specific maintenance tools
- Deep knowledge about the implemented vendor products, IEC 61850 and Ethernet networks
- Well prepared test plans and documentation (Signal spreadsheet, interlocking logics and other test procedures)
Usually, not all components of the SAS are available at the factory, e.g. in cases where IEDs are part of switchgear deliveries and shipped directly to site without proper system testing. In such cases, testing must be exclusively performed on site, with consequences in terms of efforts and costs.
Practical experience shows that the better the system is tested in the factory, the less problems occur during installation and testing on site, and the more efficient and smooth the project finally is.
During the testing procedure, bugs and errors in device parameters but sometimes also in the device’s firmware are detected and fixed. But every update of firmware version and as well as any change of device settings requires at least retesting of the involved function, ideally a retest of the entire system.
This process is not efficient if manual testing is performed, therefore a new approach towards more automated and efficient system testing is heavily needed. Such a solution is available today and based on the SCL concept which is part of the IEC 61850 standard.
IEC 61850 and the SCL Concept
IEC 61850, the international standard for Communication networks and systems for power utility automation, defines not just communication protocols, but also data models for substation equipment.
Moreover, the standard also specifies a common, vendor independent, configuration concept. Machine readable configuration information in an XML based standardized format is used in this process – the System Configuration Language (SCL).
SCL Engineering Process: The SCL concept is defined in IEC 61850-6. Its main purpose is to allow the exchange of configuration data, in a compatible way, between different configuration and testing tools.
Figure 1 shows the general concept of the engineering process of a substation automation system with the usage of SCL data exchange.
The following types of SCL files, with different extensions, are specified for exchange of information:
- SSD (System Specification Description): describes the single line diagram of the substation, voltage levels, primary equipment and required logical nodes (LN) for implementing the substation functions. The SSD file is generated by a System Specification Tool (SST)
- ICD (IED Capability Description): describes the functional capabilities of an IED type. Each IED type has its related ICD file. It contains the IED logical nodes, data and supported services. It is generated by the vendor specific IED Configuration Tool (ICT)
- SCD (System Configuration Description): contains all configured IEDs, the communication configuration and all IEC 61850 aspects for a given system. It is created by the System Configuration Tool (SCT)
- CID (Configured IED Description): contains a subset of the SCD file with all information related to one specific IED. Private extensions are allowed
In principle, there are three types of engineering tools in this process: System Specification Tool (SST), System Configuration Tool (SCT) and IED Configuration Tool (ICT). Practically, an all-in-one tool and no SSD is often used in case of single-vendor systems. In multi-vendor installations with vendor specific ICTs, typically a dedicated SCT is used. Today, more and more users understand the need for standardization and use an SST for the specification process.
The SCT allows engineers to design and configure the system-wide IEC 61850 communication dataflow. ICD files from all IEDs and the SSD file can be imported into the SCT. The tool should allow the configuration of IEC 61850 related features of the IEDs, configuration of horizontal communication links (GOOSE and Sampled Values) and configuration of vertical communication links (Client/Server Reports). By using data from the SSD file or by direct entry, the engineer can associate IED functions (Logical Nodes) to the single-line equipment and functions. Ultimately, the SCD file, documenting the complete system, is generated by the SCT.
SCL Content: The SCL language in its full scope allows to describe a model of the substation comprising of three basic parts:
- Substation: describes the single line diagram of the substation, primary equipment and functions; The substation equipment such as a Circuit Breaker is “connected” to the virtual logical nodes contained in the IED
- IED: describes all the hardware devices (IEDs) used in the substation automation system. The data model implemented in the IED including its logical devices and logical nodes are described in this part. IEDs are connected to the communication system via its access-points
- Communication: describes logically possible connections between IEDs in sub-networks by means of access-points (communication ports)
The content of a complete SCD file is comprised of these three parts plus a section with data type templates describing which data and attributes are used by the IEDs. (see Figure 2).
Substation Structure and Functional Naming: The substation structure represents the primary system architecture and describes which primary equipment functions are used, and how the equipment is connected. The objects in this section are hierarchically structured and designated according to IEC 81346.
The main purpose of this section is to derive a clear functional designation for the abstract logical nodes, which are implemented in the IEDs to the primary equipment in the substation. Otherwise it might be difficult for the system tester to find out which specific LN-instance in the IED is “connected” to which primary element in the switchgear.
Content and Usage of SCD Files: As explained above, the SCD file is the ultimate file resulting from a completed IEC 61850 system design. The SCD file is not only used by engineering tools and for documentation purposes, but also by testing tools. Testing tools can support a more efficient testing, taking advantage of the SCD file information about the substation under test. However, while the standard defines a clear concept for the engineering process, it does not define a minimum content requirement for the SCD file. Topology information in the substation section, for example, is optional. Information in the IED section depends on the capabilities of the specific IED products used in the project. So, it is strongly recommended for substation owners to include minimum requirements on the SCD file into SAS specifications used for project tenders and service contracts, such as:
- Substation section must contain all voltage levels, bays and CBs/disconnectors with their LN references (XCBR/XSWI, CSWI and CILO)
- Data Objects include “desc” description attributes with signal text as defined by the owner
- GOOSE subscriptions use <IEDName> elements in the <GSEControl> element, and use <Inputs><ExtRef type=”GOOSE”> elements
- RTUs/Gateways or HMIs must be defined, have the Report Control Blocks in the IED reserved and declared using <ClientLN> in the <ReportControl> element
- All data sets used in reports are of static type (dynamic data sets are not documented in the SCD)
The better the quality and content of the substation’s SCD file, the higher the efficiency of system testing will be. A compliant SCD file is also highly supporting later extensions of the station as described below.
New SAS Testing approach based on SCD Files
Test Approach: As mentioned, testing of the automation and control functionality are usually performed in a manual way. Since many years, tools are available offering testing capabilities on a per IED basis, allowing manual test and simulation of individual IEDs.
The method presented here extends the test from single IED testing and simulation to testing of the entire Substation Automation System. The test is entirely based on the SCD configuration file. By importing the SCD file, the entire system can be visualized and all information available in the SCD is used. The information in the substation section is used to place IEDs and switchgear equipment within their voltage levels and bays. As can be seen in Figure 4, the tester gets to view the system in a very similar way as the single-line diagram or the local substation HMI, which he is already familiar with.
The method proposed is suitable for testing the SAS during its entire project lifecycle, which project phases are described at IEC 61850-4 and illustrated in Figure 3. The tool using this method should support both monitoring as well as simulation of the system. When testing, the test set should have access to the GOOSE network traffic and a MMS connection to the IEDs.
During the specification phase, the SCD file, the signals and communication services can be validated without the need of any physical device. Later, testing of SCADA gateways and HMIs can be performed by simulating the communication behavior and signals of all IEDs – again without any real IED. During the FAT, IEDs that are not yet present can be simulated to test those ones which are already available. As the project moves into the commissioning stage, more monitoring and testing of the real IEDs is done instead of simulation. One of the key factors for an efficient approach is the option to create test plans. A test procedure can be documented and reused throughout the SAS lifecycle. Test sequences can be performed and assessed automatically. Some test cases related to the SAS system are discussed in the following sections.
Verification of Communication Links: By loading the SCD file and having access to the network traffic and MMS connection to the IEDs, the testing tool can automatically validate all GOOSE, Sampled Values and Report communication links.
The test set can poll for attributes in the IEDs and validate against the model. It can check, for example, if the Report Control Blocks are Enabled and if the Owners of the Reports are the Clients declared in the SCD file. GOOSE communication links can be verified for:
- GOOSE mismatch on the sender side: by verifying Control Block settings
- GOOSE publishing errors: by sniffing on the network and comparing against SCD
- GOOSE subscription errors: by verifying the LGOS statuses at each subscribing IED. Mismatches are also checked
Figure 5 illustrates an example where the GOOSE published by an IED is verified in the network, but a problem is identified at one of the subscribers due to a mismatch in the configuration revision. The connection link is highlighted in yellow and warning signs are displayed to indicate the issue.
Testing Interlocking Logics: Logic is implemented in IEDs to cover many automation functions. They can automatically be tested using this approach by simulating the inputs of the logic (either via IED simulation or real switchgear status) and the result of the logic can be assessed. One application example is the use of logics for interlocking schemes to ensure proper operation of disconnect and grounding switches. To represent the result of interlocking logic conditions, IEC 61850 defines the status of the release in the logical node CILO. For testing, a subset or ideally all possible combinations of inputs can be tested, and the logic output assessed by automatically reading the CILO status values. (see Figures 6 and 7).
Troubleshooting by Tracing Signals: There are multiple transfers of messages and signals within a SAS system. A signal passes multiple steps until it arrives at the control center. If there is an error in this communication, the commissioning engineer needs to follow the signal on its way through the SAS. Finding such signal errors can be very time consuming. Using the test method described in this article, it is possible to follow how the signals propagate through the SAS.
Testing RTU/Gateway and local HMI Configuration: Gateways, RTUs and local HMIs usually communicate with almost all IEDs in the system, mainly via Reports, but also GOOSE. Typically, several thousands of signals need to be tested. During commissioning, at least the most critical signals are tested point-to-point by stimulating the signal in the switchyard. All other signals can be simulated by a testing tool. A test plan can be built with the testing tool simulating all IEDs and signals of the substation for a quicker verification.
Gateways/RTUs, HMIs and other IEDs in general are often exposed to firmware updates and security patches during their lifetime. The devices can be easily re-tested (sanity check) after the update by executing the test plan already prepared for that device before it is put back into operation. Those tests can be performed in the substation with all other IEDs simulated by a modern test tool without affecting the devices in operation.
Practical Use Case: Extension of an existing substation
For an important 20 kV Indoor Substation with around 55 IEDs, 3 busbars and two bus sections in a large industrial complex, the owner decided to extend the existing substation with several additional bays. The substation was commissioned around 10 years ago with a modern PAC system based on IEC 61850. Due to the operational constraints, the extension must be performed without de-energizing and during operation of the substation. The command interlockings have been implemented by PLC functions in the IEDs and GOOSE is used for exchange of the relevant signals between the IEDs.
The bay-related interlocks are implemented in the respective bay device. In addition, a dedicated station interlocking IED calculates the station-wide interlockings (see Figure 8). To realize this, the bay devices send their switch positions and other information via GOOSE to the interlocking IED, which calculates topological information such as “busbar 1 grounded” and sends this information again via GOOSE back to the bay devices, where the final command releases are calculated. Benefit: If the central device fails, the bay-related interlocks remain available. And most important: Existing bay IEDs are not affected in case of extensions of the substation!
This way of implementation allows subsequent extension without retesting the existing bays and – when using modern testing tools – also during operation.
Factory testing of the new IEDs: As the majority of the IEDs are already in operation, the IEDs for the new bays could not be factory tested together with the existing IEDs and station level devices. Therefore, the owner decided to perform the test of the new IEDs with a spare Interlocking IED and the rest of the substation to be simulated by the testing tool (see Figure 9).
At first, the engineer imported the existing SCD file into a new project data base, added the new IEDs, updated the central interlocking IED, the HMI and the gateways in order to incorporate the new bays and finally created a new SCD file of the entire substation (existing + extension). The existing bay control and protection IEDs remained unaffected and will not be loaded with new parameter files.
For each disconnector’s open/close operation in the new bays, test cases with >50 test steps have been defined as a permutation table in spread sheet (Figure 7) and implemented in the test tool. The test cases have been created just once for a typical bay and easily copied for the others. Finally, those test steps have been executed by simulation of the existing IEDs and assessment of the relevant CILO data objects in the new IEDs (Figure 6). This way, the correct implementation of the interlocking scheme in the central interlocking IED as well as in the new bay IEDs has been verified.
Testing of the updated gateways: A second part of the extension project involves the update of the existing gateways with latest CPU hardware and firmware for cybersecurity reasons. Considering the extensive evolution of hardware and firmware during the past 10 years, a complete retest of around 2.000 signals from the IEDs to the Control Center after the update was highly recommended.
As the station is equipped with redundant gateways, one of the two gateways can be disconnected from the Station LAN without disturbing remote control from the control center. Each gateway will be upgraded, and a complete signal test will be performed by simulating all reports and SCADA signals with the test tool, verifying the correct function of the gateway up to the control center.
Conclusion
An innovative test approach was presented for testing the communication, automation, control and SCADA part of the SAS system, which is based on the SCD file information. Test plans can now be created to automate the test and document procedures that have been very time consuming until now.
Automated test plans also enable a quick re-test after security patches and firmware updates, which are performed quite often nowadays. Testing is becoming an integral part of the system and quickly evolving into a supervision and monitoring role.
Biographies:
Christian Brauner received his engineering degree at the polytechnical college St. Pölten/Austria in the field of communication technology. In the last 30 years he was employed by manufacturers of substation automation (SAS), control, protection and SCADA systems for electric utilities. He started his career as a technical sales expert in Austria with VA TECH SAT (now Siemens), later as sales manager with SPRECHER AUTOMATION. He has been involved in a variety if IEC 61850 related projects and frame contracts for large utilities all over Europe.
Since 2018 he has been working for OMICRON Vienna as sales development manager. His special focus is on testing and monitoring products and solutions for substation automation systems.
Eugenio Carvalheira received his B.Sc. in Electrical Engineering at UFPE in Brazil and his M.Sc. in Computational Engineering at the University of Erlangen in Germany. He has over 18 years of experience in the design and commissioning of power systems protection, automation and control systems. He joined OMICRON in 2008 as an Application Engineer and is currently Engineering Manager for North America based in Houston, TX. He is also an active member of IEEE PSRC working groups.
