WASA and the Roadmap to WAMPAC at SDG&E

by Eric A. Udren, Dr. Damir Novosel, Dan Brancaccio, Quanta Technology, USA, and Tariq Rahman, SDG&E, USA

San Diego Gas & Electric Company (SDG&E) is among many users who are deploying Wide-Area Situational Awareness (WASA) systems in which PMUs distributed across the grid stream data to control centers for situational awareness of dynamic wide-area behavior of the grid to system operators, as shown in Figure 1. 

The WASA deployment is evolving from an earlier grid observation system built over the last decade, now extending to most transmission lines and other system elements.  The existing system serves as a tool for engineers to observe system behavior, and to perform post-mortem analysis of disturbances and operating events.  These uses are fundamental but are just a part of a broader roadmap for many important new applications using PMU measurements, as we will explain. 
Over time and across the industry, synchrophasors will serve as the new unified wide-area data-gathering platform serving system observation, operating data collection, control, automation, market functions, asset management and maintenance support, and even high-speed wide-area fault and failure protection functions that supplement and back up local fault protection.  A full-function PMU-based deployment is described as a Wide-Area Monitoring, Protection, Automation, and Control (WAMPAC) system.

WASA and WAMPAC Functions

System Operator Situational Awareness: WASA includes control center displays for operators – voltage and phase profiles and variations, stability warnings, and normal versus unusual flow patterns.  Figure 2 shows an example.

Future SCADA and EMS Support: SCADA and EMS and familiar legacy grid-monitoring production systems on which operators rely have been separate from PMU-based WASA systems, which have been typically placed alongside and not incorporated into these systems.  However, we note that PMUs installed on all lines and apparatus comprise a whole new parallel data-gathering platform.  In place of transducers, RTUs, and IED data concentrators sending uncorrelated magnitude measurements every few seconds, the network of PMUs are streaming ac values with high accuracy at 30 or 60 (or even 120) value sets per second, with exact time and angle relationships across the grid defined by precision synchronized time tags in each value set.  PMUs gather phase and sequence values of voltages and currents, power flows, frequency, and rate of change of frequency (ROCOF).  Binary and analog point values can be included in the data streams. 
With the addition of a control messaging transport service on the same network, a dense PMU infrastructure can be the future data platform for EMS and SCADA.  All of today’s EMS and SCADA applications will perform equally well or better with the vastly richer, denser, and more precise sources of data.  In fact, some traditional EMS are now integrating down-sampled synchrophasor measurements.

Disturbance Monitoring and Analysis: WASA also includes the functions of Disturbance Monitoring Equipment (DME) used today for post-event analysis of system behavior and disturbances.  Records of streamed measurements – the envelopes of the raw power-system signals tracked and recorded continuously with frequency and ROCOF – yields records of disturbed-system oscillations, swings, and propagation of disturbance fronts across the system.   These records are very useful to understand system outages (including wide-area blackouts) and operating problems but are also used to validate system models as the response of the model to the monitored triggering event is compared to real system behavior.  Some system analysis tool suppliers are working on automation of model tuning from event data.  Good models, in turn, are key to accurate and safe determination of operating conditions and limits. 

System State and Condition Monitoring:  Conventional EMS state estimation aims to adjust erroneous measurements and fill in approximations of missing values.  A dense PMU-based data-gathering system can go far beyond this, reporting its holistic assessment of power apparatus states and pinpointing inaccurate measurements for improved state estimation and alarming of failures.  Using related adjacent or redundant measurement comparisons, the condition-monitoring function of a WASA or WAMPAC system to substitute accurate values and to dispatch maintenance activities.

Automated Wide-Area Remedial Action or System Integrity Protection Schemes:  The tuned, accurate system models serve as a basis for Remedial Action Schemes (RAS) or System Integrity Protection Schemes (SIPS) which can be carried out on the WAMPAC platform by including a high-speed wide-area control messaging service.  Today’s RAS and SIPS are typically based on planning studies of potential system failures, carried out in advance, and may not work as required if the system is in a different state than that used in the study. 
With real-time, high-speed state measurements, the protective scheme can respond directly to disturbance indicators like phase angle across a transmission corridor, rather than only on a load flow higher than a set index value after a line trip.  Separately, other advanced users are already demonstrating the capability of combining multiple RAS in a single centralized system so logic is easily updated and so that behavior for complex events is coordinated (e.g. Southern California Edison Centralized RAS or C-RAS.)

Fire and Hazard Risk Reduction: SDG&E has been deploying, for several years, installations of its distribution Falling Conductor Protection (FCP) system based on PMU measurements along the protected circuit communicating synchrophasor voltage and current streams back to a substation phasor data concentrator (PDC) and a controller platform.  The phasor data collection and control system comprises a distribution-scale WAMPAC. The controller is programmed to analyze voltage and current phase angle relationships across the length of the circuit and can detect if a phase conductor has broken and is in the process of falling.  The FCP system detects and locates a conductor break in a typical time of 200 to 300 ms and sends trip commands to circuit switching devices on both sides of the break via IEC 61850 GOOSE messaging. 
A typical distribution conductor at a height of 30 feet or 9 m takes 1.37 s to reach the ground; FCP deenergizes the failing conductor by the time it has fallen about 4 feet or 1.2 m.  A difficult-to-detect high-impedance ground fault is completely circumvented; deenergizing the conductor while still high in the air avoids the risk of human contact or vegetation fire ignition.  The FCP concepts are being more broadly deployed across the SDG&E transmission grid.

Wide-Area Fault and Swing Protection:  A recent work has addressed how phase-current synchrophasor streams gathered from a high-density PMU deployment can be compared in a wide-area current differential scheme, detecting transmission faults a few cycles more slowly than local high-speed primary relays, but much faster than intentionally-delayed remote-zone backup relays. 
A centralized PMU-based backup fault protection controller observes current summations across the region and backs up local relays.  After observing prolonged fault duration, the backup scheme surgically removes faults left by failed relays or stuck breakers, tripping only the breakers required to deal with the specific local protection malfunction.  The scheme does not inhibit the performance of local high-speed relays and isolates the fault with less time delay and more specifically focused tripping than would be carried out by distance backup relays. 
Non-communicating distance relay zones can be left in service as a safety net – they will not get a chance to operate while the PMU-based wide-area protection scheme is in service.
Wide-area synchrophasor-based backup protection is sensitive and effective even for systems with low or unpredictable fault current caused by major penetration of inverter-based generation, and it does not trip for stable or unstable system swings.  Voltage phasors, which play little or no role in wide-area fault protection, can be used to detect swings to execute out-of-step tie tripping or islanding schemes with predictable and planned separation points.

Wide-Area Voltage Stability Management: PMUs have been incorporated in various voltage stability monitoring and contingency analysis tools and deployed by users such as SDG&E. Voltage stability issues often occur as a result of an increase in load demand. If the increase is slow, an EMS model-based Voltage Stability Assessment (VSA) will detect the problem and alert operators.  If faster, dynamic voltage instabilities occur, PMU-based VSA yields quicker and more accurate detection of the potential unstable contingencies requiring faster response. 
Furthermore, if the unforeseen system instability events develop very fast, measurement-based voltage instability detection tools, such as Real-time Voltage Instability Indicator (RVII), could be a complement to model-based VSA tools, detecting active power margins and voltage instability indicators in real time. RVII tools do not require a complex power system model; they are thus not as sensitive to model inaccuracies and can detect voltage instability in less than a second. These tools are also suitable for implementation in SIPS, or instead of undervoltage protection in simpler cases. RVII is able to identify unstable cases even in the case of high system voltage (for which undervoltage load-shedding relays would not operate),and distinguishes stable cases in the presence of very low voltages (for which an undervoltage relay should not operate).

Wide-Area Control: Wide-area holistic control of the grid becomes feasible when the PMU system includes mission-critical high-reliability data collection, high-speed control communications, and accurate system models to predict the result of control actions.  Voltage profiles can be adjusted in response to variable renewable generation via integration of measurements across a wide area and evaluation in a control model, to achieve the best or safest overall result rather than optimizing for one measurement point or a fixed combination of measurements.  Fixed and variable (FACTS) reactive devices can be continuously switched or adjusted for the best holistic profile.  Load flow control and dynamic stability enhancement are expected in the future.

PMU System Self-Monitoring:  While many users do not think of self-monitoring of a protection and control system as a power-system application, it is critical to a sustainable deployment, and supports the mission-critical roles that a WAMPAC system will serve.  The complete PMU system, whose components are reviewed below, is built from self-monitoring IEDs, controllers, and communications channels using repetitive or heartbeat communications services whose integrity can be monitored and alarmed.

The overlapping of the IED and communications service monitoring means that an entire WAMPAC system can monitor its integrity and accuracy in every branch, alarm specific failures for maintenance attention, and engage backup measurements from available data.  Figure 3 shows a contemporary display of PMU and PDC data stream availability.  Maintenance testing is never required, except for peripheral breaker tripping, control, and status input interfaces which can be validated by remote control actions or system state observations.  If protective relays with PMU measurement capability are used, the fault protection operations and protection system maintenance checks of these interfaces can serve WAMPAC P&C with no additional maintenance.

PMU System Infrastructure

Figure 4 shows a typical PMU system architecture.  Elements of the system include:
Phasor Measurement Units (PMUs) - stand-alone PMUs are available, and PMU functionality may be included in other measurement platforms such as digital fault recorders.  Today, many protective relay types from multiple manufacturers are available with synchrophasor data streaming capability.  The advantage of using relays is that an array of status inputs and breaker trip and control outputs are also available, including communications protocol support that makes the relays available for high-speed wide-area control execution.  Also, relays are installed on all system apparatus, often in redundant pairs, so that a complete and redundant set of measurements can be available for the full range of future WAMPAC applications.  All phasor measurements are performed in compliance with synchrophasor standards – the latest single international standard is the joint IEC-IEEE Standard 60255-118-1 Edition 1 of 2018.  Widely used in existing PMUs is IEEE C37.118.1-2011, or the measurement specifications in the older IEEE C37.118-2005.

Phasor Data Concentrators (PDCs) - PDCs are data-handling platforms that can merge data streams from many PMUs in a substation into a concise data format for transmission to the control center or remote location over a single communications channel.  PDCs can locally archive recent data for post-mortem analysis when the communications path fails, although this may become less important with redundant high-reliability data transmission required for WAMPAC functions like backup fault protection.  To avoid a single point of failure, a substation installation will include at least two PDCs as well as redundant isolated PMU sources for critical points, and separate isolated communications connections.

Communications network - WAMPAC requires robust reliable communications between PMUs or PDCs and control centers or central processing locations.  Many utilities are building these systems for teleprotection and other mission-critical operational needs.  Today, many PMU users are streaming serial PMU data over SONET (DS0 on T1 or higher-bandwidth) paths.  Some utilities, including SDG&E, are now deploying MPLS Ethernet wide-area communications networks with multiple redundant routers and a mesh of alternative communications paths leading to the central control and monitoring location of the WASA or WAMPAC system as shown in Figure 5.  MPLS can transport synchrophasor streams in Ethernet-packet or serial-simulation formats.  Transport latencies are dropping well below 10 ms and are well-suited to WAMPAC protection and control function implementation.  Redundant PMUs and PDCs operating over highly reliable redundant channels are practical today – it is feasible to invest to build the infrastructure to support WAMPAC.

Control center data processing and computing systems - Super PDCs collect single or redundant PMU data streams from each substation or field location. Each Super PDC can in turn supply customized data streams to multiple redundant computing platforms that isolate applications, as well as to remote clients such as neighbor utility system WASA/WAMPAC systems needing boundary information, or to independent system operators.  Redundant or dual tri-mode redundant computing platforms at the central control locations gather data, make evaluations and decisions, carry out protection and control via downward command messaging, generate displays and reports for operators and maintenance personnel, and archive data for disturbance monitoring or post-mortem analysis of system model accuracy.  Going forward, the synchrophasor data processing infrastructure comprises a private cloud computing platform whose components may not be physically co-located as is familiar today.

Communications protocols - most synchrophasors are streamed today in accordance with IEEE C37.118.2-2011, or the similar communications specifications of the older IEEE C37.118-2005.  Both serial and Ethernet transport options are supported, with the requirement that users map or configure data streams and transport paths manually.  WASA used by operators for decisions, or WAMPAC, require NERC CIP-compliant security via secure networking appliances or other data protection tools in an organization-wide security program.  While some utilities use non-routable serial transport as a cybersecurity protection means today, we do not expect this to be effective or acceptable in the future.  Solutions to these issues are now provided by the latest edition of IEC 61850-8, which includes the Routable Sampled Values (R-SV) service for synchrophasor transport using key-based authentication of packets for strong cybersecurity, and IT WAN standard services for automated detection and configuration of publisher-subscriber network connections.  Using IEC 61850-8 transport of synchrophasor streams calls for a new generation of PMU communications interfaces, and will yield easiest application, highest security, and compatibility with IEC 61850 function-level engineering methods that eliminate manual static point mapping. Other new communication protocols such as Grid Protection Alliance’s Streaming Telemetry Transport Protocol (STTP) are being developed to provide additional choices for delivering high speed synchronized measurements to client systems.

Time distribution - PMUs require precise synchronized time, accurate to microsecond levels or better across the region of time-tagging of synchrophasor measurements.  GPS satellite time receivers have been the standard tools that have enabled wide-area precise synchronization of time tags for regional observation, and thus raised interest in WASA and WAMPAC.  However, looking forward, GPS is increasingly vulnerable to disruption or attack as it supports mission-critical protection and control functions or operator decision-making.  GPS receiver manufacturers have greatly enhanced the robustness of the clock function against spoofing or disruption using local and system-wide monitoring algorithms and multiple satellite array reception.

Even more promising for security is the distribution of precision time over the same Ethernet WAN used for transport of phasor streams and control messages.  Redundant coordinated master clocks distribute time signals across the grid to PMUs using IEEE 1588 Precision Time Protocol (PTP) and power utility application profiles IEC 61850-9-3 and IEEE C37.238.  The master clocks can still be synchronized to GPS time, yet power-system-wide time synchronization continues even if the GPS reception is disrupted.  Redundant GPS synchronized clocks at each substation can prevent disruptions that are the result of hardware failure.  However, if the disruption is the result of GPS spoofing an additional trusted time source is required.  SDG&E is planning to deploy a network PTP solution as shown in Figure 6.

SDG&E Program

New and challenging grid operational reliability and safety concerns drive the advancement of synchrophasor methods at SDG&E, which has been gradually and continually building high-density PMU and data-gathering infrastructure for a decade. Real-time observation of the system operating state on basic client platforms aimed at P&C engineers has uncovered situations that were not visible to operators working on the legacy SCADA/EMS platform with its RTU and data-concentrator sources of low-speed unsynchronized data.  Engineers have also used the PMU data for post-mortem analysis of system events.  This visibility by engineering has shown the benefits of PMU data collection, raising the interest of system operators and leading to the deployment of a major control center WASA installation.  Figure 7 shows an example of a STATCOM oscillation problem that was not visible to system operators using SCADA data yet was causing customer service problems.

As WASA is established, engineers continue development of infrastructure and applications building toward the future of unified WAMPAC applications including those listed above.  Figure 8 shows SDG&E’s WASA and WAMPAC development roadmap.  As we explained above, fire risk reduction and improved system protection are among areas of development and deployment at this time.

Future Directions - Industry WAMPAC Roadmap Example
SDG&E has been following an industry roadmap for synchronized measurement system deployment which begins with basic infrastructure and familiarization and builds into WASA as it advances towards WAMPAC on a unified platform serving multiple users and mission-critical protection and control applications.  The roadmap steps are:
1.  Deploy a limited array of PMUs communicating data to engineers, to get practical first- deployment experience.  Observe transmission and/or distribution grid operation and share new findings.  Continue to add measurement sites as new relays are deployed.
2. Develop an organizational roadmap of use cases or applications, and required infrastructure to support them, with a sustainable investment plan.  Coordinate with and contribute specifications to the organization’s communications and IT investment and evolution plans.
3. Specify and deploy WASA in collaboration with System Operations, with a view towards evolution to WAMPAC.  Operators will develop skills and achieve the benefits of situational awareness, SCADA/EMS validation or enhancement.  Operators, engineers, and planners get the troubleshooting, DME, post-mortem analysis, and model validation benefits of WASA.
4. Deploy development and training platforms and environments, to be augmented going forward as new WASA and WAMPAC functions are integrated.
5. Tune system dynamic and short-circuit models as operating experience accumulates.  Develop and validate secondary-system modeling (P&C systems, instrument transformer and control connections, communications performance) for a variety of reliability-enhancement and reliability standards compliance purposes.
6. Develop and deploy supplemental non-redundant high-speed WAMPAC protection functions for trial use, WAMPAC system validation, and troubleshooting of reliability or data quality problems.
7. Build out fully-redundant PMU data collection and communications.  Manage performance to high levels of reliability.  Include timing distribution in the design.
8. Add P&C functions for faults, failures, instability, and RAS/SIPS scenarios.  These can be connected to trip after an initial operational validation.
9. Develop wide-area and high-speed control algorithms and test in simulations using tuned models, as well as on streamed real-time data.  Close the control loops to enter the full WAMPAC era.
10. The reliable and redundant PMU synchrophasor gathering system with return control paths is the new data platform onto which SCADA and EMS can now migrate to achieve a unified system for monitoring, control, automation, and protection.
11.  When deploying PMU applications, evaluate the use of synchronized measurements for distribution applications such as voltage monitoring and control, DER monitoring, or intelligent load shedding.


Eric Udren has a distinguished 49-year career in design and application of protective relaying, substation control, communications systems, and synchrophasor measurements in electric utility transmission and distribution systems.  He is Life Fellow of IEEE, former Subcommittee Chair and leader of multiple WGs at IEEE PES PSRC, and member of the NERC System Protection & Control Subcommittee.  Eric is Technical Advisor to the US National Committee of IEC for TC 95 relaying standards, and charter member of IEC TC 57 WG 10 that develops IEC 61850 standard parts.  In 2019, Eric was elected to the US National Academy of Engineering.  He is Executive Advisor with Quanta Technology, LLC of Raleigh, NC, and is based in Pittsburgh, PA, USA.

Tariq Rahman P.E., is a Principal Engineer at San Diego Gas & Electric Company (SDG&E), CA. Tariq has over 30 years of electric utility industry experience. He has been leading the Transmission Synchrophasor Project and has also initiated the Distribution Synchrophasor Project at SDG&E. Mr. Rahman is a licensed Professional Engineer in the states of New York and California and is a Senior Member of IEEE. He has presented papers and been an invited panelist at various conferences and seminars including DistribuTECH, IEEE T&D Conference, NASPI, etc.

Dr. Damir Novosel is president and founder of Quanta Technology, a subsidiary of Quanta Services, a Fortune 500 company.  Previously, he was vice president of ABB Automation Products and president of KEMA T&D US. Dr. Novosel is member of US National Academy of Engineers and served as IEEE PES President and Vice President of Technical Activities. He is also a member of the CIGRE US National Committee and received the CIGRE Attwood Associate award.  Damir holds 17 US and international patents, published over 140 articles, and contributed to 5 books.

Dan Brancaccio has over 30 years of experience in systems engineering and integration. He has developed large-scale, mission-critical, enterprise applications. He developed a pilot project for Southern CA Edison for transformer health. He was the lead technical architect to the Western Interconnection Synchrophasor Program (WISP.) Dan is Executive Advisor with Quanta Technology, LLC and is based in San Diego, CA.