Distributed Control for DER and Distribution Automation

by Maik G. Seewald and Jeffrey D. Taft, Connected Energy Networks, Cisco


These newer functions are mostly well-known and include items as:

  • Flow control for Fault Isolation and Service Restoration
  • Distributed Energy Resources integration, including Distributed Generation and Distributed Storage
  • Responsive loads (command, price, and /or system frequency)
  • Integrated Volt/VAr control for voltage regulation, unity power factor
  • Inverter control for fast VAr regulation
  • Variable Energy Resource integration (transmission level)
  • Local energy network and microgrid power balance, DG power sharing and flow control
  • Demand Response and Virtual Power Plants
  • Local energy market interactions for small to medium prosumers
  • Transactive Energy
  • Load modulation of buildings, electric vehicle chargers, and data centers for local balancing
  • Power electronic (fast) grid stabilization
  • Energy services integration

Not every utility or microgrid employs all of these functions; rather they are being deployed in various subset combinations at different rates depending on specific utility and community needs and local or regional regulations.
These new functions introduce problems in the management of power grids, especially at the distribution level. Some of these problems have been recognized for some time. They include failure or protective schemes, mis-operation of voltage regulators, and power quality problems such as voltage instability in the presence of power fluctuations due to solar electric feed-in at the feeder level.

Emerging Grid Operational Issues

Emerging Grid Operational Issues
Power flow complexity at the distribution level and increasing need for electronic stabilization at both transmission and distribution levels are additional problems that come for the same set of new functions and grid changes. Much of the problem stems from coupling of otherwise apparently siloed systems through the operation of markets and electrical physics of the grid. This effect is unavoidable and is the source of many difficulties in grid management when new functions, especially distribution automation functions, are deployed at scale without new control measures being put in place. It is important to understand in more detail the changing service requirements for electric grids under the current utility industry transition. The following three issues highlight the significance of the changes on control and operational systems:

1. A consequence of the retirement of older fossil fueled generating resources and increase of VER/DER resources as part of the portfolio may result in a net decrease of rotational inertia and therefore grid stability. This is particularly problematic in areas with remote wind and solar PV resources and retirement of large steam turbine based generation near load centers. This reinforces the need for algorithms for fast dynamical control to ensure grid stabilization at transmission and distribution levels

2. The concept of transactive control, where customer premises or individual elements in those premises may interact with energy and power markets on a programmed basis, effectively puts those markets inside the power grid control loops as shown conceptually in Figure 1. This raises two issues: one is that price responsive loads may cause price and grid instability, and the second is that they may cause “flash crashes” in energy and power markets, in a fashion similar to what can happen to stock markets with programmed trading. Ordinary grid control systems and design methods do not address these issues.

3. Much has been written about the problems that arise in power grids due to reverse power flows and other behavior caused by various subsystem interactions and by use of the grid in ways not foreseen when the grids were designed. The net result of these emerging trends is that older control systems do not have the capability to manage the grid properly when penetration of variable DERs reach levels envisioned in public policy. It is quite possible for smaller scale adoption of DER on a circuit work adequately, but the real problems are only revealed after larger penetration levels have been reached.

Item 3 touches on a key issue – hidden coupling through the electrical physics of the grid. This coupling always exists and increasingly the import of such coupling is being seen. On a small scale (Proof of Concept and Pilots) the deleterious effects of such coupling are often not evident. However, as scale increases, the problems become manifest. Examples include the unfortunate interactions between Volt/VAr control and demand response (voltage violations and breaker trips), interaction between Conservation Voltage Reduction and distribution level solar PV (voltage instability at the feeder level), and the interactions of responsive loads and markets (price and control instability). Utility experience has shown that as power flow from renewable sources or under secondary control exceeds 15% of the total power on a feeder, such effects become increasingly problematic, even severely so.

Another emerging issue is the increasing speed of dynamic behavior on distribution grids due to the penetration of new grid functions and devices. As newer devices come online, dynamics move from the seconds to minutes range, down to sub-seconds, even as low as a few cycles in the case of active regulation using DER inverter control for VAr control (Figure 2).

The upshot of the foregoing is that power grids are becoming uncontrollable. This situation has certainly not gone unnoticed by utility engineers and executives. It now represents one of the major risks that utilities have on their radar screens, and many have realized that the viral nature of solar PV penetration and the other emerging trends are on a trajectory that the utilities cannot avoid or limit. Measures are being tried by utilities and by equipment suppliers to deal with some of these issues. The utilities, being pragmatic as always, have begun to apply the tools they have available, namely their existing control systems, to control these new devices, systems, and grid phenomena. Manufacturers have begun to offer new specialty control systems such as Distributed Energy Resource Management Systems (DERMS), to be operated in parallel with existing grid controls. In some cases, control is being extended across multiple grid hierarchy tiers (e.g. directly from qualified scheduling entity to ends user responsive loads), causing tier hopping and causing loops to be closed around multiple tiers. In other cases, additional controls are not coordinated with existing controls – in some cases control is actually delegated outside the control regime of the power grid via third party aggregators who bid into energy markets. The ad hoc nature of these efforts is leading to architectural chaos in control frameworks for power grids.

This has not yet been cemented via major investment, but if that should happen it would be very difficult to unwind an approach that will limit the industry’s ability to achieve the goals being set . Clearly, changes in power grid control architecture and methods are in order. In the past, power systems have sometimes been referred to as a “graveyard of advanced control theories.” In fact, existing grid control systems are very well designed for their intended purposes and advanced control mechanisms were not necessary. But keep in mind that existing grid controls were designed with the following assumptions:

  • Generation is dispatchable
  • No significant energy storage
  • Power must be kept in balance Generation follows load
  • Distribution system dynamics are relatively slow
  • Distribution simply “floats” on Transmission and can be treated as passive aggregated load

The penetration of DER (wind and solar), responsive loads, energy storage, market mechanisms, and the rise of microgrids and local energy networks violates all of these assumptions, which is both an issue and an opportunity.
There is an emerging recognition that the next generation of grid controls must be distributed in form. Existing controls are largely centralized, and often include a human in the loop.

New controls must move the human outside the loop to a supervisory role, like the “fly-by-wire” controls for aircraft, where multiple embedded controllers accept input from the pilot and then control the aircraft to carry out the instructions while continually stabilizing the aircraft.
The “fly-by-wire power grid” will accept input from human supervisors, and then operate the grid via distributed controllers while stabilizing it cary.

It is important to be clear on the difference between decentralized control and distributed control. In a decentralized grid control, controller elements are placed in geographically dispersed locations, but individual elements operate independently of each other. By contrast, distributed control has a crucial additional capability: the dispersed elements cooperate and communicate with each other on solving a common problem. This additional capability has significant implications for control algorithm design and control system communication network design, architecture and operation.
Distributed control offers compelling benefits over centralized control in utilities environments. These include:

Problem Complexity Decomposition

  • Distribution allows complex problems to be broken into smaller parts which are easier to solve using multiple processors, thus providing built-in scalability
  • Distributed implementations also facilitate modular incremental rollouts that grow appropriately and automatically as the system grows or control deployment proceeds

Temporal Alignment

  • Distributed intelligence architecture can align the operational timing needs of specific control applications with related data sources and processing, such as enabling low latency response to an event by processing data locally and providing it to the end device without a round trip back to a control center
  • Low sampling time skew can be achieved through multiple data collection agents and can easily minimize first-to-last sample time skew for improved system state snapshots compared to round robin sampling


  • No single choke point for data acquisition or processing; analytics at the lower levels of a hierarchical distributed system can be processed and passed on to higher levels in the hierarchy.
  • Problems get solved closer to the source (local) which addresses the need for granular control and quick adaptive response


  • Local autonomous operation is easily supported
  • Continued operation in the presence of communication network fragmentation is possible
  • Graceful system performance and functional degradation in the face of device and subsystem failures is achievable
  • Incremental rollout can easily be accomplished if the underlying software supports dynamic topology and zero touch deployment
  • Redundancy mechanisms between distributed subsystems are possible and foster the overall stability of the system
  • Better resilience and survivability of the entire system in case of a cyber-attack provided by isolation and redundancy mechanisms





Distributed processing also brings issues of its own, such as:

  • Device/system/application management - smart devices residing in substations, on poles, in underground structures represent significant cost to visit. Remote administration of such devices is necessary. This implies remote monitoring of the databases and applications, along with the means to reset or upgrade remotely
  • Harder to design, commission, and diagnose - distributed intelligence systems can inherently involve a larger number of interfaces and interactions than centralized systems, making design, test, and installation more complex
  • Complex communications architectures required - distributed intelligence involves more peer -to-peer interaction, so that the communication network must support the associated peer -to-peer communications

To handle the complexity of distributed control and to achieve interoperability between vendors, existing standards need to address the underlying requirements. IEC 61850 provides an excellent platform to handle this evolution in control, from the modeling and communication perspective. The paradigm of modeling of functions independently from its allocation is an excellent foundation. Extensions to existing data models, as well as enhanced communication and engineering capabilities might be necessary to describe and model distributed control comprehensively, especially emerging domains such as DA, and DER/DG.

Before looking at a solution, let’s consider the key requirement that address the issues discussed above.



Requirements for Highly Penetrated DER & Advanced DA

Requirements for Highly Penetrated DER & Advanced DA
New power grid control systems must have a number of capabilities and characteristics, some of which are not present or are not uniformly provided across the power delivery chain. These include:

Federation – since a modern grid control system must support multiple objectives, it is necessary for the grid control architecture to provide an inherent mechanism for support of federation of the controls so that they work in a coordinated fashion, as opposed to clashing, while retaining a significant degree of internal autonomy. This mechanism should work across both system and organizational boundaries

Disaggregation – macro-level commands, such as for a large amount of demand response to be achieved over a service area, must be decomposable to appropriate pieces at each succeeding level of the grid hierarchy until reaching endpoints. This is so that each level can apply constraints visible at that level to maintain grid manageability at all levels and across system and organizational boundaries

Constraint fusion – the new control function involves a great many constraints, often differing at various levels in the hierarchy, so the macro control architecture must support a means to fuse complex and wide-ranging constraints into control solutions

Agility – since the grid of the future will undergo almost continual transition, and experience wide dynamic state variations and various failures, the control systems must be capable of dynamic adaptability in both reaction to normal operating conditions in a world of stochastic generation, responsive loads, and market interactions, but also in a world where maintenance of normal operation is desired and expected in spite of device and system failures. Flow reconfiguration, stabilization and regulation across discontinuous failure events, and tolerance of unpredictable market behavior are all desirable

Boundary deference – given the nature of the organizations and systems used in grid operations and given that DER is causing more interaction with Transmission and even with Qualified Scheduling Entities such as ISO’s, it is necessary that the boundaries between systems and organizations be respected by any control framework that inherently spans them. A layered decomposition technique makes it possible to specify control boundaries that align with the pre-existing system and organization boundaries (Figure 3)

Stability criteria inclusion –the need for control system stability is obvious but existing grid control methods do not address such issues as how to incorporate stability criteria into controls that use market-like mechanism for example, or how to provide explicitly for stability in the presence of multiple control objectives and multiple processes coupled via the grid itself

Coordination– this is a technical term from control engineering that refers to mechanisms to keep various parts of a distributed and/or hierarchical control system aligned to and cooperating on common objectives. There are a number of mathematically rigorous methods for doing this but there is very little such coordination in present grid controls, as this has not been necessary in the past. The emerging trends imply a need for a formal coordination mechanism

Legacy integration – as a matter of practicality, any new form of control must be able to integrate with legacy controls since they will be in place for quite some time.

These requirements can be met in a comprehensive and rigorous way using distributed control coordination methods (known since the 1970’s) with digital communication architectural concepts and tools to arrive at a practical approach for advanced grid control.

Laminar Control: An Emerging Control Architecture
There is a long standing relationship between distributed control and optimization. Many distributed control problems have solutions based on optimization theory dating back to the 1970’s. We have also seen the emergence of new optimization methods, and in particular the primal-dual decomposition combined with Network Utility Maximization (NUM), which was originally developed for congestion control in communication networks, but which has application to multi-layer optimization.

Starting with the Network Utility Maximization formulation, optimization problems for grid control may be decomposed into layers that match hierarchical grid layers. By applying system level control criteria and constraints at the upper levels, and then allowing the lower levels to optimize “selfishly” within the bounds set by the upper layers, we can arrive at a macro control framework that encompasses both traditional and emerging control functions and models and allows for incremental transition from fully centralized to variable topology distributed control structures. We refer to this as Laminar Control. Figure 4 shows the two main methods for performing the layer decomposition, and illustrates a three layer decomposition.

The approach can be applied to as many tiers as is required, so that tiers can be defined as necessary. Individual control points may be in control centers and substations, or may be embedded in different devices such as DER unit controllers, or even household appliances. Figure 5 shows an example of mapping the layered optimization decomposition onto a power delivery infrastructure.
At each level in the multi-layer optimization, the appropriate organization, system, or device solves its own optimization problem, but in accordance with signaling from the next upper layer in the form of resource allocations or price signals. Therefore, at each layer there is autonomy of function within bounds that ensure stability and security for the system as a whole.
This framework provides the means to properly integrate new functionality in a rational way and enables both centralized and distributed implementations.



Maik G. Seewald has over 20 years of engineering, security, and technical architecture experience, and focuses on power grid automation, smart grid architecture, cyber security and business development for Cisco. He is Cisco’s representative for communication, security and energy automation. He participates actively in standard development with the focus on IEC 61850 and IEC 62351. Previously, Maik was a senior research and development architect and CISSP for Siemens, and an IT architect at T-Systems Multimedia Solutions. Maik received a degree in Informational Techniques and a Qualified Engineer degree from Dresden University.

Jeffrey D. Taft, PhD As the Cisco Connected Energy Networks Chief Architect and Distinguished Engineer, Jeff is responsible for the Cisco GridBlocks™ Reference Architecture. He directs the Business Unit’s research activities. He had smart grid chief architect roles with Accenture and IBM, and worked for Westinghouse on several key smart grid projects. He earned a PhD in Electrical Engineering from the University of Pittsburgh with a dual specialization in digital signal processing and control systems. He is a member of the IEEE PES, GridWise Architecture Council, and CIGRE, and is the holder of 17 patents.


BeijingSifang June 2016