Deploying Multiple DA Applications

Author: Tony Burge, RuggedCom, USA

Multi-Application Requirements

Networks are often driven by one application, but once in place these networks are often required to support many others. In the case of utilities, distribution automation applications provide the most immediate efficiencies; however, other applications, such as video surveillance/monitoring, voice services, and field force automation are desired to provide additional operational efficiencies and grid robustness (Figure 1.)

Throughput: This article is not written with the intention of providing full capacity modelling, rather, it provides general guidelines of the most critical requirements, of which throughput is one. Surveillance-quality video throughput requires approximately 2 mb/s using MPEG-4 compression. Lower resolution, lower frame rate video monitoring can be accomplished using as little as 256 kb/s, which still exceeds many wireless communications capacities. Most of this communication is uplink data.
SCADA polling, Volt/VAR control, Fault Detection, Isolation, and Restoration, and Capacitor Bank monitoring applications do not require substantial data - perhaps 10-15 kb/s each. However, when aggregated, the throughput for these mission critical applications can exceed 100 kb/s. Most of the communication for these applications is uplink data.

In certain substation environments, cell phone coverage is poor and a dedicated phone line is impractical or too expensive. Therefore, a Voice over IP (VoIP) line is often required to provide field engineers the ability to communicate with advisers at the back office. A good, toll-quality VoIP connection requires 64 kb/s. This communication equally splits between uplink and downlink. Finally, operational efficiencies can be extended by providing field personnel with wireless connectivity while performing maintenance - for work orders, access to agency Intranet for schematics and manuals, and more. This communication is mostly downlink data traffic and can easily require 500 kb/s. With this analysis, it is easy to infer the following throughput requirements per remote location (i.e., substation) and based on two video surveillance requirements: See Table 1.

Range: The range of the radio frequency (RF) technology and interference mitigation features implemented contribute directly to coverage. The more coverage provided by a solution, the less amount of infrastructure and capital expenditure is required. Range is determined by occupied channel bandwidth, power allocation, deployed frequency, and features such as Orthogonal Frequency Division Multiplexing (OFDM). There is often a trade off between throughput and coverage, so it is important to find the balance that provides the throughput required for multiple applications while maximizing the range (Figure 2.)

 

Latency: Latency, or response time, for power utility applications is measured in milliseconds for point-to-multipoint wireless communications. Mission-critical distribution automation applications often operate effectively with wireless technologies that support less than 100 millisecond roundtrip response time. Even the most efficient, transparent transceivers support at best 8 to 10 milliseconds, which is why point-to-multipoint RF communication is not used as a primary communication technology for generation and transmission protection and control applications that require a quarter of a cycle, or 4 millisecond response time to protect high valued equipment. (However, point-to-point RF communications may be used as a redundant communications technology for fiber in these protection and control applications.) A good target for round trip, point-to-multipoint latencies for distribution automation is under 40 milliseconds. This latency metric provides room for retries without adversely affecting distribution automation applications.
Security: Power utility infrastructure is arguably the most important asset to industrialized states, provinces, and nations. Protecting data that monitors and controls this infrastructure is a primary requirement for wireless networks deployed for such purposes. At a high level, data security can be discussed in three categories: Transmission Security, Network Authentication, and Data Segregation.

Transmission Security involves encrypting data at a transceiver prior to sending the message. The receiving transceiver then decrypts the message. This process protects the contents of the message as it is transmitted over the airwaves
Network Authentication - While transmission security facilitates the integrity of data as it is transmitted over the airwaves, network authentication is intended to prevent unauthorized access to the network itself, which is a vital component of network security. The unauthorized network access can result in denial- of-service, access to sensitive, utility-wide operational data, and manipulation of the data to interfere with the power grid itself
 Data Segregation - Within a physical network, administrators may require virtual networks to segregate data such that access to data can be limited to certain functional groups
Other Security Considerations - Other measures to be considered are intrusion detection features, tamper- evident/tamper-proof fabrication, and facility security

Prioritization: With multiple applications running over a single network, it is important that mission-critical data receives priority. Without such a mechanism, VoIP calls or field force network access could hinder fault detection notifications, out-of-tolerance voltage variation corrections, SCADA polling responses, and more.  IEEE 802.1P, Quality of Service (QoS), was defined to provide multiple levels of prioritized service. QoS provides bandwidth prioritization from the least-restrictive Best Effort to the most restrictive Unsolicited Grant Service. Additionally, QoS assists in defining service flows within each prioritization level to further improve the latency and jitter required for specific applications.
The appropriate point-to-multipoint broadband technology (advocated in this article), which balances throughput and range and provides prioritized service flows, has come under questioning regarding the lowest possible round-trip latency. A properly engineered broadband solution is able to demonstrate round-trip latencies less than 20 ms for applications requiring low response times. Table 2 is representative of round-trip latencies demonstrated in the field based on the latency requirements of the application.

Proprietary and Standards-based Protocol Support: The utility grid requires the support of legacy protocols that may have been deployed (and still in valid operation) decades ago. Key requirements for protocols are physical interfaces (e.g., DB connectors and RJ connectors) and protocol support for active and passive serial support (Modbus, Modbus TCP, DF1, and DNP-3), standards-based Layer 2 messaging (e.g., GOOSE messaging), and full TCP/IP Ethernet communications.

Uplink Biasing: Commercially-focused wireless communications solutions provide more downlink throughput than uplink. The concept of uplink biasing for power utilities is to dedicate more throughput on the uplink (from the substation to the back office.) Without a mechanism to support this uplink biasing, an organization may have 2 mb/s to a substation; however, less than 1 mb/s would be available for uplink communications.
At a minimum, a wireless broadband network for power utilities should support an uplink/downlink duty cycle of 70% uplink and 30% downlink, configurable.

Redundancy: Equipment redundancy with hot- or cold-standby assists in restoring communications when a transceiver fails. This becomes especially important when the locations of these devices are not easily accessible. Equipment provided for a wireless communications network should provide levels of redundancy at the master station/base station/access point and at the remote/subscriber unit to improve network reliability and availability.

Multi-Application Options

The various wireless technology options discussed will be compared (or contrasted) based on a subset of the requirements discussed previously (interface support, redundancy and robustness are manufacturing-specific, not technology-specific): Throughput; Range; Latency; Security; QoS (Prioritizaton); Uplink Biasing; Ecosystem

The two top levels of wireless technologies are public carrier or private infrastructure. Private infrastructure refers to utility-owned, closed-loop wireless networks.
Table 3 is a high level guideline to the benefits and deficiencies of each technology. This chart is the sole opinion of the author based on a decade of experience.

Public Carrier (PC): Public carrier infrastructure provides pervasive coverage in most populated areas, significant throughput, moderate security, and response times under 100 ms. By using approved devices it provides an ecosystem of products from various vendors. However, the concern remains whether public infrastructure can facilitate guaranteed prioritized service. Many carriers are working on features to support prioritized service-usually with a higher rate of pricing. If a utility is in an area with pervasive public carrier coverage with a guaranteed level of service at a fixed price, this would alleviate the initial capital expenditure of a private infrastructure. This is often not the case.  From a business case perspective, PCs focus on individual consumer-based users that make up the majority of the carriers customer base. Also of note is broken chain of custody for data in a third party network operations center and the embedded downlink biasing included in the technology (instead of uplink biasing).
In summary, public carriers have evolved to meet many needs of power utilities and require limited capital expenditure. They provide moderate throughput, pervasive coverage, acceptable latency, and adequate security features, but the need for additional security, channel availability during emergencies, and the ability to pass sufficient data in the uplink remain areas of concern.

Private Infrastructure: One new concept to this article is introduced at this point - deployed frequency. Frequency is not a feature necessarily, because each technology may be deployed within various frequency bands. As a general rule of thumb, and all things being equal, the lower in the RF electromagnetic spectrum a technology is deployed, the better the propagation and building penetration characteristics. For most wireless data communications, the range of point-to-multipoint frequencies available is 140 MHz to 5.8 GHz.

Narrowband - Often deployed in frequencies from 140 MHz to 900 MHz, a narrowband solution provides excellent RF propagation characteristics. Added to this is the very narrow channel size (hence the term narrowband) allotted by regulatory agencies (ETSI, IC, FCC, etc.)

The channel size allocated is usually limited to 25 kHz and often down to 6.25 kHz. While this narrow channel with high power allocations and operating in the lower portion of the RF electromagnetic spectrum provides excellent range, throughput is significantly limited - up to 50 kb/s. Even with excellent range, throughput is sufficient only for the most essential communications.
Since narrowband licenses are protected by regulatory agencies, the transceiver does not need to compete for channel usage, which provides very deterministic and low latency communications. With such low throughput, the ability to support higher level security and prioritized services are also limited, because these features require overhead  to operate. All narrowband options known to the author are proprietary; therefore, the ecosystem of products available is limited to a single manufacturer.

Mesh Wi/Fi - Broadband mesh technology is most often deployed in unlicensed bands with some deployed in designated bands for public safety or other entities. This technology is often based on IEEE 802.11 standards to some degree, which means that throughput is very high, but range is significantly limited. For a wide area network deployment, the limitation of range results in a high level of required infrastructure. Even though mesh technology facilitates node-to-node relaying or hopping, this comes at a cost: reduced throughput and increased latency at each node. Some manufacturers overcome this limitation by providing a multi-radio solution in a single box. This solution becomes more expensive with the same limitation of range (even though throughput and latency are improved). Each manufacturer implements different mesh algorithms, so the ecosystem of products is usually limited to a single vendor.

Broadband mesh technologies provide high throughput and sufficient security to support multiple applications; however, the amount of infrastructure required often makes this technology significantly higher to deploy than other technologies.

 Proprietary Broadband - This category refers to proprietary RF designs or modifications of standards, such as 802.11, that improve range in an attempt to balance throughput and coverage.  Proprietary broadband solutions have been successfully deployed for over a decade for power utilities. These solutions provide good throughput balanced with good range and acceptable latencies. Most are deployed in unlicensed or lightly licensed bands, which puts additional responsibilities on manufacturers to include interference mitigation features to facilitate deterministic communications.

Since these solutions are proprietary, the ecosystem of solutions is limited to a specific vendor for a specific implementation. Also, QoS is often implemented to a minimum level. Proprietary broadband solutions are field-proven; however, the proprietary nature locks a utility to a specific vendor and the throughput is, at most, adequate to support multiple services.

Broadband over Standards - Wireless broadband based on fully interoperable standards provides the benefits of high throughput, good range, high security, and often full implementation of QoS.  Broadband over standards, such as IEEE 802.16e, is deployed in licensed, lightly licensed, and unlicensed bands, so there is flexibility of deployment in areas where regulatory allocation of certain frequencies is limited.

Since this category is based on standards, the ecosystem of solutions available is larger and not limited to a single vendor per implementation (providing that appropriate interoperability testing and certification have occurred). In certain geographical regions, bandwidth is limited to such a degree that deploying these wider-band solutions (e.g., channel sizes of 3.5 Mhz) is not feasible. If frequencies are available, this category provides high throughput with highly secure, prioritized service and moderate coverage.

Technology-specific Considerations: There are many emerging features that facilitate advanced interference mitigation and improvements in throughput. The following list describes a few of these features the author recommends to research when selecting a wireless broadband solution.  A key point regarding broadband technologies that implement the features described in this section is the ability to sustain good range even when deploying with wider occupied channels (i.e., greater throughput).  With the advanced features described in this section, range (which is a function of receive sensitivity) is able to be maintained - this translates to high throughput with better range than available using other wide-channel, point-to-multipoint solutions.  Table 4 is representative of receive sensitivities (measured in dBm) obtainable using these technologies.

Scalable Orthogonal Frequency-Division Multiplexing Access (S-OFDMA): S-OFDMA facilitates sub-channel utilization (permitting a connected subscriber unit to use a subset of the occupied channel), the ability to access the same channel in adjacent cells or base station locations (providing more options in channel selection), and, most importantly, divides a large channel into multiple subcarriers to act as “narrowband” channels. Improving on OFDM technology, “Scalable” refers to keeping the subcarriers relatively small even when the occupied bandwidth is large. For example, the OFDM technology of 802.16d always used 256 subcarriers regardless of channel size. With S-OFDMA, the number of subcarriers can be increased to 512, 1,024, or 2,048; thereby, keeping the relative size of each subcarrier narrow. Implementing scalable subcarrier algorithms permits the use of wider channels (providing higher throughput) even in areas of in-band interference" (Figure 3.)

Forward Error Correction (FEC):  Sophisticated FEC algorithms are implemented to assist in rebuilding partially distorted messages on the receiver. Based on signal quality, very minimal FEC is incorporated, which provides higher throughput. Conversely, where signal quality is not as strong, higher order FEC is incorporated to facilitate successful message transmission. The FEC rates may be set to adapt automatically, along with modulation settings. If a portion of the subcarriers are distorted, FEC is often able to rebuild the message without requiring retransmission, which facilitates robust RF communications even in the presence of interference.

Hybrid Automatic Retransmit reQuest (Hybrid ARQ): In some cases, messages are unable to be successfully received or recovered via FEC.  In these cases, Hybrid ARQ provides a comprehensive retransmit request capability at the radio transceiver level without incurring the additional latency costs of waiting for a router in the network to perform the retransmit request. In short, Hybrid ARQ reduces latency and provides more deterministic communications in the presence of interference.

Multiple-in, Multiple-out (MIMO):  OFDM technology assists in minimizing the effects of multipath interference; however, MIMO adds the benefit of being able to take advantage of multipath interference to further enhance RF robustness.  MIMO Matrix A facilitates improved coverage by transmitting the same data on both transmission polarities. MIMO Matrix B facilitates improved throughput (where signal strength is sufficient) by sending unique data on both polarities. Advanced MIMO technology provides the ability to auto select the appropriate MIMO Matrix profile based on real-time signal characteristics. 

Biography

Tony Burge has nearly 20 years of technical, product management, and business development experience in mission critical wireless communications and software development including early Military Tactical Digital Interface Links (TADILs) to proprietary RF solutions, to Industrial broadband and WiMAX solutions. He received his Bachelor of Science degree from the University of the State of New York in Albany, New York, and earned 16 hours towards his MBA from Old Dominion University in Norfolk, Virginia. He has held senior positions with Comptek Federal Systems, VisionAIR, and GE MDS. Since May 2010, Tony has served as Business Development Manager for RuggedCom’s RuggedWireless Division.