by Chris Huntley, Schweitzer Engineering Laboratories, USA
The Big Picture
The picture on page 83 shows the delivery of time from the TAI timescale’s creator and maintainer, BIPM in Paris, to the power system’s substations.
Many countries have their own atomic clocks, allowing terrestrial time-distribution links to the substations; the alternate (much more common) approach being to use one or more of the global satellite (GNSS) systems, which though built for location applications can also provide sub-microsecond accurate time distribution.
Our Timescales
TAI (Temps Atomique International) is the smooth international time scale:
- Uses the SI second* for its ticks
- Aligned to UTC when started in 1958
- Maintained by BIPM (Bureau International des Poids et Measures in Paris)
As illustrated on page 83, BIPM uses the measurements of the SI second, as reported by over 400 atomic clocks in over 50 national laboratories worldwide. BIPM regularly issues its circular-T to inform each contributor of its offset from TAI.
e.g. see https://www.bipm.org/en/bipm-services/timescales/time-ftp/Circular-T.html#nohref
*The second’s duration of 1/86,400 of the mean solar day lasted from the Babylonians (3500 BC having a sexagesimal system for mathematics and astronomy), until 1967 when the SI second was created (to be 9,192,631,770 periods of the radiation produced by the transition between the two hyperfine ground states of cesium).
Universal Time Coordinated: UTC (Universal Time Coordinated) is the bumpy international time scale which keeps the sun at its zenith at noon, and the astronomers happy that their telescopes point within 15 seconds of arc:
- UTC is the reference for country local-times (offsets being the worldwide time zones)
- Uses the SI second for its ticks
- Uses Leap-seconds (since 1972) to mitigate the variations in the earth’s rotation
- Maintained by IERS (International Earth Rotation Society, also in Paris)
- Identical to GMT (Greenwich Mean Time)
IERS regularly issues its Bulletin-C regarding scheduled leap-second events.
e.g. see https://datacenter.iers.org/data/latestVersion/16_BULLETIN_C16.txt
The graph in Figure 1 shows the recent variations in the lengths of the day (about 30ppb), and the resultant accumulation of small adjustments (< 1972) and leap-seconds (red dots) in UTC; now totaling 37 seconds offset from TAI.
The last 50-year general speeding up of the earth’s rotation has certainly been a surprise to many, since tides and polar ice melting both cause slowdowns; redistributions of the earth core’s heavier components toward its center is a possible candidate.
Certainly, if the downward (green) slope of the last four years continues for a few more years we will likely experience the first-ever leap-second removal.
Substation Application Time Requirements
The myriad of substation applications spans a wide range of requirements for their time stampings; the substation time-distribution service must be capable of satisfying all the applications’ requirements.
UTC for SOE (Sequence of Events: The most obvious need for a system-wide clock is to be able to reconstruct the sequence of events leading up to a major system disturbance; the correct progression of participants from root cause to breaker openings requires that the recorded timestamps have an accuracy better than the propagation times between events.
In attempting to reconstruct the sequence of events causing the 2003 Northeast blackout (Figure 2) with the loss of 23,000 MW affecting 55 million people, NERC’s frustration with the SOE clocks (many had been free-running for years) prompted their issuing in 2006, PRC-018-1L which includes the requirement:
- R1.1 Internal Clocks in DME devices shall be synchronized to within 2 milliseconds or less of Universal Coordinated Time scale (UTC)
This time accuracy requirement is a good start, but an “attention to detail” is also required to mitigate the ambiguities typically seen in time stamp records; for this purpose, in 2018 the IEEE PES/PSRC completed the IEEE C37.237- “IEEE Standard for Requirements for Time Tags created by Intelligent Electronic Devices-COMTAG.”
Note: This IEEE C37.237 standard is also the new home for the IRIG-B control-bit assignments that originated in the IEEE 1344-1995 for Syncrophasor IEDs, updated in IEEE C37.118-2005 and further in IEEE C37.118.1-2011.
IEEE C37.237 is not just a new home; it is also now a normative part of a standard.
Absolute Time for Merging Units: Though intra-substation applications, like busbar protection, really only need an internal (to substation) clock; compliance to some emerging IEC standards has deprecated this simplicity; e.g. IEC 61869-9 for merging units specifies “the accuracy of the time signal (mean error from absolute time) is expected to be better than ±1 µs for accuracy characterization.” The ensuing concerns and needs for better understandings were some of the driving forces behind Cigre JWG B5/D2.67 which is developing the Technical Brochure on “Time in Communication Networks, Protection and Control Applications – Time Sources and Distribution Methods.”
Local-Time for HMI: The HMI for substation applications frequently requires events to be reported using the local timescale.
To support these, the time-distribution needs to provide, in addition to UTC, both the local time zone, and whether DST (Daylight Saving Time) is currently in effect (noted in IEEE 61850-5 clause 11.5.4).
UTC and Phase-Comparison Timeststamps: IEEE and IEC standards unfortunately continue to specify the UTC timescale for timestamps; and to add to the issues when needing numeric timestamps have often (e.g. the IEC 60255-118-1 for Syncrophasors) chosen the start of 1970 UTC as their epoch, when there was an 82 µs skew between the TAI and UTC second rollovers (as shown earlier, leap-seconds were not introduced until 1972).
As noted in IEEE 1588-2008 Clause 7.2.2: “The PTP epoch is 1 January 1970 00:00:00 TAI, which is 31 December 1969 23:59:51.999918 UTC.” Fortunately, as far as is known, all Syncrophasor IEDs have this same 82 µs error.
Arguably the most critical function in power systems has been, and still is, the comparison of the phases of power conductor currents. With emerging technologies (PMUs and SV) sampling these currents, a timestamp accuracy of 1 microsecond (for < 0.02-degree phase error at 50 Hz) is being requested for these applications (e.g. IEEE 61850-5 clause 11.4.1).
Though this 1 microsecond is a nice target, the 1% TVE specification for 50 Hz Syncrophasor applications requires only 32 microseconds (if the only source of error), and phase-comparison protection applications can usefully use time with up to a few hundred microseconds error; it is therefore important that applications be able to know and use the accuracy of the provided time.
WAN Time Distribution Technologies
NTP) Network Time Protocol: NTP, and the simpler SNTP, have provided UTC time over packet networks since 1985 by comparing the apparent latencies of interchanged messages. The values are passed through filters and subjected to statistical analysis. Outliers are discarded and an estimate of time offset is derived from the best remaining candidates. Using the UTC timescale, NTP naturally handles leap-second events with the last second of the day becoming 60 or 58 seconds.
Virtually all computers use this technology for displaying time with sub-second accuracy, but the millisecond-level perturbations from normal network packet queuing delays makes this unsuitable for substation applications.
GNSS (GPS et al): Global Navigation Satellite Systems were designed to provide location, not time; but since time is needed for their location algorithms, they can provide time; and the few-meters of location accuracy requires a few tens of nanoseconds time accuracy.
For traditional SDH/SONET interconnected substations, GPS has been successfully used to provide both TAI and UTC for the substation applications; and by also using the SDH/SONET to distribute time, the need for a GPS antenna at every substation is avoided, plus detection of rogue GNSS spoofers becomes feasible. (see Figure 3).
Unfortunately, clock designers using GPS have learned the hard way that the GPS system designers were apparently not interested in this secondary application.
For a start, GPS does not even indicate what is the current year; with tens of kilobits in each superframe would this have been so onerous?
We can thank GPS for providing the updates from IERS on newly announced Leap-second events, but rather than passing on the IERS’s day-of-year we get a day-of-week and week-of-GPS-epoch; but no information found (in Navstar’s Interface Specification IS-GPS-200) on whether the day-of-week is 0-6 or 1-7, and whether the week starts on a Sunday or on a Monday.
Then of course with only 10 bits being used for the week-of-GPS-epoch, encoding the latest Leap-second event can be a challenge, which is why we are currently being told that there will be a (non-event) Leap-second event Wednesday October 27, 2021 (being day 7 of week number 137 in the current GPS epoch). (Presumably a non-event because the LS (before) and LSF (after) second fields are both 18.) (see Figure 4).
Adopting the same mentality towards time users, Beidou reportedly chose to quietly use a different day-of-week base (0-6 c.f. 1-7) causing clocks to insert leap-seconds a day early (unfortunately even GNSS simulators also apparently got it wrong). It is no wonder that many IEEE and IEC members have tried to wean the substation applications away from using UTC, but so far to no avail.
In summary, GNSS is an impressive technology for distributing the second-rollovers (PPS) of both TAI and UTC timescales, with sub-100ns accuracies; in fact, it enabled the global synchronization of the radio receivers responsible for the black-hole image we saw recently. Its use for disseminating the Leap-second information needed to provide UTC timescale substation timestamps is more challenging.
Network Tunneling: Over the last few years, Utility bean-counters have pressured their protection engineers (OT departments) to abandon their SDH/SONET networks and transition all their inter-substation communications to their corporate IT Ethernet networks; but since the power-rating of transmission lines is very dependent on the tripping time of their protection relays, this must be implemented without suffering communication latency degradations.
The queuing required to handle random traffic makes Ethernet fundamentally a “best-effort” communication technology, even though priority assignments and bandwidth ‘shapers” have allowed some improvements to the latencies of some classes of traffic.
Four years ago, the ongoing challenges triggered the IEC to form a TF to study “Deterministic Network Technologies” for both substation LANs and inter-substation WANs. For deterministic WANs, the IEC 61850-90-13 suggested considering the ongoing IETF’s “DetNet”, using the IEEE’s TSN (Time Sensitive Network) technology being targeted initially for LANs. The other WAN technology considered deterministic, and with the attraction of not requiring new hardware for the WAN network components, is the use of Pseudowire (IETF RFC 3985) to packetize SDH/SONET TDM bit-streams so all a substation’s traffic is mapped into a single Ethernet stream which can then be transported deterministically, using (top-priority) low-latency tunnels, to provide the same performance as conventional SDH/SONET networks; thus creating a “Virtual Synchronous Network” (VSN).
Pseudowiring denotes a technology for packetizing TDM streams for transport over packet networks; it is commonly used to handle DS0 circuits like the C37.94 optic links for substation teleprotection; its use for packetizing higher bandwidth streams like SDH/SONET is not so widespread. A side benefit of this technology is its ability to provide an “NTP on Steroids” for better-accuracy time-distribution.
As reported in IEC 61850-90-13, a 155 Mbps STM-1 VSN stream on a 10 GbE network would likely comprise an ~0.1µs length packet each ~5µs and be delayed from 0 to 1.2µs at each (top-priority) queue on a 10 Gbps network. With NTP style timestamps in each packet’s header, NTP style statistical analysis, and 200,000 packets per second, micro-second accuracy time-distribution becomes available. (see Figure 5).
1588 (PTP): By specifying “hardware” timestamping for interchanged messages as they egress and ingress network ports, the IEEE 1588 standard avoids the NTP’s Achilles heel of queuing latencies thereby allowing many orders of magnitude improvement in time distribution.
With the success of its original IEEE 1588-2002v1, 1588-2008v2 was morphed into more of a toolbox so different markets could standardize their own 1588 “Profiles” to suit their individual requirements. With a strong desire by telecom providers to use their new Ethernet WANs to synchronize islands of SDH/SONET networks, ITU developed frequency-distribution profiles under G.8265, followed by time-distribution profiles under G.8275; for each there are choices to accommodate networks both with and without specialized hardware at every switch/router. (see Figure 6).
For the time accuracies being expected for substations, only the ITU G.8275.1 suffices; fortunately, the more recent interest in accurate time needed for cellular radio sites has promoted a significant penetration of G.8275.1 into the telecom networks.
The Substation
Substation IEDs needing time today may have their own GNSS receiver for use with a roof-mounted antenna but are more likely to have an IRIG-B connection to a satellite receiver with IRIG-B outputs.
The adoption of the alternative IEEE 1588 for time-distribution is increasing, but substation IEDs have a very long life-span, and support for legacy IRIG-B IEDs will likely be required for a few more decades.
The Gateway: The substation’s Gateway provides the interfaces between the external and internal time-distribution networks. Illustrated is the scenario emerging to support the unfolding transition of WANs from dedicated SDH/SONET to Ethernet Packet networks.
The Gateway’s Clock: It makes sense for this clock to use the PTP (1588’s Precision Time Protocol) timescale, i.e. the clock’s output would be a count of seconds, and fractional seconds, incrementing exactly 1 second per SI second, and with its epoch (value 0) being the start of 1970 TAI. (see Figure 7).
It is important for the clock to maintain useful time when all the time sources are unusable, a state known as “holdover” when the clock’s time drift depends on the stability of the source of its ticks.
In order for the IED applications to continue using the time, it is very important to also provide a reasonable estimate of the worst-case time error as the holdover phase progresses; for this estimate it is useful to continually run “what-if” holdover simulations while the Clock is synchronized.
With the recent significant improvement in the stability of crystal oscillators, useful time can now be provided to protection IEDs for at least several days of holdover.
The Time Source monitors and Selection: This module monitors the possible time sources for their quality in order to assess which if any are acceptable for synchronizing the Gateway’s Clock.
The Leap-second Events (from IERS Bulletin-C)
For the IRIG-B Generator, this provides (per IEEE C37.237):
- Flags during the last minute before a jump (LS and LSP)
- IERS event announcements, with days-to-expiry (SBS/TDM LatestLeapEventYear/Day/Direction/Validity)
For the 1588 Power Profile Generator, this provides (per IEEE C37.238):
- Flags during the last 12 hours before a jump (Leap59 and Leap61)
- IERS event announcements, with days-to-expiry (per IEEE C37.238a (awaiting IEEE-SA RevCOM approval))
The above also supports substation atomic clocks providing long-term UTC support (to IRIG-B and/or 1588 networks).
The HMI: For user time-displays from IEDs, the Gateway will likely have a user-interface for configuring the local time-zone, and local DST settings.
For substations not getting automatic announcements from IERS (Bulletin-C), the ability for manual updates is needed:
- GPS provides at least 6 weeks of Announcement validity (covering expected quarterly leap-second events)
- Other GNSS services need checking
- 1588 Telecom Profile does not provide the IERS Leap-second event announcements
The 1588 Power Profile Generator: The user must set the VLAN and PTP Domain number to match those assigned by the substation engineer.
For some green-field 61850 substations, it is conceivable that the base profile suffices; but for legacy substations it is likely that the extended profile (with PTP domain 254) is more appropriate (this also supports base-profile IEDs).
Biography:
Chris Huntley received his M.A.Sc. in Engineering Physics from UBC in 1960, followed by a diploma from Imperial College under an Athlone Fellowship. Chris then joined GTE Lenkurt Electric in Burnaby B.C. (subsequently under B.C. Tel, Nortel and GE). In 2007 he started a communications R&D group for Schweitzer Engineering Laboratories. After 30 years of designing many protection interfaces and an analog FDM multiplexer (46A3C), Chris moved to digital creating the first SONET multiplexer for the power utility market (Junglemux), followed two decades later by a more-flexible version (ICON) for packet networks. Chris is a senior member of IEEE, IEC and CIGRE WG.
