When Data Centers Protect Themselves: A 3.8 GW Load Transfer

At 7:56 a.m. on July 22, 2026, a mechanical fault struck a 230 kV transmission line in Northern Virginia. Protection cleared the fault normally; the relays and breakers did their jobs. The trouble followed those successful local responses.

The trouble followed. A voltage disturbance caused some data centers to switch to backup power. The sudden reduction in grid demand raised voltage, triggering a second wave of transfers. In less than two minutes, roughly 3,800 MW, or 3.8 GW, of load left the public grid. PJM and Dominion's preliminary investigation identified it as the largest such load transfer PJM had experienced.

The facilities moved to backup power and kept operating, while demand on the public grid dropped abruptly. Frequency reached 60.092 Hz. Operators reduced generation and adjusted reactive resources; the balancing-area control limit excursion ended within nine minutes, and Dominion reported customer reconnection within thirty minutes. PJM's later event account records that grid reliability was maintained through the response.

The interesting bill arrives after “data-center protection succeeded.” One campus transfers its load and changes the electrical conditions facing others. Successive campuses then send more balancing work back to the grid. Treating uninterrupted servers as the end of acceptance testing leaves half the reliability picture out.

Three disturbances in a minute, then no automatic return

To understand the response, we need to unpack "switching to backup power." A UPS may use batteries to sustain critical loads during a voltage sag and quickly resume drawing from the grid. A diesel rotary UPS using a flywheel and engine may stay on backup longer. Even with the same hardware, an operator's additional control rules can change the recovery behavior. To a server, these all provide uninterrupted power. To the grid, they are not necessarily the same kind of load.

A separate event on July 10, 2024 made this difference concrete. After a lightning arrester failed on a 230 kV line, automatic reclosing produced six successive faults and clearances over 82 seconds. Each lasted 42 to 66 milliseconds and was cleared properly, yet approximately 1,500 MW of data center load left the grid. NERC's incident report documented a typical customer-side rule: count three voltage disturbances within a minute, switch to backup, and stay there until manually reconnected.

The line's reclosing sequence was trying to restore supply. The facility's disturbance counter interpreted repeated abnormalities as a reason not to trust the external source. Around the third voltage depression, about 1,260 MW dropped off and did not return for hours. NERC attributed most of the sustained reduction to the interaction between the reclosing sequence and disturbance-counting controls.

Control rules turned brief disturbances into a sustained change in demand. The 2024 report identified the interaction between disturbance counting and reclosing; the preliminary 2026 investigation recorded two waves of load transfer. Following each event's equipment settings and restoration rules leads to configurations that can actually be changed.

The distinction matters for remediation. If equipment cannot tolerate the disturbance, hardware may need to change. If additional controls move it prematurely into a prolonged backup state, trigger conditions and recovery rules deserve scrutiny. Calling all of it "oversensitive data centers" does not tell us which layer to change.

Why using less power can also trouble the grid

When we discuss data centers and AI, we often ask whether there is enough electricity and how much new capacity GPUs will consume. July's event raised the opposite question: can the grid follow when a large block of demand suddenly stops drawing power?

Generation and demand must remain balanced. A sudden demand reduction before generation falls leaves an active-power surplus and raises frequency. Voltage also depends on reactive balance, impedance, and control equipment. One load change can put both balancing problems on the operator's desk.

Higher voltage after the first transfer triggered a second set of facilities. PJM's two recorded waves make the feedback visible: the response to the original fault became an input to the next disturbance. The sequence continued after the faulted line had been cleared.

Peak power and projected annual consumption therefore cannot fully describe a large facility's grid impact. Two campuses of the same size can require very different dispatch responses if one resumes demand promptly after fault clearance and the other stays on backup for half an hour. Equal average consumption does not imply equal dynamic behavior.

Redundancy meets a common trigger here. Each facility can own separate UPS systems, generators, and controllers while seeing the same voltage trajectory. Similar thresholds create correlated actions, and many independent devices can transfer together. That collective response belongs in interconnection studies.

The data center keeps running; the grid takes the swing

From an operator's perspective, leaving an abnormal external supply quickly is understandable: keep servers and cooling powered and the business running. The grid's objective is regional active and reactive power balance and frequency stability. A choice that protects one business can increase the public system's control workload.

A facility sets thresholds around its business risk. The grid must plan reserves, generation, and voltage control around the regional response. Decision-making sits at the campus while some adjustment costs land outside it. Individually reasonable settings can combine into a much harder operating state.

That externality is the problem. A transfer reduces local business risk while the grid absorbs the disappearing load. Interconnection requirements, dynamic models, and restoration agreements should put the bill on the table beforehand. Asking afterward whether one UPS met its specification starts a step too late.

Backup transfers can be uneventful when small, dispersed, and planned for. The difficult combination is a shared disturbance, similar triggers, large loads, and tightly clustered actions. Those quantities belong in the model; organizational boundaries should not draw the edge of the test.

This sequence reminds me of a retry storm

Software engineers know the pattern: a backend slows, clients time out, and they retry. For one caller, another attempt may succeed. For an overloaded backend, thousands of callers trying once more can lengthen the queue and cause still more timeouts.

Software engineers know this feedback loop. Participants observe a shared environment and compensate for their own tasks; their compensation changes the environment and triggers more compensation. Google's SRE Book addresses it through retry budgets, jittered backoff, bounded attempts, and shedding work under overload.

Consider three nested layers allowing three attempts each. The downstream service can receive 27 calls. Each local setting looks restrained, yet their product can overwhelm shared capacity. A budget spanning the whole chain places a total limit on all those local retries.

Electrical protection clears dangerous faults in milliseconds; HTTP backoff timings belong in software. The transferable idea is the control structure: budget additional actions, coordinate restoration, and test the collective response. Essential protection stays fast, and the consequences of that protection get an owner too.

Autoscaling poses a related problem. If several upstream services add instances because their CPUs are busy, but the bottleneck is a shared database, they may merely add more concurrent connections. A test that measures only upstream service throughput can mistake displaced pressure for improvement. Downstream queues, aggregate successes, and recovery time must also be measured.

Testing cannot stop at the UPS output

A facility tests UPS transfers, generator starts, and whether servers stay powered. The grid tests fault clearance, breaker operation, and reclosing timing. Those tests are necessary, but passing on both sides does not establish how the two systems behave together.

Nancy Leveson's STAMP approach to system safety includes unsafe interactions among functioning components. Interconnection tests should take that wider view: examine each device and the control signals its behavior sends upstream and downstream. Equipment and organizational handoffs are precisely where testing tends to stop too soon.

If I were reviewing this interconnection study, I would ask for explicit load-response behavior: which voltage or frequency conditions trigger a transfer, how long they must persist, whether brief disturbances accumulate, whether an off-grid state latches, and who decides to return. A normal power profile cannot explain these state transitions.

Then I would examine many facilities under one disturbance rather than copy a single model hundreds of times and assume independent actions. Similar thresholds exposed to the same voltage trace may cross a boundary together. The grid state changed by the first wave must also feed into the subsequent simulation.

Dynamic models need measurement-based calibration. NERC's March 2026 large-load gap assessment identifies gaps in model validation and measurement. Simulation, equipment tests, real disturbance records, and hardware-in-the-loop where needed can build evidence for parameters and responses. Field testing must respect equipment and grid constraints and be designed for the load being studied.

I would also review recovery separately. Dispatch may have reduced generation while facilities are off-grid. A large, abrupt reconnection can create an imbalance in the opposite direction. When demand returns, how fast, and with what external coordination deserves testing just as much as the exit. NERC's report on the 2024 event also flags controlled reconnection and ramp rates.

Agree in advance on ride-through and recovery

Interconnection agreements need three concrete answers: which disturbances to ride through, when transfer is permitted, and how load returns afterward. Facilities must protect server, cooling, and business limits; operators need advance knowledge of the load response. Coordination starts by putting both sets of requirements on the table.

PJM's September 8, 2026 proposal defines operating regions by voltage magnitude and duration. In the specified regions, loads may temporarily reduce active power. Once voltage returns above 0.9 and below 1.1 p.u., they must recover to more than 90% of pre-disturbance demand within two seconds. Per unit expresses values relative to nominal. Magnitude, duration, and recovery speed become parts of one requirement.

The scheme was still a proposal at the time. PJM's September 10 account also discusses transition arrangements for existing facilities. Thresholds, duration, equipment capabilities, and transition costs all need to be settled in the interconnection study.

Cost allocation matters too. If better ride-through, monitoring, and dynamic modeling mainly reduce public-grid risk while the facility bears the investment alone, technical feasibility may not produce voluntary adoption. Interconnection requirements, responsibilities, and compensation influence that choice. A UPS controller cannot settle all of them.

On September 17, PJM activated demand response during tight supply and received emergency authorization to require large customers such as data centers to use backup generation if needed. The same transfer can be a coordinated demand-reduction tool during a shortage or an unexpected collective exit during a disturbance. Timing, scale, and coordination determine its system effect.

A compiler can reshape the power profile too

There is another route that does not directly concern UPS controls: reducing fluctuations on the compute side. In traditional clusters, unrelated workloads can partly offset one another's power variation. Large synchronized training can make many accelerators compute together and then move together into communication or waiting. Time correlation matters to electrical infrastructure alongside GPU count and total peak power.

In a February 2025 technical article, Google reported cluster-level fluctuations in the tens of MW and described a TPU-compiler intervention. It detects synchronization flags associated with power variation and adjusts activity in major compute blocks around those flags, smoothing utilization over time. In the disclosed test case, fluctuation magnitude fell by nearly 50%, with less than 1% performance impact.

The reduction is in fluctuation amplitude, while average power rises slightly. Smoothing computational activity gives the facility a smoother load. This opens a control lever at the scheduling layer, alongside the separate work of improving UPS voltage protection.

The useful direction, to me, is extending scheduling beyond "keep devices busy" to facility constraints. Job start order, movable batch windows, and campus power ramps may become optimization variables. But synchronized training has dependencies, and shifting computation can add waiting. Measure useful throughput, training time, energy use, and the power profile to decide whether it pays off. GPU utilization alone is insufficient, and business costs cannot be ignored when requesting staggered work.

More persistent agents, more congested services?

Consider an agent platform whose external API slows down. Each agent adds retries, searches, and cross-checks to finish its task. The extra calls extend response times, and the next plan calls for still more checking. Each task looks more persistent while the platform becomes more congested.

Suppose an agent raises its task success rate from 95% to 98% while making other tasks wait several times longer. Who gets credited for that optimization? Platform acceptance has to account for the displaced resources and users. Those three percentage points reveal very little of the system's bill.

In an implementation, I would put total API budgets, concurrency limits, and retry allowances in the tool-execution layer, not just ask for restraint in a prompt. Budgets should cover the whole task and actual call chain. Spawning a child Agent must not issue a fresh allowance. Exhaustion should return an explicit limit or backoff state so the planner does not treat it as an ordinary failure and keep trying.

Restoration also belongs under execution-layer control. Admit work gradually as a service recovers and increase concurrency according to errors and queueing. Measure total completions, tail latency, waits imposed on other tasks, and recovery time. Returning the system to a stable state is a task in its own right.

After reading this investigation, I would add a scenario to an architecture review: assume every component correctly executes its own protective strategy; what happens next? Tests that prove each facility keeps power, each client retries, or each Agent works harder may still miss the costs of their combined actions.

I would end a reliability review with a different question: once every component has protected itself according to its rules, who manages the environment they changed together? Northern Virginia's two transfer waves put that question on the operator's desk. Software architecture reviews should ask it before an incident does.