Cloud Disaster Recovery & Multi-Region Active-Active
The Booking That Existed in Frankfurt but Not in Virginia
Test your architecture intuition: Pitch a 7-axis solution, survive two aggressive reviewer objections, and inspect the staff-level Teacher Gold Answer.
1. What It Is & Why It Exists
The Core Problem: Regional Catastrophes & Data Loss
Cloud infrastructure is vulnerable to catastrophic failures: optical fiber cuts, undersea cable severing, regional power grid collapse, data center flooding, or rogue software deployments taking down an entire cloud region (e.g., us-east-1).
To survive regional outages, systems must be engineered against two primary recovery metrics:
- RPO (Recovery Point Objective): The maximum acceptable data loss measured in time (e.g., for financial ledgers, for analytical metrics).
- RTO (Recovery Time Objective): The maximum acceptable downtime before the service is fully restored in a secondary region (e.g., for tier-1 payment gateways, for internal back-office tooling).
Synthesizing vector architecture diagram...
These three boxes are the settings of every disaster recovery plan, and they pull against each other. RPO is how much recent data you can afford to lose, set by how often or how synchronously you replicate. RTO is how long you can be down, set by how much infrastructure is already running in the second region. Cost is what you pay to keep that second region ready. Pushing RPO and RTO toward zero means continuous replication and fully running standby capacity, which raises cost. Start from the business: what does an hour of downtime or a minute of lost orders cost? Then pick the cheapest strategy that meets those targets.
2. The 4 Cloud Disaster Recovery Strategies
Synthesizing vector architecture diagram...
Comprehensive DR Comparison Matrix
| Strategy | RTO (Downtime) | RPO (Data Loss) | Relative Cost | Complexity | Traffic Routing | AWS Services |
|---|---|---|---|---|---|---|
| Backup & Restore | Hours (AWS's label) | Time since the last backup copied out of the Region | Lowest | Low | Manual DNS update | AWS Backup, Amazon S3 Glacier |
| Pilot Light | Tens of minutes (AWS's label): add DB instances, launch the fleet | The replication lag at the failure (typically seconds for Aurora Global Database) | Low | Medium | Route 53 Health Check Failover | Aurora Global Database, AMI/ECR |
| Warm Standby | Minutes (AWS's label) | The replication lag at the failure | Higher | High | Route 53 Automatic Failover | ECS/EKS scaled min instances, Aurora Global |
| Multi-Region Active-Active | Real time (AWS's label) for the Region that stays up; the failed Region's share still waits for detection, a decision, promotion and DNS: minutes | The replication lag at the failure (DynamoDB MREC: usually a few seconds); 0 only with synchronous replication such as DynamoDB MRSC | Highest | Very High | Route 53 Latency / Global Accelerator | DynamoDB Global Tables, CloudFront, Route 53 |
3. Active-Passive Disaster Recovery: Pilot Light and Warm Standby
What active-passive means
An active-passive disaster recovery strategy runs the workload in one Region and keeps a second Region for recovery only. The active Region serves all user traffic. The passive Region serves no user traffic until a failover is triggered. Three of the four strategies above are active-passive: Backup & Restore, Pilot Light and Warm Standby. Only Multi-Region Active-Active serves users from both Regions at once.
Active-passive has one property that the rest of this page spends a lot of effort on in the active-active case: writes go to exactly one Region. The database in the passive Region is a replica that receives the primary's changes. It never accepts a user write while the primary is active. So two Regions can never update the same item at the same time, and there is no cross-Region write conflict to resolve. The price is a recovery time greater than zero. A failover must detect the outage, promote the replica to writer, bring the application tier to full size, and move the users. Every one of those steps takes time, and they run one after another.
Pilot Light and Warm Standby differ in how much of the application tier is already running in the passive Region. That difference sets the RTO and the monthly cost.
Pilot Light
In a pilot light deployment, the data layer in the passive Region is always on, and the application layer is switched off.
- Always on: the database and the object store. An Aurora Global Database secondary cluster, a DynamoDB global table replica, or an S3 bucket filled by Cross-Region Replication receives the primary Region's changes continuously. This is what keeps the RPO at the replication lag instead of at the age of the last backup.
- Switched off: the application servers, workers and load balancers. Their code and configuration exist as deployable assets in the passive Region (a container image in Amazon ECR, a golden AMI copied from the primary Region, an infrastructure-as-code template). AWS's recommended form of "switched off" is to not deploy the resource at all, and to keep the automation that deploys it ready to run.
A pilot light Region cannot process a single user request until an operator or automation first deploys and starts the application tier. That is the defining limit of this strategy. Its benefit is cost: you pay for the replica database and the stored images, and nothing for compute.
Warm Standby
In a warm standby deployment, the passive Region runs a scaled-down but complete copy of production at all times.
- Always on: everything. The replica database, the load balancer, and the application servers, all deployed and running, at a fraction of the primary Region's capacity.
- Receives: no user traffic. The passive Region's servers sit behind a Route 53 failover record (or an ARC routing control) that points users at the primary Region. They do receive test traffic: because the stack is complete and running, you can send a continuous trickle of synthetic requests through it and know every day that the recovery Region works.
A warm standby Region can serve user requests immediately, at reduced capacity. Failover only has to raise the Auto Scaling group's desired capacity to the production value and move the users. Nothing has to be deployed first. That is why AWS labels its RTO "minutes" where Pilot Light is "tens of minutes".
Synthesizing vector architecture diagram...
Follow the traffic from the top. The Route 53 failover record has two targets: Region A is PRIMARY and Region B is SECONDARY. While Region A's health check passes, every user is sent to Application Load Balancer A. In the "Active Region A" panel, Auto Scaling group A runs the full production fleet (50 instances is this page's example size, a modelling choice) and writes to the Aurora Global Database primary cluster. In the "Passive Region B" panel, the same stack is deployed and running, but Auto Scaling group B holds only 10 instances (a fifth of production, also a choice) and Application Load Balancer B receives no user traffic. The dotted line from Route 53 to Application Load Balancer B shows that this path is only used after a failover. The secondary Aurora cluster receives the primary's changes through storage-level replication, typically under one second behind, and serves only reads: the synthetic test traffic that proves the Region works. Under Pilot Light the picture is the same except that Auto Scaling group B holds 0 instances and Application Load Balancer B may not exist yet. Under Hot Standby, Auto Scaling group B holds 50 instances and still receives no user traffic.
Pilot Light and Warm Standby, step by step
The two strategies share their first and last steps. They differ in the middle, where the application tier is brought to production size. The table uses the example sizes from the diagram: 50 instances in production, 10 in the warm standby, 0 in the pilot light.
| Failover step | Pilot Light | Warm Standby |
|---|---|---|
| 1. Detect that Region A is down | Health checks from outside Region A plus a business metric (requests per second from Region A falling to 0) | Same |
| 2. Decide to fail over | Operator confirms the outage and presses the failover button; the button runs the automated steps below | Same |
| 3. Promote the database | Aurora Global Database failover: the secondary cluster in Region B becomes the writer; the writes still in flight at the moment of failure are lost (the RPO) | Same |
| 4. Deploy the application tier | Run the infrastructure-as-code template to create the load balancer, target groups and Auto Scaling group in Region B | Nothing to do: already deployed and running |
| 5. Scale to production | Raise desired capacity from 0 to 50; all 50 instances must launch, pass health checks and warm their caches | Raise desired capacity from 10 to 50; 40 instances launch while the 10 running ones already answer requests |
| 6. Move the users | Switch the Route 53 failover record (or ARC routing control) to Region B; clients follow after their cached DNS answer expires | Same |
| AWS's RTO label | Tens of minutes | Minutes |
| RPO | The replication lag at the moment of failure (typically seconds for Aurora Global Database) | Same |
| What you pay for in Region B every month | The replica database and stored images; no compute | The replica database plus the scaled-down compute fleet (10 instances in this example) |
Three details of this table decide whether the failover actually works:
- Service quotas in Region B. Step 5 fails if Region B's vCPU quota is lower than production needs. With the example fleet of 50 instances of 4 vCPUs each (an
m6i.xlargehas 4 vCPUs), Region B's Running On-Demand Standard instances vCPU quota (the quota that covers the M family) must allow at least 50 × 4 = 200 vCPUs, before the disaster. A warm standby that normally runs 10 instances only proves that 40 vCPUs of the quota work. Quotas are set per Region, and a quota increase request is a control-plane operation that may be slow during a Region-wide event when many customers ask at once (the section Where the load can go: capacity, quota, residency, and the ladder of Multi-Region Failover). - Automatic DNS failover cannot wait for step 3. A Route 53 health-check failover switches the record as soon as the primary endpoint fails its checks, on the data plane, with no gate. If it fires before the Aurora secondary cluster has been promoted, users arrive in Region B and every write fails, because the database there is still read-only. For an active-passive strategy whose database needs promotion, use an ARC routing control, so that an operator or pre-approved automation flips the record only after step 3 has finished (the section Why a move to a read-only Region is gated of Multi-Region Failover). AWS's own guidance is to treat automatically initiated failover with caution and to make the manual initiation one button that runs automated steps.
- The passive Region must be exercised. A warm standby that receives synthetic test traffic every day finds a broken deployment, an expired certificate or a missing secret before the disaster does. A pilot light Region can only be tested by running steps 4 and 5 for real, so schedule a game day and time it: the measured duration of steps 3 to 6 is the real RTO, and it must be compared with the business's RTO target, not with AWS's label.
Choosing between them, and the hot standby in between
- Choose Pilot Light when the business RTO is tens of minutes or more and the monthly compute cost of an idle fleet is not justified. Internal back-office tools and batch systems usually sit here.
- Choose Warm Standby when the business RTO is minutes, or when the team needs continuous evidence that the recovery Region works. Customer-facing systems that are not tier-1 usually sit here.
- A hot standby is a warm standby scaled to full production size that still receives no user traffic. Its only advantage over a warm standby is skipping step 5. AWS notes that teams who are willing to pay for a full second environment usually run it active-active instead, so that both copies carry traffic and both are proven every second. If you do not want users in both Regions, warm standby is the cheaper and operationally simpler choice.
The sections that follow cover the fourth strategy, Multi-Region Active-Active, where both Regions accept writes and the conflict problem that active-passive avoids has to be designed for.
Unlock Complete Architecture & Production Runbooks
You have explored the free architectural preview (~58%). Spend 1 Coin to unlock the remaining 5 production deep-dive sections for a full 24 hours.