Advanced 18 min read

Fault Tolerance and Load Balancing

By the end of this lesson, you'll be able to:

Explain what fault tolerance means and why designs should assume components can fail
Explain how a load balancer contributes to fault tolerance, and what it does not fix
Compare a direct call between components with a decoupled, queue-based design under a downstream failure

Prerequisites: "Scalability, Elasticity, and High Availability"

Designing on the assumption that things fail

Fault tolerance starts from an uncomfortable but necessary assumption: individual components — a server, a task, a disk, an entire Availability Zone — will eventually fail. The practical questions worth asking for any component in an architecture: what is this component's failure domain? Is there a single point of failure here? Can an unhealthy component be replaced without a person manually intervening? Is any persistent data this component holds actually protected? What happens to work that was in-flight when it failed? And ultimately, can the system recover?

This isn't a claim that AWS itself is unreliable — it's the same discipline Module 9 applied to ECS tasks and Module 8 applied to SQS messages, now generalized: assume failure will happen, and design for what happens next, rather than assuming it away.

A load balancer's real job — and its real limits

A load balancer distributes incoming requests across multiple targets, performs health checks against them, and — depending on configuration — routes traffic only to targets currently passing those checks (Module 6, Module 9). This directly supports fault tolerance: a single failed application instance or task doesn't take the whole application down with it.

It's just as important to be precise about what a load balancer does not fix: a slow, overloaded database behind it, genuinely broken application code, or a downstream dependency that's actually failed. A load balancer distributes and health-checks traffic to the application tier — it has no influence over problems that live somewhere else in the architecture.

Decoupling a direct call: an SQS example, revisited

Diagram comparing a direct call from an Order API to a Report Processor, where the request path is impacted if the processor is unavailable, with a decoupled design where the Order API sends work through an SQS queue to a worker, so a temporarily unavailable worker does not impact the request path. — Decoupling improves resilience against exactly this failure mode — it does not make failure impossible, and it adds real complexity of its own.

This directly extends Module 8's SQS lesson: when an Order API calls a Report Processor directly, that processor being temporarily unavailable impacts the Order API's own request path. Routing the same work through an SQS queue to a worker instead means a temporarily unavailable worker doesn't block the Order API at all — the queue buffers the work until the worker recovers. This is a genuine fault-tolerance improvement, but it isn't free: it introduces eventual processing (the work no longer happens immediately), retries, duplicate handling (Module 8's idempotency discussion), and a real monitoring requirement to notice if a queue starts backing up. Decoupling is a trade-off, not a strictly-better upgrade.

Putting fault tolerance together into one representative architecture

Diagram showing Route 53 resolving to an Application Load Balancer, which distributes traffic across application tasks in two Availability Zones backed by a resilient database, with CloudWatch, IAM, and Security Groups applying across every layer. — The point of this diagram is the reasoning it represents, not memorizing this exact shape — monitoring and security apply across every layer, not as an afterthought bolted onto one box.

This is one representative shape a fault-tolerant, load-balanced architecture can take — not a template to copy without reasoning through whether it fits a given workload's own requirements.

Exercise

A web application currently runs on a single application server in a single AZ, and the requirement is to tolerate the failure of that one application instance. Using this lesson's reasoning, describe the architectural improvement that best addresses this specific requirement, and explain what it would and would not protect against. Then, separately, a report-generation feature currently calls a Report Processor directly, and needs to keep working even if the processor is occasionally slow or briefly down. What change would you make, and what new complexity does it introduce?

Show hint

For the first part, think about what a load balancer plus multiple instances actually changes, and what it leaves unaddressed (like the database). For the second, this is the direct-versus-decoupled comparison this lesson just walked through.

Fault Tolerance and Load Balancing Quiz

1. A web application currently runs on one application server in one AZ, and needs to tolerate the failure of that instance. What is the best architectural improvement?

Multiple application instances or tasks behind a load balancer A larger single instance with more CPU and memory Moving the single instance to a different AZ, alone Adding more IAM permissions to the instance's role

2. A load balancer will fix a slow, overloaded database sitting behind it.

True False

My Notes

Questions about this lesson

No questions yet — be the first to ask.

Summary

Fault tolerance means designing on the assumption that individual components will fail, using a load balancer to distribute traffic and route only to healthy targets — though a load balancer does not fix a slow database, bad application code, or a genuinely failed downstream dependency — and decoupling a direct call into a queue-based design can prevent a downstream failure from impacting the caller's own request path, at the cost of eventual processing, retries, and duplicate handling.

❬ Previous Next ❭