What happens when ten thousand shoppers hit your checkout page in the same minute? If your answer is “the server catches fire,” you need a load balancer. These tools sit quietly between your users and your servers, spreading traffic so no single machine drowns. The concept is simple. The tooling is not. AWS alone sells four different load balancer types, each with its own quirks and price model, and the other clouds have their own menus. This guide walks through what these tools actually do, how the major platforms compare, and which mistakes cost teams the most time and money. (What is Load Balancing?)
What Cloud Load Balancing Tools Actually Do
A load balancer distributes incoming traffic across multiple backend servers or regions to improve availability and responsiveness. That is the neutral, standards-body version of the definition. In the cloud era, though, the tool is a managed service rather than a box in a rack. You configure it through an API, it scales without your involvement, and you pay by the hour plus traffic processed.
Three jobs matter more than the marketing suggests:
- Health checking. The balancer probes each backend on an interval and stops routing to instances that fail. You tune the interval, timeout, and healthy/unhealthy thresholds. Get these wrong and you get flapping: servers bouncing in and out of rotation during brief GC pauses.
- Traffic distribution. Round robin, least connections, and weighted variants are the common policies. Consistent hashing keeps a given user or cache key pinned to the same backend, which matters for session affinity and cache locality.
- TLS termination. Offloading HTTPS decryption to the balancer shifts CPU cost off your application servers and centralizes certificate management.
One practical warning from teams that have run these at scale: the load balancer is usually not the bottleneck, but the health check is the most common source of self-inflicted outages. A health check that depends on a downstream database will pull your entire fleet out of rotation when that database hiccups. Point health checks at a shallow endpoint that only confirms the app process is alive.
The Big Three Compared
Every major cloud ships a managed balancer, but they organize their products differently. Here is the rough shape of each as of recent documentation.
| Platform | Layer 4 option | Layer 7 option | Global frontend | WAF available |
|---|---|---|---|---|
| AWS | Network Load Balancer (TCP/UDP, very high throughput) | Application Load Balancer (HTTP/HTTPS) | No native anycast global tier; use Route 53 or Global Accelerator | Yes, via AWS WAF on ALB |
| Google Cloud | Passthrough and proxy network load balancers | Global and regional external Application Load Balancers | Yes, single anycast IP frontends | Yes, Cloud Armor |
| Azure | Azure Load Balancer (Basic and Standard SKUs) | Application Gateway | Yes, via Azure Front Door | Yes, WAF on Application Gateway and Front Door |
A few details hide inside that table. AWS Elastic Load Balancing), launched in 2009, now spans four products: the Application Load Balancer for HTTP/HTTPS at layer 7, the Network Load Balancer for TCP/UDP at layer 4, the Gateway Load Balancer for third-party virtual appliances, and the legacy Classic Load Balancer. The Application Load Balancer arrived in 2016 and is now the default choice for web workloads.
Google Cloud’s offering is software-defined and globally distributed by design. Its global external Application Load Balancers accept traffic on a single anycast IP and route users to the nearest healthy region. That architecture is genuinely different from AWS’s regional-first model, and it changes how you think about failover.
Azure splits the job. Azure Load Balancer handles layer 4 TCP and UDP traffic in Basic and Standard SKUs, while Application Gateway provides layer 7 HTTP(S) balancing with a built-in Web Application Firewall. Azure Front Door covers the global HTTP edge case. Note that Microsoft has been retiring the Basic SKU, so check current status before you architect around it.
How to Choose Without Overthinking It
Most teams spend weeks on this decision and then change it anyway. A faster path is to answer four questions in order.
- What protocol are you balancing? Plain TCP or UDP means a layer 4 balancer. HTTP means layer 7, which unlocks host-based routing, path-based routing, and header manipulation.
- Do your users span continents? If yes, you want a global frontend with anycast routing or a CDN edge layer in front. If your users are regional, a regional balancer is cheaper and simpler.
- Do you need a WAF? If you handle payment data or personal information, the answer is probably yes, and bundling it with the layer 7 balancer saves you a separate integration.
- How does it connect to autoscaling? Load balancers feed health and traffic signals into autoscaling groups, so capacity tracks demand. Confirm the balancer registers new instances automatically. Manual registration is a classic source of “we scaled up but traffic never arrived” incidents.
The order matters. Teams that start with pricing usually end up with a layer 4 balancer doing layer 7 work through creative abuse, or a global frontend for a single-region app. Start with protocol, then geography, then security, then cost.
Where Teams Get Burned in Practice
Documentation tells you what the features are. It rarely tells you what goes wrong. Here is what actually bites.
Cross-zone and cross-region data transfer. Load balancers move bytes, and clouds charge for bytes that cross zone boundaries. A balancer configured to spread traffic evenly across three zones can triple your inter-zone transfer bill compared with zone-affinity routing. This is one of the least visible line items on a cloud invoice. A clearer way to track cloud data transfer costs is worth reading before you enable cross-zone balancing by default.
Sticky sessions as a crutch. Session affinity feels convenient until an instance dies and every pinned user loses their cart. Externalize session state to a shared store and turn affinity off unless you have a specific caching reason to keep it.
Idle timeouts that outlast your app. Every balancer has a default idle timeout, often 60 seconds. If your app holds long-polling connections or slow uploads, the balancer will cut them. Raise the timeout on both the balancer and the backend, or you will chase phantom 504 errors for a week.
Health check paths that hit the database. Covered above, but it deserves repeating because it is the single most common misconfiguration.
Forgetting the second environment. Staging balancers configured by hand drift from production within a month. Treat the balancer config as code from day one.
Load Balancing Is a Regional Decision
Where you place the balancer matters as much as which one you pick. A load balancer lives in a region, and its backends should usually live in the same region. Cross-region balancing adds latency to every request and transfer cost to every byte.
Before committing, it helps to understand how providers announce new regions and what those announcements actually promise. Reading a cloud region announcement carefully tells you whether a region has the service tier you need on day one or months later. Matching cloud regions to workload requirements is the same discipline applied to latency, data residency, and disaster recovery.
For teams running multiple products or business units, the question of whether to split workloads across cloud accounts intersects with load balancing more than people expect. Each account boundary usually means a separate balancer, a separate certificate, and a separate bill. That separation is often worth it for blast-radius reasons, but it needs to be a deliberate choice.
The Global Frontend Question
Global load balancing deserves its own section because it is where the architectures diverge most.
Google Cloud’s global external Application Load Balancer uses anycast: one IP address announced from many edge locations, with users routed to the nearest healthy backend. Cloudflare built its network on the same principle. The appeal is obvious. Users in Tokyo and Frankfurt both hit the same hostname and land on nearby infrastructure without DNS tricks.
AWS takes a different route. Its Application Load Balancer is regional, so global distribution typically means Route 53 latency routing or Global Accelerator in front of regional balancers. That is more moving parts, but it also gives you finer control over failover policy.
Azure sits somewhere in between with Front Door as the global HTTP layer and regional Load Balancer and Application Gateway behind it.
None of these is wrong. The practical question is whether you want the global routing handled inside the balancer or in a separate DNS or accelerator layer. Teams with strong infrastructure-as-code practices often prefer the separate layer because it is easier to reason about and test. Teams that want fewer components prefer the integrated global balancer.
Cost Traps Worth Knowing
Load balancer pricing has three components almost everywhere: an hourly charge per balancer, a charge per capacity unit or LCU, and data processing fees. The hourly charge is the visible part. The other two are where budgets break.
A few habits that keep costs sane:
- Consolidate balancers. One Application Load Balancer can host many services through host-based and path-based rules. Spinning up a balancer per microservice is a common and expensive default.
- Watch idle balancers. A balancer with no traffic still bills hourly. Audit quarterly.
- Check cross-zone settings. As noted, cross-zone balancing can multiply transfer costs.
- Size capacity units to reality. LCU-based pricing scales with connections and processed bytes, so a traffic spike shows up on the next bill.
Conclusion
Cloud load balancing tools are not interchangeable, but the decision is more tractable than the product pages suggest. Pick the layer that matches your protocol, decide whether you need a global frontend or a regional one, confirm the WAF story, and only then compare prices. Then spend your real effort on the parts that cause outages: health check design, timeout tuning, session handling, and configuration as code. The balancer itself will almost never fail you. The configuration around it will, and that is where careful teams earn their uptime.
Frequently Asked Questions
What is the difference between a layer 4 and a layer 7 load balancer?
A layer 4 balancer routes TCP and UDP traffic without inspecting the payload. It is fast and protocol-agnostic. A layer 7 balancer understands HTTP and HTTPS, so it can route by hostname, path, or header, terminate TLS, and apply a web application firewall. Most web applications want layer 7; databases and custom protocols usually want layer 4.
Do I need a load balancer if I only have one server?
Not for distribution, but often for TLS termination, health monitoring, and a stable entry point you can repoint later. Many teams start with a balancer in front of a single instance precisely so that scaling out later requires no DNS changes. (9481.org)
How do health checks actually work?
The balancer sends a request to a path you configure on each backend at a set interval. If the backend fails to respond within the timeout a set number of times, it is marked unhealthy and removed from rotation. It returns after passing a healthy threshold. Keep the check shallow so it tests the app, not its dependencies.
Can a load balancer improve security?
Yes, in specific ways. It can terminate TLS so certificates live in one place, hide backend IP addresses from the public internet, absorb certain volumetric attacks through its capacity, and apply a web application firewall at layer 7. It is not a substitute for application-level authentication or input validation.
Why does my load balancer cost more than expected?
Usually three reasons: too many balancers for too few services, cross-zone traffic that incurs inter-zone transfer charges, or capacity-unit pricing that scales with connections and bytes processed. Consolidating services behind shared balancers and reviewing cross-zone settings fixes most of it.
What happens to traffic during a regional outage?
It depends on your architecture. A regional balancer fails with its region. A global anycast frontend can route to healthy regions automatically. Multi-region designs that front regional balancers with a global layer or DNS failover survive regional outages; single-region designs do not, regardless of how many balancers you deploy.
