Health Probes and Graceful Degradation

Health probes are the mechanism by which load balancers and traffic managers detect whether a backend instance is capable of serving requests. Without health probes, a load balancer might continue sending traffic to a failed or unresponsive server, causing user-facing errors. Properly configured health probes enable automatic traffic rerouting away from unhealthy instances within seconds of a failure.

Azure Load Balancer supports two types of health probes:

• TCP probe — checks if the backend can accept a TCP connection on a specified port. Simple but does not verify application logic.
• HTTP/HTTPS probe — sends a GET request to a specified path and expects a 200 OK response. More accurate because it tests the application endpoint directly.

A backend is marked unhealthy if the probe fails for a configurable number of consecutive attempts.

A well-designed health endpoint (/health) does more than return 200 OK — it verifies that the application's critical dependencies are reachable. A comprehensive health check might test connectivity to the database, the cache, and any downstream APIs. If any dependency is unavailable, the endpoint returns a 5xx status code, signalling the load balancer to remove this instance from rotation.

Azure Traffic Manager also uses health probes, but at the regional level. It sends periodic HTTP or HTTPS GET requests to the configured endpoint URL in each region. If an endpoint fails to respond within the timeout window for a set number of consecutive intervals, Traffic Manager marks that endpoint as degraded and stops routing DNS queries to it, redirecting users to a healthy region.

Azure Application Gateway has more sophisticated health probe capabilities than the standard Load Balancer. It supports custom probes that specify the host header, the expected status code range (e.g., 200-399), and a body match string. Application Gateway also supports per-path routing, so different backend pools can have different health probe configurations for different URL paths.

Graceful degradation is the ability of an application to continue providing partial functionality when one or more of its dependencies fail. Instead of crashing entirely, the application detects the unavailability of a non-critical service and falls back to a degraded but still useful state. For example, if a recommendation service fails, an e-commerce site could show generic suggestions instead of crashing the entire product page.

The circuit breaker pattern prevents an application from repeatedly calling a failing downstream service. When a service starts failing, the circuit breaker opens and immediately returns an error or fallback response without making the network call. After a cool-down period, it enters a half-open state and allows a trial request. If that succeeds, the circuit closes and normal operation resumes.

For transient failures (brief network blips, temporary service overload), a retry with exponential backoff strategy is appropriate. The application retries the failed call after a delay, doubling the delay with each retry up to a maximum. Adding a jitter (random variation) to the delay prevents all retry attempts from synchronising and overwhelming a recovering service.

The bulkhead pattern isolates different parts of an application into resource pools so that a failure in one area does not consume all resources and bring down the entire system. Named after ship bulkheads that prevent one flooded compartment from sinking the whole vessel. In Azure, this might mean using separate thread pools or separate App Service plans for different services to contain failures.

A common graceful degradation technique is to serve cached or stale data when a live data source is unavailable. For example, a product catalogue page might serve yesterday's cached prices from Azure Cache for Redis rather than showing an error if the database is temporarily unreachable. Users experience a minor inconvenience (slightly stale prices) rather than a complete failure.

Graceful degradation should be visible and measured. Use Application Insights to track the rate of circuit breaker opens, fallback responses, and retry attempts as custom metrics. Set up alerts when these metrics exceed thresholds, so the on-call team is notified that the application is running in a degraded state even if the user-facing experience appears acceptable.

Test your understanding of Microsoft Azure Fundamentals (AZ-900) concepts from this lesson.

In this lesson you learned: health probes allow load balancers to detect failed backends and reroute traffic automatically; graceful degradation keeps applications partially functional when dependencies fail; and patterns like circuit breaker, retry with backoff, and bulkhead implement resilience at the application level. Next up we explore disaster recovery concepts — defining RTO, RPO, and recovery tiers.