FastAPI Backend Development Bootcamp · レッスン

PrometheusメトリクスとRED／USEダッシュボード

レイテンシー、エラー、飽和度のメトリクスを公開し、Grafanaダッシュボードでサービスの健全性を可視化します。

レッスン 3/413 ステップ

「PrometheusメトリクスとRED／USEダッシュボード」はCoddyKit上の無料FastAPI Backend Development Bootcampレッスンです。これはレッスン3/4です。下記で完全なレッスンを無料で読むことができます。その後、ブラウザ内の組み込みコードエディタと24時間対応のAIチューターでハンズオン演習できます。これはFastAPI Backend Development Bootcamp学習パスの一部であり、ウェブとCoddyKitアプリ全体で進捗が同期されます。 FastAPI Backend Development Bootcampコースには全4レッスンが含まれています。

このレッスンの一部はまだ翻訳されておらず、英語で表示されています。

Why Metrics Matter

Logs tell you what happened in one request; metrics tell you how the whole service behaves over time. A metric is a numeric time series sampled at a fixed interval, which makes it cheap to store and fast to aggregate across millions of requests.

For a FastAPI backend you mainly care about three questions:

Is it serving traffic? request rate
Is it failing? error rate
Is it slow? request latency

Prometheus is a pull-based time-series database: it periodically scrapes an HTTP /metrics endpoint your app exposes, stores the samples, and lets you query them with PromQL. Grafana then turns those queries into dashboards.

The Four Metric Types

Prometheus has four core metric types. Picking the right one is the most important modeling decision.

Counter — only goes up (or resets to 0 on restart). Use for totals: requests served, errors, bytes sent. You query its rate, not its raw value.
Gauge — goes up and down. Use for current state: in-flight requests, memory usage, queue depth, connection pool size.
Histogram — buckets observations (e.g. latency) into pre-defined ranges, plus a _sum and _count. Lets you compute quantiles server-side.
Summary — like a histogram but computes quantiles client-side; cannot be aggregated across instances. Prefer histograms for latency in distributed services.

from prometheus_client import Counter, Gauge, Histogram

REQUESTS = Counter("http_requests_total", "Total HTTP requests", ["method", "path", "status"])
IN_FLIGHT = Gauge("http_requests_in_flight", "Requests currently being served")
LATENCY = Histogram("http_request_duration_seconds", "Request latency in seconds", ["method", "path"])

REQUESTS.labels("GET", "/users", "200").inc()
IN_FLIGHT.inc()
LATENCY.labels("GET", "/users").observe(0.042)
IN_FLIGHT.dec()

print("counter, gauge and histogram updated")

Exposing /metrics in FastAPI

To let Prometheus scrape your app, mount an endpoint that renders all registered metrics in the Prometheus text exposition format. The prometheus_client library gives you generate_latest() and the correct content type.

You can wire this by hand, or use prometheus-fastapi-instrumentator which auto-instruments request count and latency. Doing it by hand first makes the mechanics clear.

Prometheus is then configured to hit http://your-app:8000/metrics on a scrape interval (commonly 15s).

from fastapi import FastAPI, Response
from prometheus_client import generate_latest, CONTENT_TYPE_LATEST

app = FastAPI()

@app.get("/metrics")
def metrics():
    return Response(generate_latest(), media_type=CONTENT_TYPE_LATEST)

Labels and Cardinality

Labels turn one metric into many time series. http_requests_total{method="GET", path="/users", status="200"} is a distinct series from the same metric with status="500".

Cardinality is the number of unique label combinations. It is the single biggest way to blow up Prometheus memory.

Good labels: bounded sets — HTTP method, status code, route template.
Dangerous labels: unbounded values — user IDs, request IDs, raw URLs with path params, timestamps.

Always label by the route template (/users/{id}) not the resolved path (/users/4827), or every user creates new series.

A Middleware to Capture RED Signals

The cleanest way to instrument every endpoint is one ASGI middleware that records request count and latency, labeled by method, the route template, and status code.

Note how we read request.scope["route"].path (or the matched path template) instead of the raw URL to keep cardinality bounded. The same three labels feed both the Rate, Errors, and Duration views.

import time
from fastapi import FastAPI, Request
from prometheus_client import Counter, Histogram

REQUESTS = Counter("http_requests_total", "Total requests", ["method", "path", "status"])
LATENCY = Histogram("http_request_duration_seconds", "Latency", ["method", "path"])

app = FastAPI()

@app.middleware("http")
async def record_metrics(request: Request, call_next):
    route = request.scope.get("route")
    path = getattr(route, "path", request.url.path)
    start = time.perf_counter()
    response = await call_next(request)
    LATENCY.labels(request.method, path).observe(time.perf_counter() - start)
    REQUESTS.labels(request.method, path, str(response.status_code)).inc()
    return response

The RED Method

The RED method (popularized by Tom Wilkie) is the standard way to monitor request-driven services like a FastAPI API. For every service track:

Rate — requests per second
Errors — failed requests per second (or as a fraction)
Duration — distribution of request latency (p50/p95/p99)

RED is request-centric: it describes the experience of your callers. Three dashboards rows per service — rate, error ratio, latency percentiles — give you a consistent, comparable view across every microservice.

PromQL for Rate and Errors

Counters are queried with rate(), which computes the per-second average increase over a time window. The window (e.g. [5m]) should be at least 4x your scrape interval.

Rate — total requests per second across all routes:

sum(rate(http_requests_total[5m]))

Error ratio — fraction of requests returning 5xx:

sum(rate(http_requests_total{status=~"5.."}[5m])) / sum(rate(http_requests_total[5m]))

The =~ operator is a regex match, so "5.." captures 500, 502, 503, etc. Use by (path) to break a result down per route.

PromQL for Latency Percentiles

Because we used a Histogram, Prometheus stores cumulative bucket counters named http_request_duration_seconds_bucket with a le ("less than or equal") label. histogram_quantile() estimates a percentile from those buckets.

p95 latency across the service:

histogram_quantile(0.95, sum(rate(http_request_duration_seconds_bucket[5m])) by (le))

You must wrap the buckets in rate(...) first and keep the le label in the by() clause, otherwise the quantile is wrong. Percentiles (p95/p99) beat averages because a single slow tail is invisible in a mean.

# Default Histogram buckets are tuned for seconds; override for fast APIs:
from prometheus_client import Histogram

LATENCY = Histogram(
    "http_request_duration_seconds",
    "Request latency in seconds",
    ["method", "path"],
    buckets=(0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1.0, 2.5, 5.0),
)

The USE Method

RED watches the request flow; the USE method (Brendan Gregg) watches the resources that serve those requests. For every resource (CPU, memory, disk, connection pool, worker threads) track:

Utilization — percent of time the resource was busy
Saturation — how much work is queued waiting for it (e.g. requests waiting for a DB connection)
Errors — error events for that resource

USE catches problems RED misses: if your DB connection pool is saturated, latency rises before error rate does. A Gauge for db_pool_in_use vs pool size is a classic saturation signal for a FastAPI app.

from prometheus_client import Gauge

DB_POOL_SIZE = Gauge("db_pool_size", "Configured max DB connections")
DB_POOL_IN_USE = Gauge("db_pool_in_use", "DB connections currently checked out")
DB_POOL_WAITERS = Gauge("db_pool_waiters", "Requests waiting for a connection")

def snapshot_pool(pool):
    DB_POOL_SIZE.set(pool.size())
    DB_POOL_IN_USE.set(pool.checkedout())
    DB_POOL_WAITERS.set(pool.overflow() if pool.overflow() > 0 else 0)

Building the Grafana Dashboard

In Grafana you add Prometheus as a data source, then build one dashboard per service with panels backed by the PromQL above:

Rate panel (time series): sum(rate(http_requests_total[5m])) by (path)
Error ratio panel (stat / gauge): the 5xx ratio expression, formatted as a percent
Latency panel (time series): p50, p95 and p99 lines from histogram_quantile
Saturation panel: db_pool_in_use / db_pool_size

Use template variables (e.g. a $path dropdown from label_values(http_requests_total, path)) so one dashboard works for every route. Set thresholds (green/amber/red) on the error and latency panels so health is readable at a glance.

Alerting on SLOs

Dashboards are for humans looking; alerts are for being told. Define alert rules in Prometheus (or Grafana) directly on your RED metrics, ideally tied to a Service Level Objective.

A common pattern is a multi-window burn-rate alert: fire when the error ratio over both a short and a long window exceeds your error budget burn rate, which avoids both flapping and slow detection.

Keep alert labels meaningful (severity, service) so Alertmanager can route pages vs. tickets correctly. Alert on symptoms users feel (high error ratio, high p99 latency), not every internal cause.

groups:
  - name: api-slo
    rules:
      - alert: HighErrorRatio
        expr: |
          sum(rate(http_requests_total{status=~"5.."}[5m]))
            / sum(rate(http_requests_total[5m])) > 0.02
        for: 10m
        labels:
          severity: page
        annotations:
          summary: "5xx error ratio above 2% for 10m"

Quick Check

You are labeling a latency Histogram for a FastAPI endpoint /orders/{order_id}. Which labeling choice keeps cardinality bounded and the dashboards correct?

対話的なレッスンで答える

Recap

You now have an end-to-end observability path for a FastAPI service:

Instrument with prometheus_client — Counters for totals, Gauges for current state, Histograms for latency.
Expose a /metrics endpoint and scrape it with Prometheus on a fixed interval.
Control cardinality by labeling with route templates and bounded sets only.
RED (Rate, Errors, Duration) describes the request experience; USE (Utilization, Saturation, Errors) describes resource health.
Query with PromQL: rate() for counters, histogram_quantile() for percentiles.
Visualize in Grafana with templated dashboards and thresholds, and alert on user-facing symptoms tied to SLOs.

Together these give you a consistent, low-overhead view of whether your service is up, failing, or slow.

無料で開始

AI チューターと学ぶ FastAPI Backend Development Bootcamp — 無料

ブラウザでリアルコードを書いて実行し、24/7 の AI チューターから瞬時にサポートを受け、ウェブまたはアプリで続きから学習できます。

コース: 21
レッスン: 84

無料で学習を始めるコース全体を表示

よくある質問

「PrometheusメトリクスとRED／USEダッシュボード」レッスンは無料ですか？

はい。「PrometheusメトリクスとRED／USEダッシュボード」の完全なテキストはこのウェブで無料で読めます。インタラクティブに演習し（組み込みコードエディタと24時間対応のAIチューター）、FastAPI Backend Development Bootcampコースの残りをアンロックするには、CoddyKit PROにアップグレードしてください。 FastAPI Backend Development Bootcampコースには全4レッスンが含まれています。

「PrometheusメトリクスとRED／USEダッシュボード」で何を学びますか？

レイテンシー、エラー、飽和度のメトリクスを公開し、Grafanaダッシュボードでサービスの健全性を可視化します。ブラウザで直接実行するハンズオンコードでFastAPI Backend Development Bootcampを演習し、24時間対応のAIチューターがレッスンを進める中での質問に答えます。

FastAPI Backend Development Bootcampを始めるのに経験は必要ですか？

事前経験は必要ありません。CoddyKitのFastAPI Backend Development Bootcampは初級者から上級者向けに構成されているため、ここから始めるか最初から始めて、自分のペースで進むことができます。これはレッスン3/4です。

「PrometheusメトリクスとRED／USEダッシュボード」レッスンにはどのくらい時間がかかりますか？

ほとんどのCoddyKitレッスンは約5～10分かかります。各レッスンはコンパクトでインタラクティブなので、着実に進歩し、ウェブとアプリ全体で正確に前回の場所から再開できます。

このFastAPI Backend Development Bootcampレッスンでコードを書いて実行できますか？

はい。すべてのFastAPI Backend Development Bootcampレッスンに組み込みコードエディタが含まれているため、ブラウザでリアルコードを書いて実行し、即座のAIフィードバックを取得できます。ローカル設定は不要です。

このコースのすべてのレッスン

← FastAPI Backend Development Bootcampに戻る