Metrics with Prometheus & Grafana

Metrics are numerical measurements that describe the state and performance of your applications and infrastructure. Think of them as vital signs for your software!

They provide valuable insights into:

• Performance: How fast is your app responding?
• Resource Usage: Is your server running out of memory?
• Availability: Is your service up and running for users?

By tracking metrics, you can understand trends, detect anomalies, and troubleshoot issues proactively.

Prometheus is an open-source monitoring system designed for reliability and scalability. It's a key tool in the cloud-native observability stack.

Its primary job is to:

• Scrape: Collect metrics from configured targets (like your applications).
• Store: Save these metrics in a time-series database.
• Query: Allow you to ask questions about the collected data.

Prometheus helps you get a clear picture of what's happening inside your Kubernetes cluster.

Prometheus uses a "pull" model to collect metrics. Instead of applications pushing data to Prometheus, Prometheus actively "scrapes" or pulls metrics from them.

Here's how it generally works:

1. Your application exposes metrics on a specific HTTP endpoint (often /metrics).
2. Prometheus is configured with a list of targets (like your application's IP and port).
3. At regular intervals, Prometheus sends an HTTP request to each target's /metrics endpoint.
4. The target responds with its current metrics, which Prometheus then stores.

Prometheus stores data as time series: streams of timestamped values. Each time series is uniquely identified by a metric name and a set of key-value pairs called labels.

Example: http_requests_total{method="GET", path="/api"}

Prometheus has core metric types:

• Counter: A cumulative metric that only increases (e.g., total requests).
• Gauge: A value that can go up or down (e.g., current CPU usage).
• Histogram/Summary: For sampling observations and counting them in configurable buckets (e.g., request durations).

For Prometheus to scrape metrics, your application needs to expose them in a format Prometheus understands. This is usually done via an HTTP endpoint.

Many programming languages have client libraries that make it easy to instrument your code:

• Increment counters on events.
• Set gauges for current states.
• Observe durations of operations.

Let's look at a super simple example of an app exposing a basic counter metric.

Here's a tiny Python Flask app that exposes a simple counter metric. It simulates counting page views.

Run this code and then try accessing /metrics in your browser/terminal (e.g., after running, visit http://localhost:5000/metrics).

In Kubernetes, deploying Prometheus efficiently often involves the Prometheus Operator. This operator helps manage Prometheus instances and configurations.

Key Kubernetes resources for Prometheus:

• ServiceMonitor: Tells Prometheus *where* to find metrics endpoints (e.g., which Services and Pods to scrape).
• PodMonitor: Similar to ServiceMonitor but targets Pods directly.

These resources allow Prometheus to automatically discover and scrape metrics from your deployed applications.

While Prometheus collects and stores metrics, Grafana is your go-to tool for visualizing them. It's an open-source analytics and interactive visualization web application.

Grafana allows you to:

• Create beautiful and informative dashboards.
• Query multiple data sources (like Prometheus).
• Set up alerts based on metric thresholds.

It transforms raw metric data into actionable insights.

Grafana doesn't store metric data itself; it queries data from external data sources. To visualize your Kubernetes metrics, you'll configure Prometheus as a data source in Grafana.

Steps:

1. In Grafana, add a new data source.
2. Select "Prometheus" as the type.
3. Provide the URL of your Prometheus server.
4. Start building dashboards using Prometheus's query language, PromQL!

This pairing gives you full control over monitoring your cluster.

Time to check your understanding of Prometheus and Grafana!

Great job! In this lesson, you learned about:

• The importance of metrics for observability.
• Prometheus as a powerful system for scraping and storing time-series metrics.
• How applications expose metrics for Prometheus to collect.
• Grafana for creating insightful dashboards and visualizing your metric data.

Understanding these tools is crucial for keeping your Kubernetes applications healthy and performant. Keep exploring how to instrument your own applications!