System Observability: Logging, Metrics & Tracing (ELK + OpenTelemetry) · Lesson

Understanding Modern Log Formats

Learn about structured logging and common formats like JSON. Understand why structured logs are superior for machine parsing and analysis compared to plain text.

Understanding Modern Log Formats is a free System Observability: Logging, Metrics & Tracing (ELK + OpenTelemetry) lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the System Observability: Logging, Metrics & Tracing (ELK + OpenTelemetry) learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

What are Application Logs?

Imagine your application as a busy worker. How do you know what it's doing? That's where logs come in!

Logs are like a diary for your software. They record events, operations, and status messages as your application runs. They tell you:

When something happened
What action was performed
If an error occurred

These records are crucial for debugging, monitoring performance, and understanding system behavior.

The Traditional Way: Plain Text

Historically, logs were often simple lines of text. Each event was written as a human-readable string.

For example, a login event might look like this:

2023-10-27 10:30:00 INFO User 'alice' logged in from IP 192.168.1.100

This format is straightforward and easy for a human to read when looking at a few lines.

Plain Text: Hard for Machines

While plain text logs are human-friendly at a glance, they pose a big challenge for computers.

To find all logins from 'alice' or count errors from a specific IP address, a machine would need to:

Guess the date format
Extract the log level ('INFO')
Parse the username ('alice')
Identify the IP address

This process, called parsing, is complex and prone to errors because there's no fixed structure.

Hello, Structured Logging!

This is where structured logging comes to the rescue! It's a modern approach that outputs log data in a consistent, machine-readable format.

Instead of free-form text, each log entry is an object with clearly defined fields (like 'timestamp', 'level', 'user_id', 'message').

Think of it like organizing your notes into a spreadsheet instead of a jumbled notebook. Each piece of information has its own column.

JSON: Your Log's New Structure

The most popular format for structured logging today is JSON (JavaScript Object Notation).

JSON is lightweight, human-readable, and incredibly easy for machines to parse. It represents data as key-value pairs.

Here's how our 'alice' login event might look as a JSON log:

{"timestamp": "2023-10-27T10:30:00Z", "level": "INFO", "message": "User logged in", "user": "alice", "ip_address": "192.168.1.100"}

The Power of Structured Logs

Using structured formats like JSON unlocks powerful capabilities:

Easier Machine Parsing: Computers can directly read and understand each data field.
Efficient Searching: Quickly find logs where user="alice" or level="ERROR".
Better Analysis: Aggregate data, count events, and build dashboards based on specific fields.
No More Guessing: No need for complex regular expressions to extract data, reducing errors.

This transforms logs from simple text files into rich, queryable data.

Example: Outputting a JSON Log

Here's a simple Java example demonstrating how you might output a structured log in JSON format. In real applications, you'd use a logging library to handle this.

Try running it to see the structured output!

public class StructuredLogger {
  public static void main(String[] args) {
    // This is a simplified way to output a JSON log string.
    // Real-world apps use dedicated logging libraries for this.
    String jsonLog = "{\"timestamp\": \"2023-10-27T10:30:00Z\", \"level\": \"INFO\", \"message\": \"User logged in successfully\", \"user_id\": 123, \"ip_address\": \"192.168.1.100\"}";
    System.out.println(jsonLog);
  }
}

Inside a JSON Log Object

Let's break down the JSON log from the previous example:

{"timestamp": "2023-10-27T10:30:00Z", "level": "INFO", "message": "User logged in successfully", "user_id": 123, "ip_address": "192.168.1.100"}

Each piece of information is a key-value pair.
"timestamp" is the key, "2023-10-27T10:30:00Z" is its value.
"level" is the key, "INFO" is its value.

This explicit labeling makes every detail instantly accessible to machines.

Essential Fields in Structured Logs

While you can add any relevant data, some fields are commonly found and highly useful in structured logs:

timestamp: The exact time the event occurred (often in ISO 8601 format).
level: The severity of the log (e.g., DEBUG, INFO, WARN, ERROR, FATAL).
message: A human-readable description of the event.
service: The name of the application or service generating the log.
transaction_id: A unique ID to link related events across different services.
user_id: The ID of the user involved in the event.

Quick Check: Why Structured Logs?

You've learned about the differences between plain text and structured logs. Think about the key benefits.

Recap: Logs Get Organized!

Congratulations! You've taken a crucial step in understanding modern application logging.

We saw that traditional plain text logs are simple but hard for computers to analyze.
Structured logging provides a consistent, machine-readable format for log data.
JSON is the most popular structured format, using key-value pairs.
Structured logs enable powerful searching, filtering, and automated analysis, making your logs far more valuable.

Next, we'll explore how these structured logs are collected and managed in centralized systems!

Frequently asked questions

Is the “Understanding Modern Log Formats” lesson free?

Yes — the full text of “Understanding Modern Log Formats” is free to read here on the web, and the System Observability: Logging, Metrics & Tracing (ELK + OpenTelemetry) course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the System Observability: Logging, Metrics & Tracing (ELK + OpenTelemetry) course, upgrade to CoddyKit PRO.

What will I learn in “Understanding Modern Log Formats”?

Learn about structured logging and common formats like JSON. Understand why structured logs are superior for machine parsing and analysis compared to plain text. You practise System Observability: Logging, Metrics & Tracing (ELK + OpenTelemetry) with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start System Observability: Logging, Metrics & Tracing (ELK + OpenTelemetry)?

No prior experience is required. System Observability: Logging, Metrics & Tracing (ELK + OpenTelemetry) on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Understanding Modern Log Formats” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this System Observability: Logging, Metrics & Tracing (ELK + OpenTelemetry) lesson?

Yes. Every System Observability: Logging, Metrics & Tracing (ELK + OpenTelemetry) lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

← Back to System Observability: Logging, Metrics & Tracing (ELK + OpenTelemetry)