Structured Logging Best Practices
Implement structured logging for easier parsing, analysis, and faster debugging of production issues.
Structured Logging Best Practices is a free Production Debugging & Incident Response Playbook lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Production Debugging & Incident Response Playbook learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
What are Logs?
Logs are records of events that happen in your application or system. Think of them as a diary for your software!
They're crucial for understanding what your program is doing, especially when things go wrong in a live "production" environment.
The Messy Truth
Often, logs are just plain text strings. This is called unstructured logging. While easy to write, unstructured logs are hard for computers to read and analyze, making debugging a slow, manual process.
Consider this example:
import logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
def process_order(order_id, item_count):
logger.info(f"Processing order {order_id} with {item_count} items.")
if item_count > 10:
logger.warning(f"Large order detected for {order_id}. Items: {item_count}.")
logger.info(f"Order {order_id} processed successfully.")
if __name__ == "__main__":
process_order("ORD-123", 5)
process_order("ORD-456", 12)What is Structured Logging?
Structured logging means your logs are formatted as machine-readable data, not just free-form text. The most common format is JSON.
Instead of a single string, each log entry is an object with key-value pairs. This makes them easy to search, filter, and analyze programmatically.
Why Structured Logging Rocks
Structured logs offer many advantages:
- Faster Debugging: Quickly find relevant events.
- Better Analysis: Easily query and aggregate data.
- Automated Tools: Integrate with monitoring and alerting systems.
- Consistency: Ensures all logs contain expected fields.
JSON is King
While other formats exist, JSON (JavaScript Object Notation) is the most popular choice for structured logging due to its simplicity and widespread support.
A JSON log entry is a self-contained object, making it incredibly versatile for storing varied data. Here's what a structured log might look like:
{
"timestamp": "2023-10-27T10:30:00Z",
"level": "INFO",
"service": "order-processor",
"message": "Order processed successfully",
"order_id": "ORD-123",
"item_count": 5
}Code It Up!
Let's see how to implement structured logging. Many languages have libraries that make this easy. Here's a basic Python example using the standard logging module with a custom JSON formatter.
import logging
import json
class JsonFormatter(logging.Formatter):
def format(self, record):
log_entry = {
"timestamp": self.formatTime(record, self.datefmt),
"level": record.levelname,
"name": record.name,
"message": record.getMessage(),
"file": record.filename,
"line": record.lineno
}
if hasattr(record, 'order_id'):
log_entry['order_id'] = record.order_id
if hasattr(record, 'item_count'):
log_entry['item_count'] = record.item_count
return json.dumps(log_entry)
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
handler = logging.StreamHandler()
handler.setFormatter(JsonFormatter())
logger.addHandler(handler)
def process_order(order_id, item_count):
extra_data = {'order_id': order_id, 'item_count': item_count}
logger.info("Processing order", extra=extra_data)
if item_count > 10:
logger.warning("Large order detected", extra=extra_data)
logger.info("Order processed successfully", extra=extra_data)
if __name__ == "__main__":
process_order("ORD-123", 5)
process_order("ORD-456", 12)Must-Have Fields
Every structured log entry should include these core fields for effective analysis:
timestamp: When the event happened (ISO 8601 format).level: Severity (INFO, WARN, ERROR, DEBUG).service: Which service or application generated the log.message: A human-readable summary of the event.hostname/pod_name: Where the log originated.
Enrich Your Logs
Beyond essential fields, add contextual data specific to the event. This is key for tracing requests across distributed systems, helping you connect the dots when debugging complex issues.
request_id: To track a single user request.user_id: To identify the user involved.transaction_id: For specific business transactions.
import logging
import json
import uuid
# Reusing the JsonFormatter from previous scene
class JsonFormatter(logging.Formatter):
def format(self, record):
log_entry = {
"timestamp": self.formatTime(record, self.datefmt),
"level": record.levelname,
"message": record.getMessage()
}
for key, value in record.__dict__.items():
if not key.startswith('_') and key not in ['name', 'levelname', 'pathname', 'filename', 'module', 'exc_info', 'exc_text', 'stack_info', 'lineno', 'funcName', 'created', 'msecs', 'relativeCreated', 'thread', 'threadName', 'processName', 'process', 'args', 'msg', 'asctime']:
log_entry[key] = value
return json.dumps(log_entry)
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
handler = logging.StreamHandler()
handler.setFormatter(JsonFormatter())
logger.addHandler(handler)
def handle_web_request(user_id):
request_id = str(uuid.uuid4())[:8]
extra_data = {'request_id': request_id, 'user_id': user_id, 'service': 'api-gateway'}
logger.info("Received web request", extra=extra_data)
if user_id == "user-vip":
logger.info("VIP user request detected", extra=extra_data)
else:
logger.debug("Standard user request", extra=extra_data)
logger.info("Request processed", extra=extra_data)
if __name__ == "__main__":
handle_web_request("user-123")
handle_web_request("user-vip")Log Levels
Log levels help categorize the severity and importance of a log message. Common levels include:
- DEBUG: Detailed info, only useful when diagnosing problems.
- INFO: Confirmation that things are working as expected.
- WARN: An unexpected event, but the application is still running.
- ERROR: An error that prevents some functionality from working.
- CRITICAL: A severe error, application might be unable to continue.
Quick Check
Structured logging is a powerful technique for improving observability. Let's test your understanding of its key advantages.
Structured Logging Recap
You've learned about the power of structured logging! By formatting your logs as machine-readable data (like JSON), you unlock faster debugging, better analysis, and seamless integration with monitoring tools.
Remember to include essential fields and contextual data to make your logs truly useful for diagnosing issues in production.
Frequently asked questions
Is the “Structured Logging Best Practices” lesson free?
Yes — the full text of “Structured Logging Best Practices” is free to read here on the web, and the Production Debugging & Incident Response Playbook course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Production Debugging & Incident Response Playbook course, upgrade to CoddyKit PRO.
What will I learn in “Structured Logging Best Practices”?
Implement structured logging for easier parsing, analysis, and faster debugging of production issues. You practise Production Debugging & Incident Response Playbook with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Production Debugging & Incident Response Playbook?
No prior experience is required. Production Debugging & Incident Response Playbook on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Structured Logging Best Practices” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Production Debugging & Incident Response Playbook lesson?
Yes. Every Production Debugging & Incident Response Playbook lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Structured Logging Best Practices
- Metrics, Dashboards, and Observability
- Designing Smart Alerting Strategies
- Log Aggregation and Retention Strategies