PricingLoginStart free
Production Debugging & Incident Response Playbook · 课时

用分布式追踪定位延迟热点

学习如何使用分布式追踪跟踪单个请求在各服务间的路径，识别延迟热点，并在生产环境调试期间将追踪与日志关联起来。

第 4 / 4 课13 个步骤

用分布式追踪定位延迟热点 是 CoddyKit 上的免费 Production Debugging & Incident Response Playbook 课时。 这是第 4 节课，共 4 节。 你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。 这是 Production Debugging & Incident Response Playbook 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 Production Debugging & Incident Response Playbook 课程共包含 4 节课。

本课时的部分内容尚未翻译，以英文显示。

Why Distributed Tracing

In a microservice system a single user request can fan out across dozens of services. When it is slow, which service is to blame?

Distributed tracing answers this by attaching a shared trace_id to a request and recording a span for every operation it touches.

  • A trace = the whole request journey
  • A span = one timed unit of work

Anatomy of a Span

Each span carries timing and context so you can reconstruct the call tree.

  • trace_id links all spans of one request
  • span_id identifies the operation
  • parent_id records who called it
  • start/end timestamps give duration
{
  "trace_id": "abc123",
  "span_id": "s2",
  "parent_id": "s1",
  "name": "db.query.users",
  "start_ms": 1042,
  "end_ms": 1310
}

Context Propagation

For spans to join one trace, the trace_id must travel with the request. This is called context propagation.

Most systems inject standard headers like traceparent (W3C Trace Context) into outgoing HTTP calls and message metadata.

If propagation breaks, traces fragment and the call tree falls apart.

GET /orders HTTP/1.1
traceparent: 00-abc123-s1-01

Instrumenting Code

You create spans around the operations you want to measure. Auto-instrumentation covers common libraries; manual spans capture your own logic.

The example below wraps a function in a span using OpenTelemetry conventions.

with tracer.start_as_current_span('charge_card') as span:
    span.set_attribute('amount', 42)
    result = payment.charge(42)
    span.set_attribute('status', result.status)

Reading the Waterfall

Tracing UIs show spans as a waterfall. The widest bar that is NOT just waiting on a child is usually your hotspot.

  • Long bars with no children = local CPU/IO cost
  • Long bars full of children = downstream cost
  • Gaps between spans = queueing or untraced work

Sampling Strategies

Tracing every request is expensive. Sampling keeps volume manageable.

  • Head sampling: decide at the start (e.g. keep 5%)
  • Tail sampling: decide after the trace ends, keeping slow or errored traces

For debugging latency, tail sampling on high duration is invaluable.

Correlating Traces and Logs

A trace tells you where; logs tell you why. Stamp every log line with the active trace_id so you can jump from a slow span straight to its logs.

import logging
logging.info('cache miss', extra={'trace_id': current_trace_id()})

Span Attributes and Events

Attributes are key/value tags on a span (db statement, HTTP status). Events are timestamped points inside a span (retry, lock acquired).

Rich attributes let you filter traces like 'all spans where db.rows > 10000', turning tracing into a query tool.

span.add_event('retry', {'attempt': 2})
span.set_attribute('db.rows', 12044)

Finding the Critical Path

Total latency is not the sum of all spans. Parallel spans overlap. The critical path is the chain of spans that actually determines end-to-end time.

Optimizing a span NOT on the critical path will not make the request faster.

Tracing Async and Queues

Across queues, the consumer runs later than the producer. Propagate the context inside the message so the consumer span links back as a follows-from relationship instead of a parent-child one.

producer:  msg.headers['traceparent'] = inject_context()
consumer:  ctx = extract_context(msg.headers)

A Debugging Workflow

Put it together when an endpoint is slow in production:

  • Filter traces for that endpoint sorted by duration
  • Open the slowest trace and read the waterfall
  • Identify the dominant span on the critical path
  • Jump to that span's logs via trace_id
  • Fix, then re-check the latency distribution

Quick Check

Test your understanding of distributed tracing.

在交互式课程中回答

Recap

You learned how distributed tracing reconstructs a request across services using trace_id, spans, and context propagation.

  • Read waterfalls to find hotspots
  • Focus on the critical path, not total span time
  • Use tail sampling to keep slow traces
  • Correlate spans with logs for the full story
免费开始

用 AI 导师学习 Production Debugging & Incident Response Playbook — 免费

在浏览器中编写并运行真实代码，获得全天候 AI 导师的即时帮助，并在网页或应用中继续学习。

课程
12
课程
48
免费开始学习查看完整课程

常见问题解答

「用分布式追踪定位延迟热点」课时是免费的吗？

是的 — 「用分布式追踪定位延迟热点」的完整文本可在网页上免费阅读。要进行交互式练习（内置代码编辑器和全天候 AI 导师）并解锁 Production Debugging & Incident Response Playbook 课程的其余内容，请升级到 CoddyKit PRO。 Production Debugging & Incident Response Playbook 课程共包含 4 节课。

「用分布式追踪定位延迟热点」这节课中我会学到什么？

学习如何使用分布式追踪跟踪单个请求在各服务间的路径，识别延迟热点，并在生产环境调试期间将追踪与日志关联起来。 你通过在浏览器中直接运行的动手代码来练习 Production Debugging & Incident Response Playbook，全天候 AI 导师会在你学习这节课的过程中回答你的问题。

学习 Production Debugging & Incident Response Playbook 需要有经验吗？

无需任何先前经验。CoddyKit 上的 Production Debugging & Incident Response Playbook 课程适合初学者到高级学习者，你可以从这里开始或从头开始，按照自己的节奏学习。 这是第 4 节课，共 4 节。

「用分布式追踪定位延迟热点」课时需要多长时间？

大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动，所以你能稳步进步，并在网页和应用中从离开的地方继续。

我能在这节 Production Debugging & Incident Response Playbook 课中编写并运行代码吗？

能。每节 Production Debugging & Incident Response Playbook 课都包含内置代码编辑器，你可以在浏览器中直接编写并运行真实代码，并获得即时 AI 反馈 — 无需本地设置。

此课程中的所有课时

  1. 远程调试在线应用
  2. 使用核心转储进行事后调试
  3. 内存与 CPU 性能分析技术
  4. 用分布式追踪定位延迟热点
← 返回 Production Debugging & Incident Response Playbook