การติดตามแบบกระจายด้วยสแปน OpenTelemetry

In a microservices system one user request can hop across an API gateway, an orders service, a payments service, and a database. When that request is slow, a single service's logs cannot tell you where the time went.

Distributed tracing stitches the whole journey together. Each unit of work becomes a span, spans are linked into a trace, and the trace reveals latency across every service boundary.

• Trace: the entire end-to-end request, identified by a traceId.
• Span: one operation (an HTTP call, a DB query) with a start time, duration, and parent.
• Context propagation: passing traceId and spanId across service boundaries, usually via HTTP headers.

OpenTelemetry (OTel) is the vendor-neutral standard for producing these spans in Node.js.

A span is the atomic building block of a trace. Every span carries the same trace identity but its own identity and timing.

• traceId: 16 bytes, shared by every span in the trace.
• spanId: 8 bytes, unique to this span.
• parentSpanId: links this span to the operation that caused it.
• name, startTime, endTime (duration = end - start).
• Attributes: key/value tags like http.method or db.system.
• Status: OK, ERROR, or UNSET.

Parent/child links form a tree. The root span is the whole request; child spans are the calls it makes. Visualized, the tree becomes the familiar waterfall you see in Jaeger or Tempo.

The fastest way to get spans is auto-instrumentation. The OTel Node SDK monkey-patches popular libraries (http, Express, pg, ioredis, etc.) so they emit spans without you writing tracing code.

Create a tracing.js file that starts the SDK before anything else, then run your app with node -r ./tracing.js app.js so it loads first.

With the SDK loaded, an incoming HTTP request automatically becomes a root span, and any outgoing http/fetch call or pg query becomes a child span underneath it.

• Inbound Express route → server span with http.method, http.route, http.status_code.
• Outbound HTTP call → client span, and the headers are injected automatically.
• Database query → client span with db.system and the statement.

This covers the boundaries for free. But auto-instrumentation does not understand your business logic — pricing rules, cache decisions, batch loops. For those you add spans manually.

To create spans manually you first obtain a tracer from the global trace API. Name it after the module or library producing the spans; the version is optional but helps when debugging instrumentation.

The tracer is the factory for all your manual spans.

Prefer tracer.startActiveSpan() over startSpan(). startActiveSpan makes the new span the active span for the duration of its callback, so any child spans created inside (including auto-instrumented ones) automatically attach as children.

The golden rules: always span.end() in a finally block, and record errors plus an ERROR status on failure.

Spans become useful when you enrich them. Three tools:

• Attributes — searchable key/value tags. Use semantic conventions (http.method, db.system, messaging.system) so backends understand them.
• Events — timestamped log lines anchored inside the span, e.g. span.addEvent('cache.miss').
• Status — set ERROR only on real failures; leave success as UNSET or OK.

Keep cardinality sane: never put a raw user ID or full SQL with literals into a high-traffic attribute if your backend indexes it — it can explode storage.

A trace only spans services if the trace context travels with the request. The W3C traceparent header carries the traceId, parent spanId, and sampling flag.

Auto-instrumentation injects and extracts this header for you on standard HTTP. When you do something non-standard (a custom transport, a message queue), you inject/extract manually with the propagation API.

The traceparent header has a fixed, parseable shape. Understanding it helps you debug broken traces (a missing child usually means a dropped header).

Format: version-traceId-parentId-flags, e.g.
00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01

• 00 — version
• 32 hex chars — the traceId
• 16 hex chars — the parent spanId
• 01 — flags (bit 0 = sampled)

Here is a tiny standalone parser to make the structure concrete.

Tracing every request at high traffic is expensive. Samplers decide which traces to keep. The decision propagates via the traceparent sampled flag, so a trace is kept or dropped consistently across all services.

• AlwaysOnSampler — keep everything (dev/low traffic).
• TraceIdRatioBasedSampler — keep a fixed fraction, e.g. 10%.
• ParentBasedSampler — respect the upstream decision; sample new roots by ratio. This is the production default.

Use head sampling (decide at the start) for simplicity, or tail sampling in a collector to always keep errors and slow traces.

Once spans reach a backend (Jaeger, Tempo, Honeycomb), you read the trace as a waterfall. Each bar is a span; its width is its duration; indentation shows parent/child.

How to find the bottleneck:

• Look for the widest child bar — that operation dominates the request.
• Watch for gaps between a parent and its first child — usually queueing, GC pauses, or un-instrumented work.
• Sequential bars that could run in parallel reveal a chance to use Promise.all.
• A red span with ERROR status points straight at the failing boundary.

The cross-service value: you can see that 80% of a 900ms request was spent inside the downstream payments service, not your own code.

You manually wrap a function in tracer.startActiveSpan('outer', cb). Inside the callback, your auto-instrumented HTTP client makes an outbound call. Which statement is correct?

You can now produce spans that expose latency across service boundaries:

• Trace = many spans sharing a traceId; each span has its own spanId and a parentSpanId.
• Auto-instrumentation (NodeSDK + auto-instrumentations-node, loaded with node -r) covers HTTP, DB, and queue boundaries for free.
• Manual spans with tracer.startActiveSpan() capture business logic; always end() in finally and set ERROR status on exceptions.
• Enrich spans with attributes and events, watching cardinality.
• Context propagation via the W3C traceparent header makes traces cross services; inject/extract manually for non-HTTP transports.
• Sampling (ParentBased + ratio) controls cost while keeping decisions consistent across services.
• Read the waterfall: widest bars, gaps, and serial calls reveal the real bottleneck.