Transforming and Building JSON Objects with jq

Raw JSON from APIs or log files rarely has the exact shape you need. You might receive a large object but only want specific fields, or you need to rename keys, flatten nested structures, or build a completely new payload to send to another service.

jq is a lightweight, powerful command-line JSON processor that makes these transformations possible in a single pipeline. In this lesson you will learn the three core techniques for reshaping data:

• Object construction — build a new JSON object from scratch
• map — apply a transformation to every element of an array
• to_entries / from_entries — treat an object's key-value pairs as an array so you can filter and rebuild them

All examples assume jq is installed (apt install jq / brew install jq).

The most fundamental jq feature is object construction: wrapping expressions in {} to build a new JSON object. You choose which fields to include and what to call them.

Syntax:

• { newKey: .existingField } — rename a field
• { name, age } — shorthand when the new key matches the field name
• { total: (.price * .qty) } — compute a value inline

The snippet below reads a product JSON and produces a leaner shape with a computed subtotal field.

Real-world JSON is often nested. jq lets you reach into nested paths inside an object constructor, flattening the structure at the same time.

Use dot-path notation inside the constructor value expression:

• { city: .address.city }
• { lat: .location.coords.lat }

The example below takes a deeply nested user record and produces a flat summary suitable for a CSV header row or an API request body.

map(expr) is jq's equivalent of a for-each: it applies expr to every element of an input array and returns a new array of the same length.

Key points:

• map(expr) is sugar for [.[] | expr]
• The expression inside can be any jq filter — including object construction
• Chain with select() to filter before transforming

The snippet processes a list of orders, keeping only the fields needed for a shipping manifest.

Inside map you can compute new values, convert types, and combine fields — not just copy them. Common patterns include:

• String interpolation: "\(.first) \(.last)"
• Arithmetic: (.price * 1.2 | round) for a 20% markup
• Conditionals: if .score >= 90 then "A" else "B" end

The example below enriches a list of employees by adding a computed fullName and a seniority label based on years of experience.

to_entries converts a JSON object into an array of {key, value} pairs. This unlocks array operations (map, select, sort) on an object's fields — something you cannot do directly on an object.

Example transformation:

• Input: {"a": 1, "b": 2}
• Output: [{"key": "a", "value": 1}, {"key": "b", "value": 2}]

The reverse operation is from_entries, which turns that array back into an object. Together they form the to_entries | map(...) | from_entries idiom for object-level transformations.

One of the most practical uses of to_entries is dynamically filtering which keys to keep or drop based on the key name itself — something object construction cannot do when you do not know the key names in advance.

Pattern:

• to_entries | map(select(.key | test("regex"))) | from_entries
• to_entries | map(select(.key != "secret")) | from_entries

The snippet below strips all keys that start with an underscore (internal/private fields) before forwarding a config object to an external service.

Object construction renames keys when you know their names at write time. to_entries lets you rename keys programmatically — for example converting camelCase to snake_case, or adding a prefix.

Inside map you update the .key field of each entry, then pipe to from_entries:

• map(.key |= gsub("(?<=[a-z])(?=[A-Z])"; "_") | .key |= ascii_downcase) — camelCase to snake_case
• map(.key |= "app_" + .) — add a prefix to every key

The example prefixes all environment variable names with APP_ to namespace them before injection into a container.

The pattern to_entries | map(...) | from_entries is so common that jq provides a shorthand: with_entries(expr).

It is exactly equivalent but more concise:

• with_entries(.value |= . * 2) — double every numeric value
• with_entries(select(.value != null)) — drop null-valued keys
• with_entries(.key |= ascii_upcase) — uppercase all keys

The snippet removes all keys whose value is null or an empty string — a common cleanup step before sending a PATCH request to a REST API.

Real transformations chain multiple jq operations together. A typical pipeline for preparing an API payload might:

1. Filter the input array with map(select(...))
2. Reshape each element with object construction
3. Add computed fields
4. Sort the result

The example below reads a list of server metrics, keeps only servers with high CPU usage, and produces a compact alert payload ready to POST to a webhook.

jq can merge inputs and construct objects that draw from multiple JSON sources using the addition operator + and variable binding with as $var.

Useful patterns:

• obj1 + obj2 — merge two objects (right-side wins on key conflicts)
• --argjson — pass a second JSON document as a variable
• $ENV — read environment variables directly inside jq

The snippet merges a base configuration with environment-specific overrides — a common pattern for 12-factor application config management in shell scripts.

You have covered the essential techniques for reshaping JSON with jq:

• Object construction {} — build new objects by picking, renaming, and computing fields from an input
• map(expr) — apply any transformation to every element of an array, including nested object construction and select() for filtering
• to_entries / from_entries — convert an object to an array of {key, value} pairs, enabling array operations on keys and values, then convert back
• with_entries(expr) — the concise shorthand for the full to_entries → map → from_entries pipeline
• Merging objects with + and --argjson for multi-source payloads

These building blocks compose: filter with select, reshape with object construction, enrich with computed fields, and chain it all in a single readable jq expression. Mastering these patterns means you can wrangle virtually any JSON payload directly in the shell without writing a dedicated script in Python or Node.