Editing YAML Configuration Files with yq

yq is a portable command-line YAML processor, similar to how jq handles JSON. It lets you read, filter, and edit YAML files without writing a script in Python or Ruby.

There are two popular tools named yq:

• mikefarah/yq (Go) — actively maintained, supports YAML, JSON, XML, TOML. This lesson uses this version.
• kislyuk/yq (Python) — a jq wrapper for YAML; syntax differs.

Install the Go version:

• brew install yq on macOS
• snap install yq on Linux
• Or download the binary: wget https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 -O /usr/local/bin/yq && chmod +x /usr/local/bin/yq

Verify: yq --version should print v4.x.x. Version 4 uses a different expression syntax than v3, so the version matters.

Before editing anything, learn to read YAML fields. Given a Kubernetes Deployment, you can extract any nested value with dot-notation paths.

Example deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-app
  namespace: production
spec:
  replicas: 3
  template:
    spec:
      containers:
        - name: app
          image: my-app:1.0.0

Key read commands:

• yq '.metadata.name' deployment.yaml — prints my-app
• yq '.spec.replicas' deployment.yaml — prints 3
• yq '.spec.template.spec.containers[0].image' deployment.yaml — prints my-app:1.0.0

Output is plain text by default (no quotes). Add -r flag or use | yq -r if you need raw strings in scripts.

The most important flag for real-world use is -i (in-place). Without it, yq prints results to stdout and leaves the file unchanged.

Syntax:

• Read only (stdout): yq '.spec.replicas' file.yaml
• In-place edit: yq -i '.spec.replicas = 5' file.yaml

The assignment operator = sets a value. The expression is a full yq filter, so you can combine reading and writing in one pass.

Important: yq -i rewrites the file completely. Comments placed on the same line as a field are generally preserved, but standalone comment blocks may move. Always commit your YAML to version control before running bulk in-place edits.

Test without -i first, then add it once you are happy with the output.

A very common CI task is bumping the Docker image tag in a Kubernetes manifest after a new image is built. With yq this becomes a one-liner.

The pattern is:

• Target the container by name using select() to avoid hard-coding array index 0.
• Use |= (update operator) or = to set the new value.

Using array index (fragile if containers list changes):

• yq -i '.spec.template.spec.containers[0].image = "my-app:2.1.0"' deployment.yaml

Using select() (robust):

• yq -i '(.spec.template.spec.containers[] | select(.name == "app")).image = "my-app:2.1.0"' deployment.yaml

In a CI pipeline you would pass the tag as a shell variable:

Beyond updating existing fields, yq can add new keys or delete existing ones.

Adding a field:

• Simply assign to a path that does not exist: yq -i '.metadata.labels.version = "v2"' file.yaml
• If the parent key (labels) is missing, yq creates it automatically.

Deleting a field:

• Use the del() function: yq -i 'del(.metadata.annotations)' file.yaml
• Delete an array element by index: yq -i 'del(.spec.template.spec.containers[1])' file.yaml

Adding an element to an array:

• yq -i '.spec.template.spec.containers += [{"name": "sidecar", "image": "envoy:latest"}]' file.yaml

Kubernetes manifests often bundle multiple resources in a single file separated by ---. By default yq processes all documents in such a file.

Key techniques:

• List all document kinds: yq '.[].kind' multi.yaml — note the leading .[] to iterate documents.
• Target a specific document by kind: yq 'select(.kind == "Service")' multi.yaml
• Edit only matching documents in place:

yq -i 'select(.kind == "Deployment").spec.replicas = 2' multi.yaml

Documents that do not match the select() predicate are passed through unchanged, so your Service, ConfigMap, and other resources stay intact.

To split a multi-doc file into individual files you can loop over yq output or use:

• yq -s '.kind' multi.yaml — writes one file per document named after its .kind value.

CI configuration files (.github/workflows/*.yml, .gitlab-ci.yml) are also YAML. The same yq commands work, though the paths can be deeply nested.

Common CI patching tasks:

• Pin a runner version: update runs-on across all jobs.
• Update an action version: find steps that use a particular action and bump its uses field.
• Toggle a flag: enable or disable a workflow-level setting.

Example: update all steps that use actions/checkout to v4:

yq -i '(.jobs[].steps[] | select(.uses == "actions/checkout@v3")).uses = "actions/checkout@v4"' .github/workflows/ci.yml

This idiom — iterate with [], narrow with select(), assign with = — is the core pattern for any structured YAML edit.

Hardcoding values in yq expressions makes scripts brittle. yq supports injecting shell variables using the env() function or the strenv() shorthand.

• env(VAR_NAME) — reads the environment variable and converts it to the appropriate YAML type (number stays a number, string stays a string).
• strenv(VAR_NAME) — always returns a string, useful for image tags.

This avoids the quoting nightmare of interpolating variables inside double-quoted shell strings with embedded YAML paths.

Pattern:

export IMAGE_TAG="my-app:3.0.0"
yq -i '.spec.template.spec.containers[0].image = strenv(IMAGE_TAG)' deployment.yaml

Use env() when setting numeric fields like replicas so the YAML type is preserved (integer, not a quoted string).

Sometimes you need to apply a patch file (a small override YAML) onto a base config — for example, environment-specific overrides in Kustomize-style workflows.

yq can merge two files using the * merge operator:

• yq '. *= load("patch.yaml")' base.yaml — deep-merges patch into base, writing to stdout.
• Add -i to update base in place: yq -i '. *= load("patch.yaml")' base.yaml

Merge behaviour:

• Scalar values in the patch overwrite the base.
• Mappings are deep-merged (keys not in patch are preserved).
• Sequences (arrays) are replaced by default, not appended. Use *+ to append instead.

This pattern replaces brittle sed scripts that break on whitespace changes.

Before applying a patched YAML to a cluster, it is good practice to validate it and optionally convert it to JSON for other tools.

Validate syntax:

• yq '.' file.yaml && echo "Valid" — yq exits with code 1 on parse errors, so this works in CI gates.

Convert YAML to JSON:

• yq -o=json '.' file.yaml — outputs pretty JSON.
• Pipe to jq for further JSON processing: yq -o=json '.' file.yaml | jq '.metadata.name'

Convert JSON to YAML:

• yq -P '.' file.json — the -P flag forces YAML (prettyprint) output when the input is JSON.

These conversions make yq a bridge between YAML-native tools (Helm, kubectl) and JSON-native tools (Terraform, AWS CLI, jq).

Putting all the techniques together: a real CI script that patches a Kubernetes Deployment manifest as part of a GitOps pipeline.

The script:

1. Validates the input YAML before touching it.
2. Uses env() / strenv() for all variable substitutions.
3. Updates the container image tag using a name-based select().
4. Bumps the replica count.
5. Stamps a deploy-time annotation with the current timestamp.
6. Validates the output again before committing.

This pattern ensures that even if the pipeline runs concurrently, each step is atomic and auditable.

Test your understanding of editing multi-document Kubernetes YAML files with yq.

You have completed the lesson on editing YAML configuration files with yq. Here is a concise summary of everything covered:

• Installation: Use the mikefarah/yq Go binary (v4). Verify with yq --version.
• Reading: Dot-notation paths like .spec.replicas; array access with [0] or [] iteration.
• In-place editing: The -i flag rewrites the file. Always preview without -i first.
• Robust targeting: Prefer select(.name == "app") over hard-coded array indices.
• Adding / deleting: Assign to a new path to create it; use del() to remove fields.
• Multi-document files: Use select(.kind == "...") to target one resource and leave others untouched.
• CI variables: Use strenv(VAR) for strings and env(VAR) for typed values — avoids shell quoting bugs.
• Merging: . *= load("patch.yaml") deep-merges an override file without losing unpatched keys.
• Validation & conversion: yq '.' as a lint gate; -o=json and -P for format conversion.

The core pattern for any YAML patch in CI is: validate → select → assign → validate. Combine this with strenv() and select() and you will never need to reach for fragile sed one-liners again.