0Pricing
PostgreSQL Performance & Query Optimization · Lesson

Recursive CTEs and Graph Queries

Explore how to optimize queries involving hierarchical data and graph traversal using recursive CTEs.

Recursive CTEs and Graph Queries is a free PostgreSQL Performance & Query Optimization lesson on CoddyKit — lesson 2 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the PostgreSQL Performance & Query Optimization learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

What is Hierarchical Data?

Many real-world datasets have a natural hierarchy. Think of an organizational chart where employees report to managers, or a bill of materials where components are made of sub-components.

Standard SQL queries can struggle to navigate these relationships efficiently across multiple levels without complex, nested subqueries or joins. This is where Recursive Common Table Expressions shine!

Meet Recursive CTEs

A Common Table Expression (CTE) acts like a temporary, named result set you can reference within a single SQL statement. They improve readability and organize complex queries.

A Recursive CTE is special because it can refer to itself, allowing it to repeatedly execute to process hierarchical or graph-like data. It's perfect for "find all descendants" or "trace a path" types of problems.

The Starting Point: Base Member

Every recursive CTE has two main parts, combined with UNION ALL. The first is the base member.

This non-recursive part defines the initial set of rows for the recursion. It's the "root" or starting point of your traversal. Think of it as the first step in your journey through the data.

Let's use an employees table with employee_id, name, and manager_id.

CREATE TABLE employees (
  employee_id INT PRIMARY KEY,
  name VARCHAR(50),
  manager_id INT
);

INSERT INTO employees (employee_id, name, manager_id) VALUES
(1, 'Alice', NULL),
(2, 'Bob', 1),
(3, 'Charlie', 1),
(4, 'David', 2),
(5, 'Eve', 2);

WITH RECURSIVE subordinates AS (
  SELECT employee_id, name, manager_id, 0 AS level
  FROM employees
  WHERE employee_id = 1
)
SELECT * FROM subordinates;

Iterating with the Recursive Member

The second part is the recursive member. This part references the CTE itself (subordinates in our example) and joins it with the base table (employees) to find the next level of data.

It runs repeatedly, processing the results from the previous iteration, until no new rows are returned. This is the "step-by-step" part of the journey.

CREATE TABLE employees (
  employee_id INT PRIMARY KEY,
  name VARCHAR(50),
  manager_id INT
);

INSERT INTO employees (employee_id, name, manager_id) VALUES
(1, 'Alice', NULL),
(2, 'Bob', 1),
(3, 'Charlie', 1),
(4, 'David', 2),
(5, 'Eve', 2);

WITH RECURSIVE subordinates AS (
  -- Base Member
  SELECT employee_id, name, manager_id, 0 AS level
  FROM employees
  WHERE employee_id = 1

  UNION ALL

  -- Recursive Member
  SELECT e.employee_id, e.name, e.manager_id, s.level + 1
  FROM employees e
  JOIN subordinates s ON e.manager_id = s.employee_id
)
SELECT * FROM subordinates WHERE level = 1; -- Just showing the first recursive step

How Recursion Stops

A recursive CTE needs a way to stop! The recursion automatically terminates when the recursive member produces no new rows. If it kept finding new rows forever, you'd have an infinite loop!

It's crucial that your recursive member's join condition and filters eventually stop matching rows, ensuring the query finishes. In our example, it stops when there are no more employees whose manager_id matches an employee_id found so far.

Tracing an Org Chart

Let's put it all together to find all subordinates of 'Alice' (employee ID 1), along with their reporting level.

The base member starts with Alice. The recursive member then finds Alice's direct reports (level 1), then their reports (level 2), and so on, until no more subordinates are found.

CREATE TABLE employees (
  employee_id INT PRIMARY KEY,
  name VARCHAR(50),
  manager_id INT
);

INSERT INTO employees (employee_id, name, manager_id) VALUES
(1, 'Alice', NULL),
(2, 'Bob', 1),
(3, 'Charlie', 1),
(4, 'David', 2),
(5, 'Eve', 2),
(6, 'Frank', 3);

WITH RECURSIVE subordinates AS (
  SELECT employee_id, name, manager_id, 0 AS level
  FROM employees
  WHERE employee_id = 1

  UNION ALL

  SELECT e.employee_id, e.name, e.manager_id, s.level + 1
  FROM employees e
  JOIN subordinates s ON e.manager_id = s.employee_id
)
SELECT employee_id, name, level
FROM subordinates
ORDER BY level, employee_id;

`UNION ALL` for Performance

You might wonder why we use UNION ALL and not just UNION.

  • UNION ALL: Combines all rows from both result sets, including duplicates. It's generally faster because it doesn't need to check for and remove duplicates.
  • UNION: Combines rows and removes any duplicates. In a recursive CTE, duplicate checking can add significant overhead and is often not necessary if your logic ensures unique paths or elements at each level.

For recursive traversals, UNION ALL is almost always preferred unless you specifically need to eliminate duplicates that your logic might produce.

Navigating Graphs: Friends of Friends

Recursive CTEs are also powerful for graph traversal. Imagine finding all connections in a social network or tracing dependencies.

Let's use a simple connections table to find all people connected to 'Alice' (ID 1) up to 2 levels deep.

CREATE TABLE connections (
  person_id INT,
  connected_to_id INT
);

INSERT INTO connections (person_id, connected_to_id) VALUES
(1, 2), -- Alice -> Bob
(1, 3), -- Alice -> Charlie
(2, 4), -- Bob -> David
(3, 5), -- Charlie -> Eve
(4, 6), -- David -> Frank
(5, 7); -- Eve -> Grace

WITH RECURSIVE path_finder AS (
  SELECT person_id AS start_node,
         connected_to_id AS end_node,
         1 AS depth
  FROM connections
  WHERE person_id = 1

  UNION ALL

  SELECT pf.start_node, c.connected_to_id, pf.depth + 1
  FROM connections c
  JOIN path_finder pf ON c.person_id = pf.end_node
  WHERE pf.depth < 2 -- Limit depth to avoid infinite loops or excessive recursion
)
SELECT DISTINCT start_node, end_node, depth
FROM path_finder
ORDER BY depth, end_node;

Optimizing Recursive Queries

Recursive CTEs can be powerful, but also resource-intensive if not managed well. Here are some tips:

  • Limit Depth: Always include a termination condition for depth (like level < max_depth) to prevent infinite loops or excessively long queries.
  • Index Keys: Ensure columns used in join conditions (e.g., employee_id, manager_id, person_id, connected_to_id) are indexed.
  • Filter Early: Apply filters in the base member to reduce the initial dataset.
  • Avoid Cycles: If your data can contain cycles (e.g., A -> B -> A), you might need to track the path taken (e.g., an array of visited nodes) to prevent infinite loops. PostgreSQL 14+ offers CYCLE clause for this.

Recursive CTE Structure

Consider a recursive CTE used to find all parts in a bill of materials, starting from a final product. The CTE is named bom_path.

Which of the following describes the correct structure and purpose of the recursive member of this CTE?

Recursive CTEs: A Recap

You've explored the power of Recursive CTEs!

  • They are essential for querying hierarchical data (like organizational charts) and performing graph traversals (like finding paths or connections).
  • A recursive CTE consists of a base member (starting point) and a recursive member (iterative step), combined by UNION ALL.
  • Recursion stops when the recursive member produces no new rows, but adding a depth limit is often a good practice.
  • Always consider indexing relevant columns and filtering early for optimal performance.

Mastering recursive CTEs opens up new possibilities for querying complex, interconnected datasets in PostgreSQL!

Frequently asked questions

Is the “Recursive CTEs and Graph Queries” lesson free?

Yes — the full text of “Recursive CTEs and Graph Queries” is free to read here on the web, and the PostgreSQL Performance & Query Optimization course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the PostgreSQL Performance & Query Optimization course, upgrade to CoddyKit PRO.

What will I learn in “Recursive CTEs and Graph Queries”?

Explore how to optimize queries involving hierarchical data and graph traversal using recursive CTEs. You practise PostgreSQL Performance & Query Optimization with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start PostgreSQL Performance & Query Optimization?

No prior experience is required. PostgreSQL Performance & Query Optimization on CoddyKit is structured for beginners through advanced learners; this is — lesson 2 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Recursive CTEs and Graph Queries” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this PostgreSQL Performance & Query Optimization lesson?

Yes. Every PostgreSQL Performance & Query Optimization lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. Optimizing Aggregates and Window Functions
  2. Recursive CTEs and Graph Queries
  3. Using Materialized Views for Performance
  4. Optimizing Queries with FILTER and Conditional Aggregation
← Back to PostgreSQL Performance & Query Optimization