PostgreSQL Performance & Query Optimization · レッスン

相関するカラムの多変量統計

プランナーが独立していると仮定する依存関係を捉えるCREATE STATISTICSオブジェクトを作成します。

「相関するカラムの多変量統計」はCoddyKit上の無料PostgreSQL Performance & Query Optimizationレッスンです。これはレッスン2/4です。下記で完全なレッスンを無料で読むことができます。その後、ブラウザ内の組み込みコードエディタと24時間対応のAIチューターでハンズオン演習できます。これはPostgreSQL Performance & Query Optimization学習パスの一部であり、ウェブとCoddyKitアプリ全体で進捗が同期されます。 PostgreSQL Performance & Query Optimizationコースには全4レッスンが含まれています。

このレッスンの一部はまだ翻訳されておらず、英語で表示されています。

The Independence Assumption

When PostgreSQL estimates how many rows a query will return, it leans on per-column statistics stored in pg_statistic. To combine predicates on multiple columns, the planner makes a crucial simplifying assumption: the columns are statistically independent.

Under independence, the selectivity of WHERE a = 1 AND b = 2 is computed as sel(a=1) * sel(b=2). That multiplication is fast and correct — but only when the columns truly are unrelated.

In real schemas, columns are frequently correlated: a city implies a postal code, a product implies a category, an order date implies a fiscal quarter. When the planner multiplies selectivities for correlated columns, its estimate collapses far below reality.

How Bad Estimates Hurt You

A row-count estimate that is off by orders of magnitude steers the planner toward the wrong plan:

Underestimate → planner picks a nested loop expecting 3 rows, but 300,000 arrive → the loop executes its inner side hundreds of thousands of times.
Underestimate → planner chooses an index scan + heap fetches instead of a single sequential scan that would have been cheaper.
Bad join order → a large intermediate result is materialized early, blowing up memory and spilling to disk.

The symptom you see in EXPLAIN ANALYZE is a wide gap between rows= (estimated) and actual rows=. That gap is your signal that correlation may be the culprit.

EXPLAIN ANALYZE
SELECT * FROM addresses
WHERE city = 'New York'
  AND state = 'NY';

Seeing the Misestimate

Consider an addresses table where city functionally determines state — every row with city = 'New York' also has state = 'NY'. The two predicates select the same rows, so the combined selectivity equals sel(city) alone.

But the planner multiplies: sel(city) * sel(state), producing an estimate that can be 10× or 100× too small. In the EXPLAIN ANALYZE output below, watch the gap between the estimated and actual row counts on the scan node.

EXPLAIN ANALYZE
SELECT count(*) FROM addresses
WHERE city = 'New York'
  AND state = 'NY';
-- Seq Scan ... (rows=12 ...) (actual ... rows=8400 ...)
--                    ^estimate           ^reality

Enter CREATE STATISTICS

PostgreSQL 10+ lets you teach the planner about column relationships with extended statistics objects, created via CREATE STATISTICS.

An extended statistics object names a set of columns (or expressions) on one table and one or more kinds of statistics to gather over them. Once created and analyzed, the planner consults these multivariate statistics instead of blindly multiplying per-column selectivities.

The three kinds are:

ndistinct — number of distinct combinations of the listed columns.
dependencies — functional-dependency degrees between columns.
mcv — most-common-value lists over the column group.

CREATE STATISTICS stat_addr_city_state
    ON city, state
    FROM addresses;

ANALYZE addresses;

Functional Dependencies

The dependencies kind captures functional dependencies: how strongly the value of one column implies the value of another. PostgreSQL stores a degree between 0 and 1 for each direction.

For our table, city → state has a degree near 1.0 (knowing the city fully determines the state), while state → city is much lower (a state has many cities).

When the planner evaluates WHERE city = ? AND state = ? and finds a strong city → state dependency, it stops multiplying and instead keeps essentially the selectivity of the determining column.

CREATE STATISTICS stat_addr_deps (dependencies)
    ON city, state
    FROM addresses;

ANALYZE addresses;

Inspecting the Stored Dependencies

After ANALYZE, the computed values live in the catalog view pg_stats_ext (raw form) and pg_stats_ext_exprs for expression stats. The dependencies column shows each directional degree.

A degree at or near 1.000000 for "1 => 2" (column 1 implies column 2) confirms a near-perfect functional dependency — exactly the case where the independence assumption was hurting you.

SELECT statistics_name,
       attnames,
       dependencies
FROM pg_stats_ext
WHERE statistics_name = 'stat_addr_deps';
-- dependencies: {"1 => 2": 1.000000, "2 => 1": 0.140000}

ndistinct for GROUP BY and Joins

The ndistinct kind records the number of distinct combinations across the listed columns. Without it, the planner estimates distinct combos as the product of per-column distinct counts, which overshoots badly for correlated columns.

This matters most for GROUP BY a, b, c (estimating the number of groups) and for grouped aggregates feeding a hash aggregate. A wrong group estimate leads to under-sized hash tables and disk spills, or to a wrongly chosen sort-based aggregate.

CREATE STATISTICS stat_sales_ndist (ndistinct)
    ON region, country, city
    FROM sales;

ANALYZE sales;

EXPLAIN
SELECT region, country, city, count(*)
FROM sales
GROUP BY region, country, city;

MCV for Skewed Combinations

Functional dependencies assume a uniform, table-wide relationship. But sometimes the correlation is value-specific — certain combinations are extremely common while others never occur. That is where the mcv (most-common-values) kind shines.

An MCV list over a column group stores the actual frequent combinations and their frequencies, so the planner can estimate predicates like WHERE category = 'A' AND status = 'shipped' using the real observed frequency of that pair rather than a derived approximation.

MCV is the most powerful but also the most storage-intensive kind; reach for it when dependencies alone do not fix the estimate.

CREATE STATISTICS stat_orders_mcv (mcv)
    ON category, status
    FROM orders;

ANALYZE orders;

Combining Kinds in One Object

You can request multiple kinds in a single statistics object. If you omit the kind list entirely, PostgreSQL builds all applicable kinds for that column set.

A common, practical recipe is to list ndistinct, dependencies, mcv together for a set of columns that appear together in both WHERE and GROUP BY clauses. One ANALYZE then populates everything.

Note that mcv and dependencies support up to a limited number of columns, and you should keep statistics objects focused on columns that are actually queried together — not every column pair in the table.

CREATE STATISTICS stat_addr_all (ndistinct, dependencies, mcv)
    ON city, state, zip
    FROM addresses;

ANALYZE addresses;

Statistics on Expressions

PostgreSQL 14+ extends CREATE STATISTICS to expressions, not just bare columns. If your queries filter on date_trunc('month', created_at) or lower(email), the planner normally has no statistics for that computed value and falls back to a generic guess.

A single-expression statistics object gives the planner per-expression stats; a multi-column object mixing expressions and columns captures correlation between a computed value and a stored column.

CREATE STATISTICS stat_login_expr
    ON lower(email), date_trunc('day', created_at)
    FROM logins;

ANALYZE logins;

Workflow, Maintenance, and Cleanup

Extended statistics are not automatic — you create them deliberately based on observed misestimates. A reliable workflow:

Find a query whose EXPLAIN ANALYZE shows a large estimate-vs-actual gap on a multi-column predicate.
Create a statistics object over exactly those correlated columns.
Run ANALYZE on the table (or wait for autovacuum's analyze) to populate it.
Re-run EXPLAIN ANALYZE and confirm the estimate now tracks reality.

Statistics objects are refreshed by every ANALYZE, so they stay current automatically once created. Drop ones you no longer need with DROP STATISTICS to avoid paying their ANALYZE cost.

DROP STATISTICS IF EXISTS stat_addr_deps;

Quick Check: Choosing the Right Kind

You have a query SELECT count(*) FROM addresses WHERE city = $1 AND state = $2. EXPLAIN ANALYZE shows the planner estimates 15 rows but 9,000 actually match, because city fully determines state. Which extended statistics kind most directly fixes this estimate?

Recap

The planner assumes columns are independent and multiplies per-column selectivities — which underestimates rows when columns are correlated, leading to bad plans.

CREATE STATISTICS teaches the planner about multi-column relationships.
dependencies fixes equality-filter underestimates from functional dependencies (e.g. city → state).
ndistinct fixes distinct-combination estimates for GROUP BY and grouped aggregates.
mcv captures value-specific skewed combinations for the most accurate per-pair selectivity.
Create on expressions (PG14+), refresh via ANALYZE, verify with EXPLAIN ANALYZE, and inspect results in pg_stats_ext.

Target only columns truly queried together, then confirm the estimate-vs-actual gap closes.

よくある質問

「相関するカラムの多変量統計」レッスンは無料ですか？

はい。「相関するカラムの多変量統計」の完全なテキストはこのウェブで無料で読めます。インタラクティブに演習し（組み込みコードエディタと24時間対応のAIチューター）、PostgreSQL Performance & Query Optimizationコースの残りをアンロックするには、CoddyKit PROにアップグレードしてください。 PostgreSQL Performance & Query Optimizationコースには全4レッスンが含まれています。

「相関するカラムの多変量統計」で何を学びますか？

プランナーが独立していると仮定する依存関係を捉えるCREATE STATISTICSオブジェクトを作成します。ブラウザで直接実行するハンズオンコードでPostgreSQL Performance & Query Optimizationを演習し、24時間対応のAIチューターがレッスンを進める中での質問に答えます。

PostgreSQL Performance & Query Optimizationを始めるのに経験は必要ですか？

事前経験は必要ありません。CoddyKitのPostgreSQL Performance & Query Optimizationは初級者から上級者向けに構成されているため、ここから始めるか最初から始めて、自分のペースで進むことができます。これはレッスン2/4です。

「相関するカラムの多変量統計」レッスンにはどのくらい時間がかかりますか？

ほとんどのCoddyKitレッスンは約5～10分かかります。各レッスンはコンパクトでインタラクティブなので、着実に進歩し、ウェブとアプリ全体で正確に前回の場所から再開できます。

このPostgreSQL Performance & Query Optimizationレッスンでコードを書いて実行できますか？

はい。すべてのPostgreSQL Performance & Query Optimizationレッスンに組み込みコードエディタが含まれているため、ブラウザでリアルコードを書いて実行し、即座のAIフィードバックを取得できます。ローカル設定は不要です。

このコースのすべてのレッスン

← PostgreSQL Performance & Query Optimizationに戻る