Многомерная статистика для коррелированных столбцов

When PostgreSQL estimates how many rows a query will return, it leans on per-column statistics stored in pg_statistic. To combine predicates on multiple columns, the planner makes a crucial simplifying assumption: the columns are statistically independent.

Under independence, the selectivity of WHERE a = 1 AND b = 2 is computed as sel(a=1) * sel(b=2). That multiplication is fast and correct — but only when the columns truly are unrelated.

In real schemas, columns are frequently correlated: a city implies a postal code, a product implies a category, an order date implies a fiscal quarter. When the planner multiplies selectivities for correlated columns, its estimate collapses far below reality.

A row-count estimate that is off by orders of magnitude steers the planner toward the wrong plan:

• Underestimate → planner picks a nested loop expecting 3 rows, but 300,000 arrive → the loop executes its inner side hundreds of thousands of times.
• Underestimate → planner chooses an index scan + heap fetches instead of a single sequential scan that would have been cheaper.
• Bad join order → a large intermediate result is materialized early, blowing up memory and spilling to disk.

The symptom you see in EXPLAIN ANALYZE is a wide gap between rows= (estimated) and actual rows=. That gap is your signal that correlation may be the culprit.

Consider an addresses table where city functionally determines state — every row with city = 'New York' also has state = 'NY'. The two predicates select the same rows, so the combined selectivity equals sel(city) alone.

But the planner multiplies: sel(city) * sel(state), producing an estimate that can be 10× or 100× too small. In the EXPLAIN ANALYZE output below, watch the gap between the estimated and actual row counts on the scan node.

PostgreSQL 10+ lets you teach the planner about column relationships with extended statistics objects, created via CREATE STATISTICS.

An extended statistics object names a set of columns (or expressions) on one table and one or more kinds of statistics to gather over them. Once created and analyzed, the planner consults these multivariate statistics instead of blindly multiplying per-column selectivities.

The three kinds are:

• ndistinct — number of distinct combinations of the listed columns.
• dependencies — functional-dependency degrees between columns.
• mcv — most-common-value lists over the column group.

The dependencies kind captures functional dependencies: how strongly the value of one column implies the value of another. PostgreSQL stores a degree between 0 and 1 for each direction.

For our table, city → state has a degree near 1.0 (knowing the city fully determines the state), while state → city is much lower (a state has many cities).

When the planner evaluates WHERE city = ? AND state = ? and finds a strong city → state dependency, it stops multiplying and instead keeps essentially the selectivity of the determining column.

After ANALYZE, the computed values live in the catalog view pg_stats_ext (raw form) and pg_stats_ext_exprs for expression stats. The dependencies column shows each directional degree.

A degree at or near 1.000000 for "1 => 2" (column 1 implies column 2) confirms a near-perfect functional dependency — exactly the case where the independence assumption was hurting you.

The ndistinct kind records the number of distinct combinations across the listed columns. Without it, the planner estimates distinct combos as the product of per-column distinct counts, which overshoots badly for correlated columns.

This matters most for GROUP BY a, b, c (estimating the number of groups) and for grouped aggregates feeding a hash aggregate. A wrong group estimate leads to under-sized hash tables and disk spills, or to a wrongly chosen sort-based aggregate.

Functional dependencies assume a uniform, table-wide relationship. But sometimes the correlation is value-specific — certain combinations are extremely common while others never occur. That is where the mcv (most-common-values) kind shines.

An MCV list over a column group stores the actual frequent combinations and their frequencies, so the planner can estimate predicates like WHERE category = 'A' AND status = 'shipped' using the real observed frequency of that pair rather than a derived approximation.

MCV is the most powerful but also the most storage-intensive kind; reach for it when dependencies alone do not fix the estimate.

You can request multiple kinds in a single statistics object. If you omit the kind list entirely, PostgreSQL builds all applicable kinds for that column set.

A common, practical recipe is to list ndistinct, dependencies, mcv together for a set of columns that appear together in both WHERE and GROUP BY clauses. One ANALYZE then populates everything.

Note that mcv and dependencies support up to a limited number of columns, and you should keep statistics objects focused on columns that are actually queried together — not every column pair in the table.

PostgreSQL 14+ extends CREATE STATISTICS to expressions, not just bare columns. If your queries filter on date_trunc('month', created_at) or lower(email), the planner normally has no statistics for that computed value and falls back to a generic guess.

A single-expression statistics object gives the planner per-expression stats; a multi-column object mixing expressions and columns captures correlation between a computed value and a stored column.

Extended statistics are not automatic — you create them deliberately based on observed misestimates. A reliable workflow:

• Find a query whose EXPLAIN ANALYZE shows a large estimate-vs-actual gap on a multi-column predicate.
• Create a statistics object over exactly those correlated columns.
• Run ANALYZE on the table (or wait for autovacuum's analyze) to populate it.
• Re-run EXPLAIN ANALYZE and confirm the estimate now tracks reality.

Statistics objects are refreshed by every ANALYZE, so they stay current automatically once created. Drop ones you no longer need with DROP STATISTICS to avoid paying their ANALYZE cost.

You have a query SELECT count(*) FROM addresses WHERE city = $1 AND state = $2. EXPLAIN ANALYZE shows the planner estimates 15 rows but 9,000 actually match, because city fully determines state. Which extended statistics kind most directly fixes this estimate?

The planner assumes columns are independent and multiplies per-column selectivities — which underestimates rows when columns are correlated, leading to bad plans.

• CREATE STATISTICS teaches the planner about multi-column relationships.
• dependencies fixes equality-filter underestimates from functional dependencies (e.g. city → state).
• ndistinct fixes distinct-combination estimates for GROUP BY and grouped aggregates.
• mcv captures value-specific skewed combinations for the most accurate per-pair selectivity.
• Create on expressions (PG14+), refresh via ANALYZE, verify with EXPLAIN ANALYZE, and inspect results in pg_stats_ext.

Target only columns truly queried together, then confirm the estimate-vs-actual gap closes.