Chi-Squared Test for Independence

The chi-squared test for independence tests whether two categorical variables are statistically independent or whether there is an association between them. For example: 'Is customer churn independent of subscription tier?' or 'Is product preference independent of age group?' Unlike t-tests that compare numeric means, chi-squared tests compare observed frequencies in a contingency table to the frequencies we would expect if the variables were independent.

A contingency table (also called a cross-tabulation) shows the count of observations for each combination of two categorical variables. pd.crosstab(df['var1'], df['var2']) builds this table directly from a DataFrame. Each cell contains the count of observations where row category and column category co-occur. This is the input to scipy.stats.chi2_contingency().

scipy.stats.chi2_contingency(observed) takes the observed contingency table (as a NumPy array or Pandas DataFrame) and returns four values: the chi-squared statistic, the p-value, the degrees of freedom, and the expected frequency table. The null hypothesis is H₀: the two variables are independent. If p ≤ 0.05, we reject independence and conclude there is a statistically significant association.

The expected frequencies represent what the cell counts would look like if the two variables were perfectly independent. For each cell, expected = (row_total × column_total) / grand_total. The chi-squared statistic measures the sum of squared differences between observed and expected counts, normalised by the expected counts. Large chi-squared means observed counts deviate strongly from independence; small chi-squared means the data is consistent with independence.

For a contingency table with r rows and c columns, the degrees of freedom is df = (r-1) × (c-1). A 2×2 table has df=1; a 3×4 table has df=6. The critical value of chi-squared increases with degrees of freedom: for df=1 at α=0.05, critical value ≈ 3.84; for df=6 at α=0.05, critical value ≈ 12.6. The chi2_contingency function handles this automatically, but knowing the formula helps you understand why chi-squared from larger tables requires bigger statistics to be significant.

The chi-squared test is only reliable when all expected frequencies are at least 5 (some sources say at least 1 with no more than 20% below 5). Small expected counts make the chi-squared approximation inaccurate. When this assumption is violated, use Fisher's exact test (scipy.stats.fisher_exact()) for 2×2 tables, or collapse rare categories to increase expected counts. Always inspect the expected frequency matrix returned by chi2_contingency.

scipy.stats.fisher_exact(table) computes the exact probability of observing the given 2×2 table (or one more extreme) under the null hypothesis of independence, without any approximation. It is always valid regardless of sample size or cell counts — the p-value is exact, not approximate. The cost is computational: it enumerates all possible tables, making it slow for large totals. For 2×2 tables with any cell count below 5, always prefer Fisher's exact test over chi-squared.

Like Cohen's d for t-tests, the chi-squared statistic alone does not measure the strength of association — it is inflated by sample size. Cramér's V normalises chi-squared to a 0–1 scale: 0 means no association, 1 means perfect association. V = sqrt(chi2 / (n × min(r-1, c-1))). Guidelines: < 0.1 is weak, 0.1–0.3 is moderate, > 0.3 is strong. Always report Cramér's V alongside the p-value for a complete picture.

A different application of chi-squared is the goodness-of-fit test: it tests whether observed frequencies match a hypothesised distribution. For example, 'Are die rolls uniformly distributed?' or 'Does our website traffic follow the expected day-of-week pattern?' Use scipy.stats.chisquare(f_obs, f_exp) where f_exp is the expected frequencies. The null hypothesis is that the data follows the specified distribution.

When reporting chi-squared results, it helps to show proportions rather than raw counts so readers can see the relative association. pd.crosstab(var1, var2, normalize='index') shows the row proportion (what fraction of each row category falls in each column). normalize='columns' shows column proportions. Comparing these proportions visually before running the test helps you anticipate the direction of the association and interpret the result in context.

The chi-squared test is the standard test for A/B testing conversion rates. Create a 2×2 contingency table with rows = (control, variant) and columns = (converted, not converted). The chi-squared test tells you whether the conversion rate differs significantly between groups. This is equivalent to a two-proportion z-test for large samples. For small samples (any expected count < 5), use Fisher's exact test instead.

Test your understanding of Data Analysis concepts from this lesson.

In this lesson you learned: pd.crosstab() + chi2_contingency() test whether two categorical variables are independent based on observed vs. expected frequencies, Fisher's exact test is the safe alternative when expected counts are below 5, and Cramér's V measures association strength independently of sample size. Next up we compare means across three or more groups with ANOVA and post-hoc tests.