Univariate Analysis

Univariate analysis examines one column at a time in isolation, ignoring relationships between columns. The goal is to understand each variable's distribution, detect outliers, identify skewness, and spot data quality issues before any modelling begins. Univariate analysis differs for numeric and categorical columns — numerics need distribution plots and summary statistics, while categoricals need frequency counts and proportions.

Plot a histogram for each numeric column to see the shape of its distribution. Look for symmetry vs. skewness (tail on one side), modality (one peak vs. several), and obvious anomalies (values at extreme ends that seem implausible). In Pandas, df.hist() creates a quick multi-column histogram grid without writing a loop, but Seaborn's histplot gives finer control for individual columns.

For each numeric column, compute and compare mean vs. median. When mean > median significantly, the distribution is right-skewed (long right tail, common for income and price data). When mean < median, it is left-skewed. Also check standard deviation relative to the mean: a std larger than the mean for a non-negative variable is a sign of high variability or outliers. Compute skewness directly with df.skew().

Box plots are the quickest visual tool for spotting outliers in numeric columns. Any point beyond the whiskers (1.5×IQR from Q1 or Q3) is plotted individually and flagged as a potential outlier. A column with many outlier points warrants investigation: are they data entry errors, legitimate extreme values, or evidence of a non-normal distribution? Box plots also reveal right or left skew from the asymmetric position of the median inside the box.

The IQR (Interquartile Range) method defines outliers as values below Q1 - 1.5×IQR or above Q3 + 1.5×IQR. You can compute this programmatically to count and inspect outliers in every numeric column without plotting. This is useful in automated pipelines where you need to log anomaly counts rather than generate charts. Deciding what to do with outliers — remove, cap, or flag — should be a deliberate choice informed by domain knowledge.

For each categorical column, call value_counts() to see how observations are distributed across categories. Always check: does any single category dominate (>80%)? This causes class imbalance in classification tasks. Are there rare categories with only 1-2 observations (typos or data entry errors)? Does the distribution match business expectations (e.g. a 'country' column that shows most orders from an unexpected country)?

Visualise categorical value counts as bar charts using sns.countplot or by calling value_counts().plot(kind='bar'). Sort bars from most to least frequent so the viewer can immediately see dominant categories. For binary variables (yes/no, male/female), a pie chart is acceptable — but for more than 3 categories, bar charts are always clearer. Rotate tick labels 45 degrees when category names are long.

Highly right-skewed numeric columns (skewness > 1.5) often benefit from a log transform to make them more symmetric. This is important for many statistical tests and some ML algorithms that assume normality. Apply np.log1p() (log(x+1), safe for values near zero) and re-check skewness. Compare histograms before and after to confirm the distribution became more bell-shaped.

A column with zero variance (all values are identical) provides no information to any model and should be dropped. A column with near-zero variance (99% of rows have the same value) is similarly useless. Compute variance with df.var() and filter. Also check for columns where nunique() == len(df) — these are likely ID columns that should not be used as features but may be useful as row identifiers.

Rather than repeating the same analysis manually for each column, write a loop that iterates over all numeric columns and prints key statistics in a structured format. This makes it easy to scan dozens of columns quickly and flag those that need attention. The output can be saved to a CSV file as part of a pipeline audit log that stakeholders can review before data is used in modelling.

After completing univariate analysis, document your findings systematically. For each column note: the distribution shape (skewed, bimodal, normal-ish), missing value percentage and planned imputation strategy, outlier count and decision (cap, remove, flag), and any suspicious values requiring domain clarification. This documentation becomes the data quality log that guides the cleaning steps and protects decisions from being forgotten or contested later.

Test your understanding of univariate analysis from this lesson.

In this lesson you learned: histograms reveal distribution shape for numeric columns, box plots and IQR fencing identify outliers, and value_counts reveals category frequency for categorical columns. Automating these checks in a loop creates a reusable quality report. Next up we explore bivariate and correlation analysis — understanding relationships between pairs of columns.