Basic DataFrame Inspection

Whenever you load a new dataset, a systematic inspection sequence saves hours of debugging. The Pandas toolkit for this includes: head() to see the first rows, tail() for the last rows, info() for column types and nulls, describe() for statistics, and shape for dimensions. Running these five checks takes less than a minute and reveals most data quality issues immediately.

df.head(n) returns the first n rows (default 5) as a DataFrame. df.tail(n) returns the last n rows. Together they help you verify that the data was parsed correctly — checking that column names are in the header row, that numeric columns contain numbers, and that date strings were not misinterpreted. They are non-destructive and return new DataFrames.

df.info() prints a concise summary: the index dtype, the number of non-null values per column, each column's dtype, and the total memory usage. Columns with fewer non-null values than the total row count have missing data. Object dtype often means a string column (or a mixed-type column) that may need cleaning before analysis.

df.describe() computes count, mean, std, min, 25th, 50th (median), 75th percentile, and max for all numeric columns. Pass include='all' to also summarise object (string) columns. Pass include='object' to only summarise categoricals. The output is itself a DataFrame, so you can save it or style it for a report.

df.shape is a tuple (nrows, ncols). df.ndim always returns 2 for DataFrames. df.size is the total number of cells. Use len(df) for the number of rows (same as df.shape[0]). These are the simplest sanity checks — confirm the expected row count after loading and after each filtering step.

df.dtypes returns a Series mapping each column name to its dtype. df.select_dtypes(include='number') returns a new DataFrame with only numeric columns — useful for computing correlations or applying numeric transformations without accidentally operating on string columns. Pass exclude='object' to drop text columns.

df.isnull().sum() gives the count of missing values per column — the quickest way to find which columns need cleaning. Divide by len(df) to get the proportion of missing values. A column with more than 50% missing often needs special treatment: either drop it or apply domain-specific imputation.

Calling df['col'].value_counts() lists the frequency of each unique value in that column, sorted by count. Add normalize=True for percentages. Use dropna=False to also count NaN as a category. This is the fastest way to check for category imbalance or unexpected values in a categorical column.

df.columns.tolist() gives a plain Python list of column names — useful for programmatic manipulation or for logging. df.index shows the row labels and their type. Check df.index.is_unique to confirm no duplicate row labels exist, which is a prerequisite for many merge operations and time series calculations.

df.sample(n) returns n randomly selected rows without replacement. df.sample(frac=0.1) returns 10% of the rows. Set random_state= for reproducibility. Random sampling is better than head() for spotting issues in the middle of a large dataset that might not appear in the first few rows.

df.memory_usage(deep=True) returns the memory used by each column in bytes. deep=True forces accurate measurement of object-dtype columns whose strings live outside the DataFrame buffer. Sum the values to get total memory. Use this before and after dtype downcasting to confirm you have reduced memory consumption.

Test your understanding of basic DataFrame inspection from this lesson.

In this lesson you learned: head() and tail() let you visually inspect the start and end of a DataFrame, info() reveals dtypes and missing value counts in a single call, and describe() provides statistical summaries for numeric columns. Next up we load data from the most common file formats — CSV, Excel, and JSON — into DataFrames.