シフトとラグ特徴量
shift()でラグ列とリード列を作成し、diff()で期間ごとの変化を計算して、変化率を求めます。
「シフトとラグ特徴量」はCoddyKit上の無料Pandas & NumPy Academyレッスンです。 これはレッスン3/4です。 下記で完全なレッスンを無料で読むことができます。その後、ブラウザ内の組み込みコードエディタと24時間対応のAIチューターでハンズオン演習できます。 これはPandas & NumPy Academy学習パスの一部であり、ウェブとCoddyKitアプリ全体で進捗が同期されます。 Pandas & NumPy Academyコースには全4レッスンが含まれています。
このレッスンの一部はまだ翻訳されておらず、英語で表示されています。
Why Lag Features Matter
In time series analysis, the value at a previous time step (a lag) is often one of the best predictors of the current value. For example, yesterday's sales are informative about today's sales. Creating lag columns lets you use historical values as features in machine learning models or in statistical analysis of autocorrelation. Pandas provides shift() to create lag features in a single vectorised call.
shift(): Moving Values Forward or Backward
Series.shift(n) moves all values down by n positions (positive n creates a lag), filling the first n rows with NaN. Passing a negative n moves values up (creates a lead, shifting future values back to the current row). The index remains unchanged — only the values are displaced.
import pandas as pd
import numpy as np
df = pd.DataFrame(
{'sales': [100, 120, 115, 130, 140, 125]},
index=pd.date_range('2024-01-01', periods=6, freq='D')
)
# Lag-1: yesterday's sales
df['sales_lag1'] = df['sales'].shift(1)
# Lead-1: tomorrow's sales (shifted up)
df['sales_lead1'] = df['sales'].shift(-1)
print(df)Creating Multiple Lag Columns
You typically create several lag features at once for machine learning — lag-1, lag-7 (same day last week), and lag-30 (same day last month). The most readable way is to create them in a loop and assign each to a new column. The resulting DataFrame has the original series plus all its lagged versions ready for modelling.
for lag in [1, 2, 3, 7]:
df[f'sales_lag{lag}'] = df['sales'].shift(lag)
print(df.columns.tolist())
# ['sales', 'sales_lag1', 'sales_lag2', 'sales_lag3', 'sales_lag7']
# Drop rows with NaN from the largest lag period
df_model = df.dropna()
print('Ready for modelling, shape:', df_model.shape)shift() with freq for Time-Based Shifting
Instead of shifting by an integer number of rows, you can shift by a time offset using the freq parameter. df.shift(1, freq='ME') shifts the index forward by one month-end, keeping the values in place but changing the index. This is useful for aligning two time series that are offset by a fixed time period without rearranging the values.
monthly = pd.Series(
[100, 120, 115],
index=pd.date_range('2024-01-31', periods=3, freq='ME')
)
# Shift the index forward by 1 month (values stay, index moves)
shifted_index = monthly.shift(1, freq='ME')
print('Original index:', monthly.index.tolist())
print('Shifted index: ', shifted_index.index.tolist())
# Original: [2024-01-31, 2024-02-29, 2024-03-31]
# Shifted: [2024-02-29, 2024-03-31, 2024-04-30]diff(): Period-over-Period Change
Series.diff(n) computes the difference between a value and the value n positions before it: x[t] - x[t-n]. This is commonly used to compute day-over-day changes, week-over-week differences, or year-over-year deltas. The first n rows are NaN since there are no previous values to subtract from.
df['sales_diff1'] = df['sales'].diff(1) # day-over-day change
df['sales_diff7'] = df['sales'].diff(7) # week-over-week change
print(df[['sales', 'sales_diff1']].head())
# sales sales_diff1
# 2024-01-01 100 NaN
# 2024-01-02 120 20.0 <- +20 from yesterday
# 2024-01-03 115 -5.0 <- -5 from yesterdaypct_change(): Percentage Change
Series.pct_change(n) computes the relative percentage change: (x[t] - x[t-n]) / x[t-n]. This is essential for financial analysis (daily returns), growth rate computation, and any situation where the absolute change is less meaningful than the relative change. Multiply by 100 to get percentage points.
df['pct_chg'] = df['sales'].pct_change().round(3)
df['pct_chg_7d'] = df['sales'].pct_change(7).round(3)
print(df[['sales', 'pct_chg']].head())
# sales pct_chg
# 2024-01-01 100 NaN
# 2024-01-02 120 0.200 <- 20% increase
# 2024-01-03 115 -0.042 <- 4.2% decreaseCombining shift() with Arithmetic
You can combine shift() with arithmetic to compute derived time features. For example: the ratio of today's value to last week's value, the cumulative sum since a fixed start, or the difference from the year-ago value. All of these follow the same pattern: shift the original series and perform element-wise arithmetic with the current values.
# Week-over-week growth index (today / last week)
df['wow_ratio'] = (df['sales'] / df['sales'].shift(7)).round(3)
# Deviation from 3-day lag
df['dev_3d'] = df['sales'] - df['sales'].shift(3)
# Is today higher than yesterday? (boolean feature)
df['higher_than_yesterday'] = df['sales'] > df['sales'].shift(1)
print(df[['sales', 'wow_ratio', 'higher_than_yesterday']].head())cumsum() and cumprod() for Cumulative Features
Series.cumsum() computes the running total from the first row up to each row, while Series.cumprod() computes the cumulative product. These are useful for cumulative revenue, cumulative returns (compound growth), and year-to-date totals. They require no arguments and work on any numeric Series or DataFrame column.
# Cumulative sales (year-to-date total)
df['ytd_sales'] = df['sales'].cumsum()
# Cumulative product: compound growth factor
returns = pd.Series([0.02, -0.01, 0.03, 0.01, -0.005])
df_ret = pd.DataFrame({'daily_return': returns})
df_ret['compound'] = (1 + df_ret['daily_return']).cumprod() - 1
print(df_ret)Lag Features for Machine Learning
When preparing time series data for machine learning, a standard feature engineering step is to create a matrix of lag features. Each column represents the target variable at a different past time step. After creating lags, drop rows with NaN (which appear at the start due to missing history) and split into train and test sets without shuffling, respecting the temporal order.
def make_lag_matrix(series, n_lags):
df_lags = pd.DataFrame({'y': series})
for lag in range(1, n_lags + 1):
df_lags[f'lag_{lag}'] = series.shift(lag)
return df_lags.dropna()
lag_df = make_lag_matrix(df['sales'], n_lags=3)
print(lag_df)
# y lag_1 lag_2 lag_3
# ... ... ... ...Shifting Within Groups
When your data contains multiple entities (e.g., multiple products or regions in one DataFrame), you must compute shifts within each entity's group separately. Use groupby().shift() to ensure that the lag for product A's first row is NaN and not the last value of product B. Mixing groups without this step is a common and subtle data leakage bug.
df_multi = pd.DataFrame({
'product': ['A', 'A', 'A', 'B', 'B', 'B'],
'sales': [100, 120, 115, 200, 210, 195]
})
# WRONG: lag crosses product boundary
df_multi['lag_wrong'] = df_multi['sales'].shift(1)
# CORRECT: lag within each product
df_multi['lag_correct'] = df_multi.groupby('product')['sales'].shift(1)
print(df_multi)Interpreting pct_change Signs
A positive pct_change() value means the current value is higher than the previous one; a negative value means it is lower. Watch out for division-by-zero when the previous value is zero — the result is NaN or inf depending on the sign of the current value. Use replace([float('inf'), float('-inf')], float('nan')) to clean infinite values after pct_change().
import numpy as np
s = pd.Series([0, 100, 50, 200])
chg = s.pct_change()
print(chg)
# 0 NaN <- no prior value
# 1 inf <- 0 -> 100 (division by zero)
# 2 -0.5 <- 100 -> 50 (-50%)
# 3 3.0 <- 50 -> 200 (+300%)
# Clean infinite values
chg_clean = chg.replace([float('inf'), float('-inf')], float('nan'))
print(chg_clean)Quick Check
Test your understanding of shifting and lag features from this lesson.
Lesson Recap
In this lesson you learned: shift(n) creates lag features by moving values down by n positions; diff(n) computes period-over-period absolute change; pct_change(n) computes relative percentage change; and when working with multiple groups you must use groupby().shift() to avoid data leakage. Next up we extract temporal features using the .dt accessor.
よくある質問
「シフトとラグ特徴量」レッスンは無料ですか？
はい。「シフトとラグ特徴量」の完全なテキストはこのウェブで無料で読めます。インタラクティブに演習し（組み込みコードエディタと24時間対応のAIチューター）、Pandas & NumPy Academyコースの残りをアンロックするには、CoddyKit PROにアップグレードしてください。 Pandas & NumPy Academyコースには全4レッスンが含まれています。
「シフトとラグ特徴量」で何を学びますか？
shift()でラグ列とリード列を作成し、diff()で期間ごとの変化を計算して、変化率を求めます。 ブラウザで直接実行するハンズオンコードでPandas & NumPy Academyを演習し、24時間対応のAIチューターがレッスンを進める中での質問に答えます。
Pandas & NumPy Academyを始めるのに経験は必要ですか？
事前経験は必要ありません。CoddyKitのPandas & NumPy Academyは初級者から上級者向けに構成されているため、ここから始めるか最初から始めて、自分のペースで進むことができます。 これはレッスン3/4です。
「シフトとラグ特徴量」レッスンにはどのくらい時間がかかりますか？
ほとんどのCoddyKitレッスンは約5～10分かかります。各レッスンはコンパクトでインタラクティブなので、着実に進歩し、ウェブとアプリ全体で正確に前回の場所から再開できます。
このPandas & NumPy Academyレッスンでコードを書いて実行できますか？
はい。すべてのPandas & NumPy Academyレッスンに組み込みコードエディタが含まれているため、ブラウザでリアルコードを書いて実行し、即座のAIフィードバックを取得できます。ローカル設定は不要です。