Writing DataFrames to Database Tables
Persist a cleaned DataFrame to a new or existing table with DataFrame.to_sql(), controlling if_exists and chunksize.
Why Write DataFrames to Databases?
After cleaning and transforming data in Pandas, you often need to persist the results back to a database: to make them available to other applications, dashboards, or team members; to store incremental analysis results; or to build a data mart from a raw data lake. DataFrame.to_sql() is the standard Pandas method for writing data to any SQLAlchemy-supported database in a single call.
Basic to_sql() Usage
df.to_sql('table_name', con=engine, if_exists='replace', index=False) writes the DataFrame to a database table. The if_exists parameter controls what happens if the table already exists: 'fail' raises an error, 'replace' drops and recreates the table, and 'append' adds new rows without touching existing ones. Always set index=False unless you explicitly want to store the DataFrame index as a column in the database.
import pandas as pd
import sqlalchemy as sa
engine = sa.create_engine('sqlite:///results.db')
df = pd.DataFrame({
'date': pd.date_range('2024-01-01', periods=5),
'revenue': [1200.0, 980.5, 1450.0, 760.3, 1100.0],
'region': ['North', 'South', 'East', 'West', 'North']
})
df.to_sql('daily_revenue', con=engine,
if_exists='replace', index=False)
print('Table written successfully')All lessons in this course
- Connecting to a Database with SQLAlchemy
- Running SQL Queries from Pandas
- Writing DataFrames to Database Tables
- Pandas vs. SQL: Choosing the Right Tool