Why Git Alone Cannot Version Data
The limits of Git for gigabyte-sized datasets.
Why Git Alone Cannot Version Data is a free MLOps Academy lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the MLOps Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Code Is Tiny, Data Is Huge
Git was built for source code: small text files that change line by line. Your datasets are often gigabytes of binary data, a totally different beast.
Git Stores Every Version
Git keeps a full snapshot of each file version forever. Commit a 2 GB file ten times and your repo can balloon toward 20 GB of history. 😬
Binary Diffs Are Useless
Git shines on text because it can show line diffs. With binary files like images or parquet, it cannot diff meaningfully and just stores the whole new copy.
Cloning Becomes Painful
Because history lives in the repo, a teammate cloning it must download every past dataset version too. A simple clone can take ages and fill the disk.
Hosting Limits Bite
GitHub caps individual files at 100 MB and warns past 50 MB. Most real datasets blow right past that and the push is rejected.
Git LFS Helps, But Only So Far
Git LFS swaps big files for pointers, easing the size problem. But it lacks ML features like pipelines, remotes per project, and reproducible data stages.
You Still Need Reproducibility
An experiment is only trustworthy if you can recreate the exact data it used. Git alone cannot reliably link a commit to a specific dataset snapshot.
The Big Idea: Store Pointers
The fix is to keep a tiny pointer file in Git that names the data, while the heavy bytes live in cheap object storage. Git tracks the pointer, not the payload.
# Git tracks this small text pointer, not the 2GB file
outs:
- md5: a1b2c3d4e5f6...
size: 2147483648
path: data/train.csvEnter DVC
DVC (Data Version Control) does exactly this. It pairs Git for code with a separate cache and remote for data, giving you Git-like commands for datasets.
Data and Code Stay in Sync
With DVC, checking out an old Git commit also restores the matching data version. Your code and dataset move together as one consistent unit.
Familiar Workflow
DVC borrows Git's mental model: you add, commit, push, and checkout data. If you know Git basics, you already half-understand DVC. 🎉
Quick Check
Why does committing large datasets straight into Git cause trouble?
Recap
Git is great for code but chokes on big binary data, bloating history and hitting size limits. DVC stores tiny pointers in Git and keeps the heavy data elsewhere, in sync.
Frequently asked questions
Is the “Why Git Alone Cannot Version Data” lesson free?
Yes — the full text of “Why Git Alone Cannot Version Data” is free to read here on the web, and the MLOps Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the MLOps Academy course, upgrade to CoddyKit PRO.
What will I learn in “Why Git Alone Cannot Version Data”?
The limits of Git for gigabyte-sized datasets. You practise MLOps Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start MLOps Academy?
No prior experience is required. MLOps Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Why Git Alone Cannot Version Data” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this MLOps Academy lesson?
Yes. Every MLOps Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Why Git Alone Cannot Version Data
- Initialize DVC and Track a Dataset
- Push Data to Remote Storage
- Roll Back to an Earlier Dataset