What Makes a Model Large
Scale, pre-training, and emergent skill.
What Makes a Model Large is a free NLP Academy lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the NLP Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Big by Design
A large language model is a transformer trained on huge amounts of text, with billions of parameters that store patterns of language. 🤖
What Parameters Are
The model's parameters are the learned numbers that shape its predictions. More parameters give it more room to capture subtle patterns.
Trained on the Internet
During pre-training, the model reads vast text and learns by predicting the next word over and over, billions of times.
The Next-Word Game
That simple goal is powerful: to guess the next word well, the model must absorb grammar, facts, and reasoning hidden in the text.
prompt = "The capital of France is"
# the model predicts: ParisScale Means Three Things
Scaling up means more parameters, more training data, and more compute. Growing all three together is what unlocks new ability.
Emergent Skills
Past a certain size, models show emergent abilities like translation or reasoning that smaller versions simply could not do.
One Model, Many Tasks
A single large model is a general tool: the same weights can summarize, answer questions, write code, and translate.
Foundation Models
These broad pre-trained models are called foundation models because you build specific applications on top of them.
Polishing With Alignment
After pre-training, models are fine-tuned on instructions and human feedback so they follow requests and stay helpful.
The Context Window
An LLM can only read so much at once. That limit is the context window, measured in tokens, and it caps how much you can send.
Why It Still Errs
Even huge models can confidently invent facts. This is called hallucination, and it is why you verify important outputs.
Quick Check
What core task is a large language model trained on during pre-training?
Recap
Large models scale parameters, data, and compute. That scale brings emergent skills, but also a context limit and the risk of hallucination. ✅
Frequently asked questions
Is the “What Makes a Model Large” lesson free?
Yes — the full text of “What Makes a Model Large” is free to read here on the web, and the NLP Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the NLP Academy course, upgrade to CoddyKit PRO.
What will I learn in “What Makes a Model Large”?
Scale, pre-training, and emergent skill. You practise NLP Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start NLP Academy?
No prior experience is required. NLP Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “What Makes a Model Large” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this NLP Academy lesson?
Yes. Every NLP Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- What Makes a Model Large
- Calling an LLM From Python
- Zero-Shot and Few-Shot Prompting
- Structured Output and Guardrails