Tokens, Spans, and Doc Objects
Navigate spaCy's data structures.
Tokens, Spans, and Doc Objects is a free NLP Academy lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the NLP Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Three Core Objects
spaCy gives you three building blocks: the Doc, the Token, and the Span. Learn these and the whole library opens up.
The Doc Container
A Doc is the full processed document. It holds the original text plus every token and all the analysis spaCy produced.
doc = nlp("Maria leads the data team.")Index Like a List
You can grab one item by position because a Doc is indexable. Indexing returns a single Token, just like a Python list.
first = doc[0]What a Token Holds
A Token is one unit of text plus rich metadata: its part of speech, its lemma, and whether it is punctuation.
print(doc[0].text, doc[0].pos_)Useful Token Flags
Tokens carry handy flags like is_stop, is_alpha, and is_punct. They let you filter words without writing your own checks.
for t in doc:
print(t.text, t.is_stop)Slice to Get a Span
Slice a Doc and you get a Span, a contiguous run of tokens. It is perfect for phrases like a person or a place.
phrase = doc[0:2]Spans Keep Context
A Span is not a copy. It still points back into the Doc, so it knows its position and shares all the original analysis.
Entities Are Spans
Named entities show up as Spans in doc.ents. Each one bundles the words plus a label like PERSON or ORG.
for ent in doc.ents:
print(ent.text, ent.label_)Noun Chunks
spaCy also offers noun_chunks, base noun phrases served as Spans. They are a quick way to pull out the things a sentence talks about.
list(doc.noun_chunks)Sentences as Spans
Need sentences? Iterate doc.sents. Each sentence comes back as a Span, so segmentation is already handled for you.
for sent in doc.sents:
print(sent.text)Original Text Stays
Through all of this the original string is preserved in doc.text. Tokens and spans are views, never destructive edits.
Quick Check
What do you get when you slice a Doc with doc[0:3]?
Recap
The Doc holds everything, indexing gives one Token, and slicing gives a Span. Entities, noun chunks, and sentences all arrive as Spans. 🧩
Frequently asked questions
Is the “Tokens, Spans, and Doc Objects” lesson free?
Yes — the full text of “Tokens, Spans, and Doc Objects” is free to read here on the web, and the NLP Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the NLP Academy course, upgrade to CoddyKit PRO.
What will I learn in “Tokens, Spans, and Doc Objects”?
Navigate spaCy's data structures. You practise NLP Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start NLP Academy?
No prior experience is required. NLP Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Tokens, Spans, and Doc Objects” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this NLP Academy lesson?
Yes. Every NLP Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Why spaCy for Real Projects
- Loading a Model and Processing a Doc
- Tokens, Spans, and Doc Objects
- Customizing the Pipeline