LLM as Text Classifier
Using prompts for sentiment, intent, topic, and multi-label classification.
LLM as Text Classifier is a free AI Prompt Engineering lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Prompt Engineering learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
LLMs as Text Classifiers
Traditional text classification requires labeled training data, model fine-tuning, and deployment infrastructure. LLMs can classify text with just a prompt — no training data required.
LLM-based classifiers excel when:
- The categories require semantic understanding (not just keyword matching)
- You need to add new categories without retraining
- You have limited labeled examples
- The categories are nuanced (intent behind a message, not just its topic)
Sentiment Classification
Sentiment is one of the most common classification tasks. A well-crafted prompt outperforms simple keyword matching for nuanced cases:
import anthropic, json
client = anthropic.Anthropic(api_key='YOUR_API_KEY')
def classify_sentiment(text):
prompt = f'''
Classify the sentiment of the text below.
Return ONLY JSON: {{"sentiment": "positive|negative|neutral", "confidence": "high|medium|low"}}
Definitions:
- positive: Overall favorable opinion or emotion
- negative: Overall unfavorable opinion or dissatisfaction
- neutral: Factual, balanced, or no clear sentiment
Text: {text}
'''
r = client.messages.create(
model='claude-opus-4-5', max_tokens=50,
messages=[{'role': 'user', 'content': prompt}]
)
return json.loads(r.content[0].text)
print(classify_sentiment('The product works but setup was painful.'))
print(classify_sentiment('Delivery was incredibly fast and packaging was perfect!'))Intent Classification
Intent classification identifies what the user is trying to do — essential for chatbots, support systems, and search applications:
INTENT_CATEGORIES = [
'purchase_intent: User wants to buy or is ready to purchase',
'complaint: User is dissatisfied and reporting a problem',
'question: User is asking for information or help',
'cancellation: User wants to cancel a service or subscription',
'compliment: User is expressing satisfaction or praise',
'other: Does not fit any category above'
]
def classify_intent(message):
categories_str = '\n'.join(f'- {c}' for c in INTENT_CATEGORIES)
prompt = f'''
Classify the intent of this customer message.
Return JSON: {{"intent": str, "confidence": "high|medium|low"}}
Categories:
{categories_str}
Message: {message}
'''
r = client.messages.create(model='claude-opus-4-5', max_tokens=80, messages=[{'role': 'user', 'content': prompt}])
return json.loads(r.content[0].text)
print(classify_intent('I love the app but I need to cancel my plan.'))
print(classify_intent('How do I export my data to CSV?'))Topic Classification
Topic classification assigns a thematic category to text. Useful for routing content, filtering feeds, and categorizing support tickets:
def classify_topic(article_text, topics):
topic_list = ', '.join(topics)
prompt = f'''
Classify the topic of this article. Choose EXACTLY ONE from the list.
Return JSON: {{"topic": str, "secondary_topic": str or null}}
Available topics: {topic_list}
Article (first 300 chars): {article_text[:300]}
'''
r = client.messages.create(model='claude-opus-4-5', max_tokens=80, messages=[{'role': 'user', 'content': prompt}])
return json.loads(r.content[0].text)
topics = ['technology', 'sports', 'politics', 'business', 'science', 'health', 'entertainment']
text = 'The FDA approved a new mRNA vaccine for seasonal influenza, marking a breakthrough in vaccine technology.'
result = classify_topic(text, topics)
print(result)Urgency Classification
Urgency classification helps prioritize support tickets, emails, and incidents. Defining urgency levels precisely is critical:
URGENCY_PROMPT = '''
Classify the urgency of this support ticket.
Return JSON: {{"urgency": str, "reason": str}}
Urgency levels:
- critical: Service is completely down, data loss occurring, or security breach
- high: Major functionality broken, many users affected, no workaround
- medium: Non-critical feature broken, workaround exists, single user affected
- low: Cosmetic issue, enhancement request, general question
Be conservative: only use critical if the ticket explicitly describes a system-wide outage or data loss.
Ticket: {ticket}
'''
def classify_urgency(ticket_text):
prompt = URGENCY_PROMPT.replace('{ticket}', ticket_text)
r = client.messages.create(model='claude-opus-4-5', max_tokens=100, messages=[{'role': 'user', 'content': prompt}])
return json.loads(r.content[0].text)
print(classify_urgency('The entire production database is down. All customers affected.'))
print(classify_urgency('Dark mode button is slightly off-center.'))Multi-Label Classification
Sometimes a piece of text belongs to multiple categories simultaneously. Multi-label classification returns all applicable categories:
def multi_label_classify(text, labels):
label_list = ', '.join(labels)
prompt = f'''
Classify this text. It may belong to multiple categories.
Return JSON: {{"labels": [str], "primary_label": str}}
Available labels: {label_list}
Rules:
- Include all labels that clearly apply
- Do NOT include labels that only marginally apply
- primary_label is the single most relevant label
Text: {text}
'''
r = client.messages.create(model='claude-opus-4-5', max_tokens=100, messages=[{'role': 'user', 'content': prompt}])
return json.loads(r.content[0].text)
labels = ['technical', 'billing', 'account', 'bug_report', 'feature_request', 'security']
text = 'I found a bug that exposes other users billing information on my account page.'
print(multi_label_classify(text, labels))Classification Prompt Templates
Reusable classification prompt template that works for any category set:
def build_classifier(category_definitions, additional_rules=''):
cats = '\n'.join(f'- {k}: {v}' for k, v in category_definitions.items())
return f'''
Classify the input text into exactly one category below.
Return JSON: {{"category": str, "confidence": "high|medium|low"}}
Categories:
{cats}
{('\nAdditional rules:\n' + additional_rules) if additional_rules else ''}
Text: {{text}}
'''
language_classifier = build_classifier({
'formal': 'Business or academic writing, professional context',
'informal': 'Casual, conversational, slang or colloquial',
'technical': 'Domain-specific jargon, code, or specialized terminology',
'emotional': 'High emotional content, personal, expressive'
})
print(language_classifier[:200])Handling Ambiguous Classifications
Some inputs genuinely fit multiple categories. Design classification prompts to handle ambiguity explicitly:
AMBIGUITY_PROMPT = '''
Classify this customer message. If the message is ambiguous or could fit multiple categories,
choose the category that would be most useful for routing it to the correct team.
Return JSON:
{{
"category": str,
"is_ambiguous": true | false,
"alternative": str or null,
"reasoning": str
}}
Categories: billing, technical_support, sales, account_management
Message: {message}
'''
message = 'I upgraded my plan but I am still seeing the free tier features.'
r = client.messages.create(
model='claude-opus-4-5', max_tokens=150,
messages=[{'role': 'user', 'content': AMBIGUITY_PROMPT.format(message=message)}]
)
print(json.loads(r.content[0].text))Batch Classification for Efficiency
Classifying items one by one is expensive. Batch classification processes multiple inputs in one API call:
def batch_classify(items, categories):
items_str = '\n'.join(f'{i+1}. {item}' for i, item in enumerate(items))
cat_str = ', '.join(categories)
prompt = f'''
Classify each item below. Categories: {cat_str}
Return JSON: {{"results": [{{"id": int, "category": str, "confidence": str}}]}}
Items:
{items_str}
'''
r = client.messages.create(model='claude-opus-4-5', max_tokens=300, messages=[{'role': 'user', 'content': prompt}])
return json.loads(r.content[0].text)['results']
texts = [
'Absolutely love this product!',
'It crashed twice today.',
'What is your refund policy?',
'The color options are limited.'
]
results = batch_classify(texts, ['positive_feedback', 'bug_report', 'inquiry', 'feature_request'])
for r in results:
print(f'{texts[r["id"]-1][:30]}... -> {r["category"]} ({r["confidence"]})')Evaluating Classifier Accuracy
LLM classifiers need systematic evaluation against labeled examples. Build a small test set and measure accuracy:
labeled_test_set = [
{'text': 'Great product, no issues!', 'expected': 'positive'},
{'text': 'The app keeps crashing on startup.', 'expected': 'negative'},
{'text': 'The screen resolution is 1080p.', 'expected': 'neutral'},
{'text': 'Worst experience of my life.', 'expected': 'negative'},
{'text': 'Works as advertised, decent value.', 'expected': 'positive'},
]
def evaluate_classifier(test_set, classify_fn):
correct = 0
for example in test_set:
result = classify_fn(example['text'])
if result['sentiment'] == example['expected']:
correct += 1
else:
print(f'WRONG: "{example["text"][:40]}" -> got {result["sentiment"]}, expected {example["expected"]}')
accuracy = correct / len(test_set)
print(f'Accuracy: {correct}/{len(test_set)} = {accuracy:.0%}')
return accuracy
evaluate_classifier(labeled_test_set, classify_sentiment)Few-Shot Examples in Classification Prompts
Adding few-shot examples to a classification prompt improves accuracy for edge cases and nuanced categories. Place 2-3 examples that demonstrate the distinction between similar categories:
few_shot_classifier = '''
Classify each customer message as: billing, technical, or general.
Examples:
Input: "I was charged twice for this month." -> billing
Input: "The app crashes when I open the dashboard." -> technical
Input: "Do you have a mobile app?" -> general
Input: "My invoice shows the wrong plan." -> billing
Input: "I cannot log in, I get error 403." -> technical
Now classify:
Input: {message}
Return JSON: {"category": str, "confidence": "high|medium|low"}
'''
message = "My subscription was renewed but I cancelled last week."
prompt = few_shot_classifier.replace("{message}", message)
print(prompt)Quick Check
What is the main advantage of using an LLM for text classification compared to a traditional trained classifier?
LLM as Classifier — Key Takeaways
LLM-based classification is powerful, flexible, and fast to deploy:
- Define categories with descriptions — not just names — to handle nuanced cases correctly
- Return JSON with category + confidence for every classification
- Common patterns: sentiment (positive/negative/neutral), intent, topic, urgency
- Multi-label classification returns all applicable categories plus a primary label
- Batch classification processes multiple inputs in one API call for efficiency
- Handle ambiguous inputs explicitly — ask for the primary category plus alternatives and reasoning
- Evaluate accuracy against a labeled test set before production deployment
Frequently asked questions
Is the “LLM as Text Classifier” lesson free?
Yes — the full text of “LLM as Text Classifier” is free to read here on the web, and the AI Prompt Engineering course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Prompt Engineering course, upgrade to CoddyKit PRO.
What will I learn in “LLM as Text Classifier”?
Using prompts for sentiment, intent, topic, and multi-label classification. You practise AI Prompt Engineering with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start AI Prompt Engineering?
No prior experience is required. AI Prompt Engineering on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “LLM as Text Classifier” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this AI Prompt Engineering lesson?
Yes. Every AI Prompt Engineering lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.