What AI Cannot Do
Limitations: real-time data, memory, reasoning errors, and confidently wrong answers.
What AI Cannot Do is a free AI Prompt Engineering lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Prompt Engineering learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
The Limitations You Must Know
AI language models are powerful — but they have hard limits. Misunderstanding these limits leads to wasted effort, wrong answers, and frustrated users.
The four biggest limitations: no real-time internet access, no persistent memory between sessions, confident hallucinations, and reasoning errors in math and logic.
No Real-Time Internet Access
By default, LLMs are completely offline at inference time. They cannot:
- Look up today's stock prices
- Check the current weather
- Access URLs you mention
- Search Google or any other source
If you ask 'What is Tesla's stock price right now?', the model will either refuse or guess based on training data — which may be months or years out of date.
import anthropic
client = anthropic.Anthropic(api_key='sk-ant-your-key-here')
response = client.messages.create(
model='claude-opus-4-5',
max_tokens=128,
messages=[{
'role': 'user',
'content': 'What is Bitcoin\'s price right now in USD?'
}]
)
# The model will acknowledge it cannot access real-time data
print(response.content[0].text)
# To add real-time data, you must inject it yourself:
current_price = 67500 # fetched from an exchange API by YOUR code
response2 = client.messages.create(
model='claude-opus-4-5',
max_tokens=128,
messages=[{
'role': 'user',
'content': f'Bitcoin price as of now: ${current_price}. Is this above or below $70,000?'
}]
)
print(response2.content[0].text)Knowledge Cutoff Dates
Every LLM is trained on a snapshot of the internet up to a specific date — its knowledge cutoff.
Events, laws, products, research papers, and people that emerged after the cutoff are unknown to the model. It may still answer confidently, but those answers come from extrapolation, not actual knowledge.
Always check the model's stated cutoff date for time-sensitive tasks.
import openai
client = openai.OpenAI(api_key='sk-your-key-here')
# Ask the model to disclose its cutoff and caveats
response = client.chat.completions.create(
model='gpt-4o',
messages=[{
'role': 'user',
'content': (
'I need to know about the latest AI models released in the past 3 months. '
'Please state your knowledge cutoff date and any caveats before answering.'
)
}]
)
print(response.choices[0].message.content)
# Best practice: inject a date stamp so the model knows the current date
from datetime import date
today = date.today().isoformat()
response2 = client.chat.completions.create(
model='gpt-4o',
system=f'Today is {today}. Your knowledge cutoff may be earlier — say so if relevant.',
messages=[{'role': 'user', 'content': 'What are the latest LLM releases?'}]
)No Persistent Memory Between Sessions
When you start a new session, the model has no memory of any previous conversation — even one from 5 minutes ago.
This is not a bug; it is how the stateless API works. Every session starts from a blank context.
To persist information across sessions you must store it yourself (in a database or file) and re-inject it into the system message or conversation history.
import json
import anthropic
client = anthropic.Anthropic(api_key='sk-ant-your-key-here')
# Simulate storing user preferences between sessions
def load_user_profile(user_id):
# In production: load from database
return {'name': 'Alice', 'preferred_language': 'Python', 'skill_level': 'intermediate'}
def build_system_message(profile):
return (
f'The user\'s name is {profile["name"]}. '
f'They prefer {profile["preferred_language"]} examples. '
f'Their skill level is {profile["skill_level"]}. '
f'Tailor all responses accordingly.'
)
profile = load_user_profile('user-123')
response = client.messages.create(
model='claude-opus-4-5',
max_tokens=256,
system=build_system_message(profile),
messages=[{'role': 'user', 'content': 'Show me how to read a file.'}]
)
print(response.content[0].text)Hallucinations: Confident and Wrong
Hallucination is when the model generates plausible-sounding text that is factually incorrect — and states it with full confidence.
Common hallucination types:
- Fabricated citations and paper titles
- Wrong dates, names, or statistics
- Invented company details or product specs
- Non-existent APIs or function names
The model has no internal fact-checker — it generates what seems statistically likely, not what is verified.
import anthropic
client = anthropic.Anthropic(api_key='sk-ant-your-key-here')
# Asking for a citation is a classic hallucination trigger
response = client.messages.create(
model='claude-opus-4-5',
max_tokens=256,
messages=[{
'role': 'user',
'content': (
'Cite 3 peer-reviewed papers about the effect of social media on teen anxiety. '
'Include author names, journal names, and publication years.'
)
}]
)
print(response.content[0].text)
# WARNING: verify every citation independently — some may be fabricatedReducing Hallucinations
You cannot eliminate hallucinations, but you can reduce them significantly:
- Provide the source material — ask the model to answer only from a pasted document
- Ask for confidence — instruct the model to say 'I don't know' when unsure
- Use lower temperature — reduces wild guesses
- Verify independently — always fact-check important outputs
- Use retrieval augmentation — inject real-time facts before asking
import anthropic
client = anthropic.Anthropic(api_key='sk-ant-your-key-here')
# Ground the model with provided source material
document = (
'According to the 2023 Pew Research report, 46% of US teens say '
'they are online almost constantly, up from 24% in 2014-2015.'
)
response = client.messages.create(
model='claude-opus-4-5',
max_tokens=256,
system=(
'Answer ONLY using the provided document. '
'If the answer is not in the document, say: "The document does not cover this."'
),
messages=[{
'role': 'user',
'content': f'Document:\n{document}\n\nQuestion: What percentage of US teens are online almost constantly?'
}]
)
print(response.content[0].text)Reasoning Errors in Math
LLMs are not calculators. They generate tokens that look like correct math — but they make errors on:
- Multi-step arithmetic
- Large number operations
- Percentage and unit conversions
- Logic puzzles with many variables
For anything involving numbers, always use code execution or an external calculator, then have the model interpret the result.
import openai
client = openai.OpenAI(api_key='sk-your-key-here')
# Bad practice: ask the LLM to compute a complex calculation directly
response_direct = client.chat.completions.create(
model='gpt-4o',
messages=[{'role': 'user', 'content': 'What is 17.83% of 348,921.47?'}]
)
print('LLM answer:', response_direct.choices[0].message.content)
# Good practice: compute in Python, then ask LLM to explain it
result = round(348921.47 * 0.1783, 2)
response_explained = client.chat.completions.create(
model='gpt-4o',
messages=[{
'role': 'user',
'content': f'17.83% of 348,921.47 is ${result}. Explain what this means for a budget report.'
}]
)
print('Explained:', response_explained.choices[0].message.content)Complex Logic and Reasoning Limits
LLMs struggle with tasks that require maintaining many simultaneous constraints or tracking state across many steps:
- Long logical proofs with many steps
- Scheduling problems with many constraints
- Code with deep nested logic
- Graph traversal or combinatorial problems
Chain-of-thought prompting (asking the model to 'think step by step') improves performance significantly — but does not eliminate errors.
import anthropic
client = anthropic.Anthropic(api_key='sk-ant-your-key-here')
# Chain-of-thought improves complex reasoning
response = client.messages.create(
model='claude-opus-4-5',
max_tokens=512,
messages=[{
'role': 'user',
'content': (
'A train leaves Station A at 9:00 AM traveling at 80 km/h. '
'Another train leaves Station B (300 km away) at 10:00 AM traveling at 100 km/h toward Station A. '
'At what time do they meet?\n\n'
'Think step by step before giving your answer.'
)
}]
)
print(response.content[0].text)File and Image Limitations
Base LLM APIs have file-handling constraints you should know:
- You cannot send a PDF and have the model 'read' it unless you extract the text first
- Image inputs require a multimodal model (GPT-4o, Claude with vision enabled)
- Audio, video, and spreadsheets typically require preprocessing before the model can use them
Always convert documents to text before including them in a prompt unless using a multimodal endpoint.
import anthropic
import base64
from pathlib import Path
client = anthropic.Anthropic(api_key='sk-ant-your-key-here')
# Images require explicit base64 encoding and vision-capable model
image_data = base64.standard_b64encode(Path('chart.png').read_bytes()).decode('utf-8')
response = client.messages.create(
model='claude-opus-4-5',
max_tokens=256,
messages=[{
'role': 'user',
'content': [
{
'type': 'image',
'source': {'type': 'base64', 'media_type': 'image/png', 'data': image_data}
},
{'type': 'text', 'text': 'Describe what this chart shows.'}
]
}]
)
print(response.content[0].text)What AI Does Well — A Balanced View
Knowing the limits helps you use AI where it excels:
- Language tasks: writing, editing, summarizing, translating — excellent
- Pattern recognition in text: classifying, extracting — excellent
- Brainstorming: generating many diverse ideas — excellent
- Math and logic: delegate to code, use AI for interpretation — use tools
- Real-time facts: inject data yourself, use AI for reasoning — use retrieval
- Memory: store externally, inject back — use a database
# Pattern: inject real-time context + use AI for reasoning, not retrieval
import anthropic
from datetime import datetime
client = anthropic.Anthropic(api_key='sk-ant-your-key-here')
# Your application fetches these from real sources
weather_data = {'city': 'London', 'temp_c': 12, 'condition': 'rainy'}
news_headline = 'UK inflation drops to 2.3% in April 2025'
context = (
f'Current date: {datetime.now().strftime("%Y-%m-%d")}\n'
f'Weather in {weather_data["city"]}: {weather_data["temp_c"]}C, {weather_data["condition"]}\n'
f'Today\'s top news: {news_headline}'
)
response = client.messages.create(
model='claude-opus-4-5',
max_tokens=256,
system='You are a helpful assistant. Use only the provided context for current facts.',
messages=[{'role': 'user', 'content': f'{context}\n\nWhat should I wear today and what is the economic mood?'}]
)
print(response.content[0].text)The Golden Rule: Verify AI Outputs
The single most important habit when using AI: verify outputs before acting on them.
- Facts → check primary sources
- Code → run it and test edge cases
- Math → compute independently
- Citations → search for them in Google Scholar
- Medical / legal / financial advice → consult a licensed professional
Use AI to draft, generate, and brainstorm — use your own judgment to validate.
import openai
client = openai.OpenAI(api_key='sk-your-key-here')
# Ask the model to flag its own uncertainty
response = client.chat.completions.create(
model='gpt-4o',
messages=[
{
'role': 'system',
'content': (
'After every response, add a line starting with CONFIDENCE: '
'and rate your certainty as HIGH, MEDIUM, or LOW, '
'with a brief reason.'
)
},
{
'role': 'user',
'content': 'Who won the 2023 FIFA Women\'s World Cup and what was the final score?'
}
]
)
print(response.choices[0].message.content)Knowledge Check
A developer asks an LLM to calculate 23.7% of 1,456,820 and use the result to write a financial report summary. What is the risk in this workflow?
AI Limitations — Recap
The key limitations to always keep in mind:
- No real-time internet — inject live data from your own code
- Knowledge cutoff — the model knows nothing after its training date
- No session memory — store context externally and re-inject it
- Hallucinations — verify facts, citations, and code independently
- Math errors — compute in code, use AI for interpretation
- Logic limits — use chain-of-thought; still verify complex reasoning
Understanding limits is what separates effective AI users from frustrated ones.
Frequently asked questions
Is the “What AI Cannot Do” lesson free?
Yes — the full text of “What AI Cannot Do” is free to read here on the web, and the AI Prompt Engineering course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Prompt Engineering course, upgrade to CoddyKit PRO.
What will I learn in “What AI Cannot Do”?
Limitations: real-time data, memory, reasoning errors, and confidently wrong answers. You practise AI Prompt Engineering with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start AI Prompt Engineering?
No prior experience is required. AI Prompt Engineering on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “What AI Cannot Do” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this AI Prompt Engineering lesson?
Yes. Every AI Prompt Engineering lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.