AI Prompt Engineering · Lesson

What Is Temperature in LLMs?

Temperature as creative control: 0=deterministic, 2=chaotic, and everything between.

What Is Temperature in LLMs? is a free AI Prompt Engineering lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Prompt Engineering learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

How LLMs Choose the Next Token

At each step, an LLM outputs a probability distribution over all tokens in its vocabulary (~50,000 tokens for GPT). The model assigns each token a score called a logit — a raw, unnormalized number. Higher logit = more likely token.

Temperature is the parameter that controls how these raw logits are converted into probabilities before sampling.

The Softmax Function

Logits are converted to probabilities using the softmax function. Softmax takes a vector of logits and outputs a probability distribution that sums to 1.

For a small vocabulary example with 4 tokens:

import numpy as np

# Raw logits from the model
logits = np.array([2.0, 1.0, 0.5, -1.0])  # scores for 4 tokens

# Standard softmax (temperature = 1)
def softmax(logits, temperature=1.0):
    scaled = logits / temperature
    exp_scaled = np.exp(scaled - np.max(scaled))  # subtract max for numerical stability
    return exp_scaled / exp_scaled.sum()

probs = softmax(logits, temperature=1.0)
print('Probabilities:', np.round(probs, 3))
# [0.567, 0.208, 0.129, 0.095]
# Token 0 is most likely at 56.7%

Temperature = 0: Greedy Decoding

When temperature approaches 0, the softmax distribution collapses: the highest-logit token gets probability ~1.0 and all others approach 0. The model always picks the single most probable token.

This is called greedy decoding. It is deterministic — the same prompt always produces the same output. Zero randomness, zero creativity.

# Temperature = 0 (greedy)
probs_temp0 = softmax(logits, temperature=0.01)  # near-zero
print('Probs at T=0.01:', np.round(probs_temp0, 4))
# [~1.0, ~0.0, ~0.0, ~0.0]

# In practice, temperature=0 is implemented as argmax:
def greedy_sample(logits):
    return np.argmax(logits)  # always returns the index of the highest logit

token_idx = greedy_sample(logits)
print(f'Selected token index: {token_idx}')  # always 0

Temperature = 1: Standard Sampling

Temperature = 1 applies softmax with no scaling — the probability distribution reflects the model's natural confidence. The model samples according to these probabilities: likely tokens appear often, unlikely tokens appear rarely but sometimes.

This is the default for most conversational use cases. It produces varied, natural output while still being coherent.

# Temperature = 1 (standard)
probs_temp1 = softmax(logits, temperature=1.0)
print('Probs at T=1.0:', np.round(probs_temp1, 3))
# [0.567, 0.208, 0.129, 0.095]

# Sampling from this distribution:
def sample_token(probs):
    vocab = ['the', 'a', 'an', 'is']
    return np.random.choice(vocab, p=probs)

# Run 10 times to see variation
for _ in range(10):
    print(sample_token(probs_temp1), end=' ')
# Output varies each time, but 'the' appears most often

Temperature = 2: Chaotic Sampling

High temperature (> 1) flattens the probability distribution — all tokens become more equally likely. The model becomes unpredictable and often incoherent.

Temperature > 1.5 is rarely used in practice. It may produce creative unexpected outputs but usually produces nonsense.

# Temperature = 2 (chaotic)
probs_temp2 = softmax(logits, temperature=2.0)
print('Probs at T=2.0:', np.round(probs_temp2, 3))
# [0.385, 0.261, 0.211, 0.143]
# Much flatter — the 4th token (logit=-1.0) now has 14.3% chance
# (vs 9.5% at T=1.0)

# Visualization: compare distributions
for temp in [0.1, 0.5, 1.0, 1.5, 2.0]:
    probs = softmax(logits, temperature=temp)
    print(f'T={temp}: {np.round(probs, 3)}')

Temperature as a Sharpness Control

Conceptually, temperature controls sharpness of the distribution:

Low temperature (0.1–0.4): sharp peak — model is confident, picks safe/common tokens
Medium temperature (0.6–1.0): natural shape — balanced creativity and coherence
High temperature (1.2–2.0): flat distribution — model explores unlikely tokens freely

Think of it as a creativity dial: low = precise, high = adventurous.

import matplotlib
# Conceptual: what distribution shape looks like at different temps
temps = {'T=0.2 (sharp)': 0.2, 'T=1.0 (normal)': 1.0, 'T=2.0 (flat)': 2.0}
for label, temp in temps.items():
    probs = softmax(logits, temp)
    bar = '#' * int(probs[0] * 40)
    print(f'{label}: top token = {probs[0]:.1%} |{bar}|')
# T=0.2: top token = 97.2% |########################################|
# T=1.0: top token = 56.7% |######################|
# T=2.0: top token = 38.5% |###############|

Temperature and Determinism

An important nuance: temperature = 0 makes sampling deterministic. For temperature > 0, each run produces a different output because sampling is a random process. The distribution is fixed, but the sample varies.

For reproducible outputs in testing, always set temperature = 0. For production with desired variation, use a seed if the API supports it.

import openai
client = openai.OpenAI(api_key='sk-...')

# Deterministic: temperature=0
resp1 = client.chat.completions.create(
    model='gpt-4o',
    messages=[{'role': 'user', 'content': 'What is the capital of France?'}],
    temperature=0
)
resp2 = client.chat.completions.create(
    model='gpt-4o',
    messages=[{'role': 'user', 'content': 'What is the capital of France?'}],
    temperature=0
)
print(resp1.choices[0].message.content == resp2.choices[0].message.content)  # True (usually)

How Temperature Interacts with Top-p

Temperature and top-p (nucleus sampling) both shape the sampling distribution but at different stages:

Temperature: scales logits before softmax — changes the shape of the full distribution
Top-p: truncates the distribution to the top-p probability mass after softmax — samples only from the most likely token set

Using both together provides fine control. The typical recommendation: adjust only one at a time. Changing both simultaneously makes the effect hard to predict.

# Using temperature with top_p in OpenAI API
resp = client.chat.completions.create(
    model='gpt-4o',
    messages=[{'role': 'user', 'content': 'Write a one-line haiku about code.'}],
    temperature=0.9,  # moderately diverse distribution
    top_p=0.95        # sample from top 95% of probability mass
)

Practical Temperature Values by Task

Quick reference for common task types:

Factual Q&A: 0 — must be accurate, no variation needed
Code generation: 0–0.2 — syntax must be correct
Summarization: 0.3–0.5 — some variation acceptable
Chat / conversational: 0.7–0.9 — natural, varied responses
Creative writing: 0.9–1.2 — diversity desired
Brainstorming / ideation: 1.0–1.5 — explore unexpected options

TEMPERATURE_PRESETS = {
    'factual_qa': 0.0,
    'code_generation': 0.1,
    'summarization': 0.4,
    'chat': 0.8,
    'creative_writing': 1.1,
    'brainstorming': 1.3
}

def call_with_preset(task_type, prompt):
    temp = TEMPERATURE_PRESETS.get(task_type, 0.7)
    return client.chat.completions.create(
        model='gpt-4o',
        messages=[{'role': 'user', 'content': prompt}],
        temperature=temp
    )

Temperature in the API

Temperature is accepted by all major LLM APIs. The valid range is 0–2 for OpenAI, 0–1 for Anthropic Claude. Exceeding the maximum raises an error.

# OpenAI: temperature 0 to 2
client.chat.completions.create(
    model='gpt-4o',
    messages=[{'role': 'user', 'content': prompt}],
    temperature=1.2  # Valid for OpenAI
)

# Anthropic Claude: temperature 0 to 1
import anthropic
claude = anthropic.Anthropic(api_key='sk-ant-...')
claude.messages.create(
    model='claude-opus-4-5',
    max_tokens=1024,
    temperature=0.8,  # Max is 1.0 for Claude
    messages=[{'role': 'user', 'content': prompt}]
)

When Temperature Is Not Enough

Temperature alone does not always produce the diversity or precision you need. When temperature=0 still produces varied output (it can on some models with floating point non-determinism), use seed for true reproducibility. When high temperature produces nonsense, limit the vocabulary with top-p or top-k instead of pushing temperature higher.

# Seed for reproducibility (OpenAI API)
resp = client.chat.completions.create(
    model='gpt-4o',
    messages=[{'role': 'user', 'content': prompt}],
    temperature=0,
    seed=42  # Guarantees same output across calls on same model version
)

# Note: same output only guaranteed with same model version
# A model update can change outputs even with the same seed

Knowledge Check

What happens to the probability distribution over tokens when temperature is set to a very high value (e.g., 2.0)?

Recap: Temperature in LLMs

Temperature controls the randomness of token sampling by scaling logits before the softmax:

T=0: greedy — always picks the most probable token, deterministic
T=1: standard — samples according to natural probability distribution
T>1: chaotic — flattens distribution, more randomness, less coherence

Use low temperature for factual/code tasks, medium for chat, high for creative tasks. Adjust only temperature or top-p at a time, not both. Next lesson: top-p nucleus sampling.

Frequently asked questions

Is the “What Is Temperature in LLMs?” lesson free?

Yes — the full text of “What Is Temperature in LLMs?” is free to read here on the web, and the AI Prompt Engineering course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Prompt Engineering course, upgrade to CoddyKit PRO.

What will I learn in “What Is Temperature in LLMs?”?

Temperature as creative control: 0=deterministic, 2=chaotic, and everything between. You practise AI Prompt Engineering with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start AI Prompt Engineering?

No prior experience is required. AI Prompt Engineering on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “What Is Temperature in LLMs?” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this AI Prompt Engineering lesson?

Yes. Every AI Prompt Engineering lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

← Back to AI Prompt Engineering