추론 모델은 어떻게 다른가요?

추론 모델이란 무엇인가요

일반 모델과 추론 모델: 보이는 차이

OpenAI o 시리즈: 애플리케이션 프로그래밍 인터페이스 동작

import openai

client = openai.OpenAI(api_key='sk-...')

# o3 — reasoning happens internally
response = client.chat.completions.create(
    model='o3',
    messages=[
        {'role': 'user', 'content': 'Solve: A train leaves Chicago at 9am going 60mph. Another leaves NYC at 10am going 80mph. If the distance is 790 miles, when do they meet?'}
    ],
    # reasoning_effort='high'  # Optional: 'low', 'medium', 'high'
)

print(response.choices[0].message.content)
# Check how many tokens were used for thinking:
print('Input tokens:', response.usage.prompt_tokens)
print('Output tokens:', response.usage.completion_tokens)

Claude 확장 사고: 애플리케이션 프로그래밍 인터페이스 동작

import anthropic

client = anthropic.Anthropic(api_key='sk-ant-...')

# Enable extended thinking
response = client.messages.create(
    model='claude-opus-4-5',
    max_tokens=16000,
    thinking={
        'type': 'enabled',
        'budget_tokens': 10000  # Max tokens for thinking
    },
    messages=[{
        'role': 'user',
        'content': 'What is the 100th prime number?'
    }]
)

# Response contains both thinking blocks and text blocks
for block in response.content:
    if block.type == 'thinking':
        print('THINKING:', block.thinking[:200], '...')
    elif block.type == 'text':
        print('ANSWER:', block.text)

사고 토큰 실시간 전송

import anthropic

client = anthropic.Anthropic(api_key='sk-ant-...')

def stream_with_thinking(question):
    with client.messages.stream(
        model='claude-opus-4-5',
        max_tokens=16000,
        thinking={'type': 'enabled', 'budget_tokens': 8000},
        messages=[{'role': 'user', 'content': question}]
    ) as stream:
        current_block_type = None
        for event in stream:
            # Track which block type we're in
            if hasattr(event, 'type'):
                if 'thinking' in str(event.type):
                    current_block_type = 'thinking'
                elif 'text' in str(event.type):
                    current_block_type = 'text'
            # Print text delta
            if hasattr(event, 'delta') and hasattr(event.delta, 'text'):
                prefix = '[THINK] ' if current_block_type == 'thinking' else '[ANS] '
                print(prefix + event.delta.text, end='', flush=True)

stream_with_thinking('Explain why P != NP is unproven.')

내부에서 일어나는 일: 메모장

budget_tokens: 사고 깊이 조절

import anthropic

client = anthropic.Anthropic(api_key='sk-ant-...')

def ask_with_budget(question, budget):
    r = client.messages.create(
        model='claude-opus-4-5',
        max_tokens=budget + 2048,  # must exceed budget_tokens
        thinking={'type': 'enabled', 'budget_tokens': budget},
        messages=[{'role': 'user', 'content': question}]
    )
    thinking_tokens = sum(
        len(b.thinking.split()) * 1.3  # rough estimate
        for b in r.content if b.type == 'thinking'
    )
    answer = next(b.text for b in r.content if b.type == 'text')
    return answer, int(thinking_tokens)

# Same hard question with different budgets
q = 'Prove that the square root of 2 is irrational.'
ans_small, tok_small = ask_with_budget(q, 1024)
ans_large, tok_large = ask_with_budget(q, 8000)
print(f'Small budget ({tok_small} thinking tokens): {ans_small[:100]}')
print(f'Large budget ({tok_large} thinking tokens): {ans_large[:100]}')

온도 설정과 추론 모델

import anthropic

client = anthropic.Anthropic(api_key='sk-ant-...')

# For Claude with extended thinking, temperature=1 is the default
# and best practice
response = client.messages.create(
    model='claude-opus-4-5',
    max_tokens=10000,
    temperature=1,  # Required to be 1 when extended thinking is enabled
    thinking={
        'type': 'enabled',
        'budget_tokens': 5000
    },
    messages=[{
        'role': 'user',
        'content': 'Write a Python function to find all prime numbers up to N.'
    }]
)
print(response.content[-1].text[:300])

성능 평가: 추론이 강점을 보이는 경우

지연 시간의 현실

import time
import anthropic

client = anthropic.Anthropic(api_key='sk-ant-...')

def timed_reasoning_call(question, budget):
    start = time.time()
    response = client.messages.create(
        model='claude-opus-4-5',
        max_tokens=budget + 2000,
        thinking={'type': 'enabled', 'budget_tokens': budget},
        messages=[{'role': 'user', 'content': question}]
    )
    elapsed = time.time() - start
    answer = next(b.text for b in response.content if b.type == 'text')
    print(f'Latency: {elapsed:.1f}s | Answer: {answer[:100]}')
    return answer

# Hard problem — expect 20-45 seconds
timed_reasoning_call(
    'Design a database schema for a multi-tenant SaaS billing system.',
    budget=8000
)

추론 모델과 시스템 프롬프트

지식 확인: 추론 모델의 사고

복습: 추론 모델의 차이