세계 모델과 룩어헤드

행동하기 전에 예측하기

세계 모델이란 무엇입니까

prediction_prompt = '''
Current state: {state}
Proposed action: {action}
What is the most likely outcome? Return JSON: {new_state, success_probability}.
'''

선행 탐색이 도움이 되는 이유

Simple 1-Step Lookahead

candidates = [a1, a2, a3]
predictions = [predict(state, a) for a in candidates]
best = max(zip(predictions, candidates), key=lambda p: p[0].success_probability)[1]
act(best)

다단계 선행 탐색(트리 탐색)

def search(state, depth=2):
    if depth == 0:
        return value(state)
    actions = candidate_actions(state)
    best_score = -math.inf
    for a in actions:
        next_state = predict(state, a)
        score = search(next_state, depth - 1)
        best_score = max(best_score, score)
    return best_score

사고의 트리

import random

def continue_thought(question):
    return question + ' -> idea' + str(random.randint(1, 100))

def score(t):
    return len(t)

def expand_top_k(thoughts, scores, k):
    ranked = sorted(zip(thoughts, scores), key=lambda x: -x[1])
    return [t for t, s in ranked[:k]]

def ToT(question):
    thoughts = [continue_thought(question) for _ in range(5)]
    for depth in range(2):
        scores = [score(t) for t in thoughts]
        thoughts = expand_top_k(thoughts, scores, k=3)
    return thoughts

result = ToT('How to reduce agent latency?')
print('Top thoughts:', result)

몬테카를로 트리 탐색(MCTS)

선행 탐색 비용

휴리스틱 가지치기

예측 보정

for trace in recent_traces:
    predicted = trace.predicted_outcome
    actual = trace.actual_outcome
    log.info('prediction-accuracy', match=(predicted == actual))

선행 탐색을 사용하지 말아야 할 때 NOT

선행 탐색을 사용해야 할 때 TO

o1 방식의 내부 선행 탐색

선행 탐색의 절충

복습