Визуализация запусков агентов (Langfuse, LangSmith)

Два ведущих инструмента

Обзор Langfuse

Langfuse Quick Start

# pip install langfuse openai
from langfuse.openai import openai

response = openai.chat.completions.create(
    model='gpt-4o-mini',
    messages=[{'role': 'user', 'content': 'Hi'}],
    name='greeting'   # span name
)
# Trace automatically appears in Langfuse UI

Ручные span в Langfuse

from langfuse import Langfuse
lf = Langfuse()

trace = lf.trace(name='agent-run', user_id='u-42')
span = trace.span(name='search-tool', input={'query': 'shipping'})
results = search_kb('shipping')
span.end(output=results)

LangSmith Quick Start

# pip install langsmith
import os
os.environ['LANGCHAIN_TRACING_V2'] = 'true'
os.environ['LANGCHAIN_API_KEY'] = '...'

# Any LangChain code automatically logs to LangSmith
from langchain_openai import ChatOpenAI
llm = ChatOpenAI()
llm.invoke('Hello')
# Trace appears in https://smith.langchain.com

LangSmith без LangChain

from langsmith import traceable

@traceable
def ask_agent(query: str):
    # your custom agent code here
    return final_answer

Что показывает интерфейс

Теги и фильтрация

trace = lf.trace(name='agent-run', tags=['production', 'tenant:acme', 'tool:rag'])

Отзывы пользователей

trace.score(name='user-rating', value=1)   # thumbs up
trace.score(name='user-rating', value=0)   # thumbs down

Наборы данных и повторные запуски

Self-Hosting Langfuse

# docker-compose with Postgres + Clickhouse
version: '3'
services:
  langfuse:
    image: langfuse/langfuse:latest
    ports: ['3000:3000']
    environment:
      DATABASE_URL: postgresql://...
      NEXTAUTH_SECRET: '...'
      SALT: '...'
# Then point LANGFUSE_HOST=http://localhost:3000

Сравнение инструментов

Самостоятельное размещение?

Повторим