AI Engineering Academy · Aula

Projetando a arquitetura de produção

Projetando a arquitetura de produção é uma aula grátis de AI Engineering Academy no CoddyKit. Esta é a aula 1 de 4. Você pode ler a aula completa abaixo gratuitamente — depois pratica ao vivo no navegador com um editor de código integrado e um tutor de IA 24/7. Faz parte do caminho de aprendizado de AI Engineering Academy, e seu progresso é sincronizado entre a web e o app CoddyKit. O curso de AI Engineering Academy inclui 4 aulas no total.

Partes desta aula ainda não foram traduzidas e aparecem em inglês.

Choosing a Capstone Project

The capstone project ties together everything from the track: RAG, agents, streaming, caching, observability, and security. A good capstone project is meaningfully complex — requiring at least three distinct AI components — yet scoped small enough to ship in days rather than months. Classic examples: a production-grade document Q&A assistant, an autonomous research agent with human-in-the-loop oversight, or an enterprise data extraction pipeline.

Identifying System Components

Start by listing the distinct components your system needs. A production AI engineering project typically includes: an ingestion layer (document loading, chunking, embedding, vector store indexing), a retrieval layer (hybrid search, re-ranking), an agent layer (function calling, tool execution), an API layer (FastAPI backend, streaming endpoints), and an observability layer (tracing, metrics, alerts). Sketch the data flow between them before writing code.

# Component inventory for a Document QA Assistant:
COMPONENTS = [
    'document_ingestion',   # PDF/Word -> chunks -> embeddings -> pgvector
    'hybrid_retriever',     # BM25 + dense + RRF
    'reranker',             # Cohere rerank
    'qa_agent',             # GPT-4o with RAG + function calling
    'semantic_cache',       # Redis + embedding similarity
    'streaming_api',        # FastAPI StreamingResponse
    'tracing',              # LangSmith or Langfuse
    'prompt_injection_filter', # Input sanitization
    'eval_pipeline',        # Automated quality scoring
]

Choosing the Right Stack

Select technology based on your team's familiarity and the system's actual requirements rather than novelty. A reasonable default stack: FastAPI for the API layer, pgvector for vector storage (reuses existing PostgreSQL infrastructure), LangChain LCEL for pipeline composition, Redis for semantic caching and rate-limit state, LangSmith for tracing, and PostgreSQL for user data and evaluation results. Add components only when simpler options do not meet requirements.

# Technology decisions and their rationale
STACK = {
    'api':           ('FastAPI',    'Async support, OpenAPI docs, streaming easy'),
    'vector_store':  ('pgvector',   'Already on PostgreSQL, no extra infra'),
    'llm_primary':   ('gpt-4o',     'Best quality for the use case'),
    'llm_fallback':  ('claude-3.5', 'Different provider for resilience'),
    'cache':         ('Redis',      'Sub-ms lookup, TTL support built-in'),
    'tracing':       ('LangSmith',  'Native LangChain integration'),
    'embedding':     ('text-embedding-3-small', 'Good quality, low cost'),
    'reranker':      ('Cohere',     'Best-in-class rerank API'),
}

Drawing the Architecture Diagram

Document the system architecture as a data-flow diagram showing each component, the data that flows between them, and the direction of flow. Include both the ingestion path (offline: documents → chunks → embeddings → vector store) and the query path (online: user query → cache check → retrieval → reranking → LLM → streaming response). This diagram is your north star for implementation and helps new team members understand the system instantly.

# Data flow (ASCII art):
#
# INGESTION PATH (offline):
# Documents --> Loader --> Chunker --> Embedder --> pgvector
#                                             |
#                                         BM25 index
#
# QUERY PATH (online):
# User Query
#    |-> Semantic Cache (hit: return) -> miss:
#    |-> Hybrid Retriever (BM25 + dense)
#    |-> Cohere Reranker
#    |-> LangChain LCEL Chain
#    |-> GPT-4o (streaming) --> FastAPI StreamingResponse
#    |-> LangSmith (trace every step)

Defining API Contracts Early

Define your API endpoints and their request/response schemas in FastAPI before implementing backend logic. This creates a contract between the API and any frontend consumers and makes parallel development possible. Document every endpoint with OpenAPI descriptions. At minimum, define endpoints for: document ingestion, chat/query, conversation history, evaluation results, and system health.

from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI(title='Document QA Assistant', version='1.0')

class QueryRequest(BaseModel):
    question: str
    conversation_id: str | None = None
    max_chunks: int = 5
    stream: bool = True

class QueryResponse(BaseModel):
    answer: str
    sources: list
    cached: bool
    latency_ms: int
    trace_id: str

@app.post('/query', response_model=QueryResponse)
async def query_endpoint(request: QueryRequest):
    pass  # implementation next lesson

Estimating and Budgeting Costs

Estimate your system's monthly API cost before writing a single line of backend code. Count: expected daily active users × queries per user × average tokens per query. For a system with 100 DAU, 10 queries/day, and 3,000 tokens per query at GPT-4o pricing, that is 3 million tokens per day — roughly $45/day or $1,350/month. This estimate tells you whether the system is economically viable and which optimizations (caching, model routing) are worth implementing.

# Cost estimation model
DAU = 100           # daily active users
QPD = 10            # queries per user per day
TOKENS_PER_QUERY = {
    'prompt_tokens': 2000,  # system + context + question
    'completion_tokens': 500
}

PRICE_GPT4O_INPUT = 2.50 / 1_000_000   # per token
PRICE_GPT4O_OUTPUT = 10.00 / 1_000_000  # per token

daily_cost = DAU * QPD * (
    TOKENS_PER_QUERY['prompt_tokens'] * PRICE_GPT4O_INPUT +
    TOKENS_PER_QUERY['completion_tokens'] * PRICE_GPT4O_OUTPUT
)
print(f'Daily: ${daily_cost:.2f}, Monthly: ${daily_cost * 30:.2f}')

Planning the Ingestion Pipeline

Design the ingestion pipeline as an offline batch process that runs on demand or on a schedule. Define what document types you will support (PDF, DOCX, HTML, plain text), the chunking strategy, the embedding model, and the metadata fields to store alongside each vector. Metadata is critical for filtered retrieval — without it, you cannot restrict search to documents from a specific date range, author, or category.

from dataclasses import dataclass
from typing import Optional

@dataclass
class ChunkMetadata:
    doc_id: str
    source_file: str
    page_number: Optional[int]
    section_title: Optional[str]
    created_at: str
    author: Optional[str]
    doc_type: str  # 'pdf', 'docx', 'html'

# Ingestion config
INGESTION_CONFIG = {
    'chunk_size': 800,
    'chunk_overlap': 100,
    'embedding_model': 'text-embedding-3-small',
    'embedding_dimensions': 1536,
    'batch_size': 100,  # chunks per embedding API call
}

Security Architecture Decisions

Make security decisions upfront rather than retrofitting them later. Define: how users authenticate (JWT, API keys), what data can be retrieved per user (row-level security in pgvector queries), how prompt injection is detected, what output scanning is applied, and which actions require multi-factor confirmation. Each decision has performance implications that affect architecture choices throughout the system.

SECURITY_DECISIONS = {
    'auth': 'JWT with 24h expiry',
    'data_isolation': 'tenant_id column in all vector metadata, filter on every query',
    'injection_detection': 'rule-based pre-filter + LLM secondary check for complex inputs',
    'output_scanning': 'check for PII, system prompt leakage patterns',
    'destructive_actions': 'require confirmation token for delete operations',
    'rate_limiting': '20 queries/minute per user, 429 with Retry-After header',
    'key_storage': 'AWS Secrets Manager, rotated every 90 days'
}

Defining Success Metrics

Define what success looks like for your system before you build it. A set of concrete, measurable success criteria keeps development focused and gives you clear go/no-go criteria for deployment. Include metrics for: quality (eval score on test set), latency (p95 TTFT and total), cost (per-query cost target), and reliability (uptime SLA). Post these on the project README so all contributors share the same target.

SUCCESS_METRICS = {
    # Quality
    'min_correctness_score': 4.0,       # out of 5, LLM-as-judge
    'min_retrieval_hit_rate': 0.85,     # top-5 chunk contains answer
    # Latency
    'p95_ttft_ms': 800,                 # time to first token
    'p95_total_latency_ms': 8000,       # full response
    # Cost
    'max_cost_per_query_usd': 0.05,     # $0.05 per Q&A
    # Reliability
    'target_uptime': 0.999,             # 99.9%
    'cache_hit_rate_target': 0.25,      # 25% queries served from cache
}

Documenting Architecture Trade-offs

Every architecture decision involves trade-offs. Document them explicitly in an Architecture Decision Record (ADR): what decision was made, what alternatives were considered, and why this option was chosen. For example: 'Chose pgvector over Pinecone because we already run PostgreSQL, reducing operational overhead. Trade-off: maximum scale is limited to ~10M vectors without sharding.' Future team members will thank you for this transparency.

# Architecture Decision Records (ADRs):
#
# ADR-001: Use pgvector for vector storage
# Decision: pgvector in existing PostgreSQL
# Alternatives: Pinecone, Weaviate, Qdrant
# Reason: No new infra, row-level security native, familiar operations
# Trade-offs: Limited to ~5M vectors before performance degrades
#
# ADR-002: GPT-4o as primary model
# Decision: gpt-4o for all user-facing queries
# Alternatives: gpt-4o-mini (cheaper), Claude (alternative)
# Reason: Highest quality for use case, Claude as fallback
# Trade-offs: $0.04/query vs $0.002 for gpt-4o-mini

Sketching the Deployment Topology

Define how your system will be deployed before writing any infrastructure code. Map each component to a deployment unit: the FastAPI backend as a Docker container, the ingestion pipeline as a separate worker service, pgvector as a managed PostgreSQL instance, Redis as a managed cache. Specify which components are stateless (can be scaled horizontally) versus stateful (require careful scaling strategies). A deployment diagram saves hours of rework later.

# Deployment topology:
#
# Internet -> Load Balancer (AWS ALB)
#                |
#            FastAPI API  (stateless, 2-4 containers, auto-scale)
#                |
#         +------+------+
#         |             |
#    pgvector        Redis Cache
#  (AWS RDS Postgres) (Elasticache)
#         |
#    Ingestion Worker  (separate container, manual trigger)
#         |
#    LangSmith (external SaaS, traces only)
#
# All containers in same VPC, no public access to DB/cache

Quick Check

Test your understanding of production AI system architecture design.

Lesson Recap

In this lesson you learned: component inventory and data-flow diagrams create a shared architectural vision before coding begins, API contracts defined upfront enable parallel development and make requirements explicit, and success metrics defined in advance keep the team aligned on quality, latency, cost, and reliability targets. Next up we implement the core RAG and agent features.

Perguntas Frequentes

A aula “Projetando a arquitetura de produção” é grátis?

Sim — o texto completo de “Projetando a arquitetura de produção” é grátis para ler aqui na web. Para praticá-la interativamente (um editor de código integrado e um tutor de IA 24/7) e desbloquear o restante do curso de AI Engineering Academy, atualize para CoddyKit PRO. O curso de AI Engineering Academy inclui 4 aulas no total.

O que vou aprender em “Projetando a arquitetura de produção”?

Escolha um projeto de conclusão, esboce a arquitetura completa, incluindo o pipeline de RAG, a camada do agente, o armazenamento em cache, a observabilidade e a API, e documente as decisões de design… Você pratica AI Engineering Academy com código prático que executa diretamente no navegador, e um tutor de IA 24/7 responde suas dúvidas enquanto trabalha na aula.

Preciso ter experiência prévia para começar AI Engineering Academy?

Nenhuma experiência prévia é necessária. AI Engineering Academy no CoddyKit é estruturado para alunos iniciantes até avançados, então você pode começar aqui ou desde o início e aprender no seu ritmo. Esta é a aula 1 de 4.

Quanto tempo leva a aula “Projetando a arquitetura de produção”?

A maioria das aulas CoddyKit leva cerca de 5–10 minutos. Cada uma é compacta e interativa, então você faz progresso constante e retoma exatamente de onde parou entre web e app.

Posso escrever e executar código nesta aula de AI Engineering Academy?

Sim. Cada aula de AI Engineering Academy inclui um editor de código integrado, então você escreve e executa código real direto no navegador e recebe feedback de IA instantaneamente — nenhuma configuração local necessária.

Todas as aulas deste curso

Projetando a arquitetura de produção
Implementando recursos essenciais de RAG e agentes
Reforço: segurança, armazenamento em cache e confiabilidade
Avaliação, implantação e retrospectiva

← Voltar para AI Engineering Academy