AI Engineering Academy · Урок

Проектирование архитектуры для промышленной эксплуатации

«Проектирование архитектуры для промышленной эксплуатации» — бесплатный урок AI Engineering Academy на CoddyKit. Это урок 1 из 4. Ты можешь прочитать весь урок бесплатно ниже — а потом практиковать его прямо в браузере с встроенным редактором кода и ИИ-репетитором 24/7. Это часть пути обучения AI Engineering Academy, и твой прогресс синхронизируется между веб-версией и приложением CoddyKit. Курс AI Engineering Academy содержит 4 уроков всего.

Части этого урока еще не переведены и отображаются на английском.

Choosing a Capstone Project

The capstone project ties together everything from the track: RAG, agents, streaming, caching, observability, and security. A good capstone project is meaningfully complex — requiring at least three distinct AI components — yet scoped small enough to ship in days rather than months. Classic examples: a production-grade document Q&A assistant, an autonomous research agent with human-in-the-loop oversight, or an enterprise data extraction pipeline.

Identifying System Components

Start by listing the distinct components your system needs. A production AI engineering project typically includes: an ingestion layer (document loading, chunking, embedding, vector store indexing), a retrieval layer (hybrid search, re-ranking), an agent layer (function calling, tool execution), an API layer (FastAPI backend, streaming endpoints), and an observability layer (tracing, metrics, alerts). Sketch the data flow between them before writing code.

# Component inventory for a Document QA Assistant:
COMPONENTS = [
    'document_ingestion',   # PDF/Word -> chunks -> embeddings -> pgvector
    'hybrid_retriever',     # BM25 + dense + RRF
    'reranker',             # Cohere rerank
    'qa_agent',             # GPT-4o with RAG + function calling
    'semantic_cache',       # Redis + embedding similarity
    'streaming_api',        # FastAPI StreamingResponse
    'tracing',              # LangSmith or Langfuse
    'prompt_injection_filter', # Input sanitization
    'eval_pipeline',        # Automated quality scoring
]

Choosing the Right Stack

Select technology based on your team's familiarity and the system's actual requirements rather than novelty. A reasonable default stack: FastAPI for the API layer, pgvector for vector storage (reuses existing PostgreSQL infrastructure), LangChain LCEL for pipeline composition, Redis for semantic caching and rate-limit state, LangSmith for tracing, and PostgreSQL for user data and evaluation results. Add components only when simpler options do not meet requirements.

# Technology decisions and their rationale
STACK = {
    'api':           ('FastAPI',    'Async support, OpenAPI docs, streaming easy'),
    'vector_store':  ('pgvector',   'Already on PostgreSQL, no extra infra'),
    'llm_primary':   ('gpt-4o',     'Best quality for the use case'),
    'llm_fallback':  ('claude-3.5', 'Different provider for resilience'),
    'cache':         ('Redis',      'Sub-ms lookup, TTL support built-in'),
    'tracing':       ('LangSmith',  'Native LangChain integration'),
    'embedding':     ('text-embedding-3-small', 'Good quality, low cost'),
    'reranker':      ('Cohere',     'Best-in-class rerank API'),
}

Drawing the Architecture Diagram

Document the system architecture as a data-flow diagram showing each component, the data that flows between them, and the direction of flow. Include both the ingestion path (offline: documents → chunks → embeddings → vector store) and the query path (online: user query → cache check → retrieval → reranking → LLM → streaming response). This diagram is your north star for implementation and helps new team members understand the system instantly.

# Data flow (ASCII art):
#
# INGESTION PATH (offline):
# Documents --> Loader --> Chunker --> Embedder --> pgvector
#                                             |
#                                         BM25 index
#
# QUERY PATH (online):
# User Query
#    |-> Semantic Cache (hit: return) -> miss:
#    |-> Hybrid Retriever (BM25 + dense)
#    |-> Cohere Reranker
#    |-> LangChain LCEL Chain
#    |-> GPT-4o (streaming) --> FastAPI StreamingResponse
#    |-> LangSmith (trace every step)

Defining API Contracts Early

Define your API endpoints and their request/response schemas in FastAPI before implementing backend logic. This creates a contract between the API and any frontend consumers and makes parallel development possible. Document every endpoint with OpenAPI descriptions. At minimum, define endpoints for: document ingestion, chat/query, conversation history, evaluation results, and system health.

from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI(title='Document QA Assistant', version='1.0')

class QueryRequest(BaseModel):
    question: str
    conversation_id: str | None = None
    max_chunks: int = 5
    stream: bool = True

class QueryResponse(BaseModel):
    answer: str
    sources: list
    cached: bool
    latency_ms: int
    trace_id: str

@app.post('/query', response_model=QueryResponse)
async def query_endpoint(request: QueryRequest):
    pass  # implementation next lesson

Estimating and Budgeting Costs

Estimate your system's monthly API cost before writing a single line of backend code. Count: expected daily active users × queries per user × average tokens per query. For a system with 100 DAU, 10 queries/day, and 3,000 tokens per query at GPT-4o pricing, that is 3 million tokens per day — roughly $45/day or $1,350/month. This estimate tells you whether the system is economically viable and which optimizations (caching, model routing) are worth implementing.

# Cost estimation model
DAU = 100           # daily active users
QPD = 10            # queries per user per day
TOKENS_PER_QUERY = {
    'prompt_tokens': 2000,  # system + context + question
    'completion_tokens': 500
}

PRICE_GPT4O_INPUT = 2.50 / 1_000_000   # per token
PRICE_GPT4O_OUTPUT = 10.00 / 1_000_000  # per token

daily_cost = DAU * QPD * (
    TOKENS_PER_QUERY['prompt_tokens'] * PRICE_GPT4O_INPUT +
    TOKENS_PER_QUERY['completion_tokens'] * PRICE_GPT4O_OUTPUT
)
print(f'Daily: ${daily_cost:.2f}, Monthly: ${daily_cost * 30:.2f}')

Planning the Ingestion Pipeline

Design the ingestion pipeline as an offline batch process that runs on demand or on a schedule. Define what document types you will support (PDF, DOCX, HTML, plain text), the chunking strategy, the embedding model, and the metadata fields to store alongside each vector. Metadata is critical for filtered retrieval — without it, you cannot restrict search to documents from a specific date range, author, or category.

from dataclasses import dataclass
from typing import Optional

@dataclass
class ChunkMetadata:
    doc_id: str
    source_file: str
    page_number: Optional[int]
    section_title: Optional[str]
    created_at: str
    author: Optional[str]
    doc_type: str  # 'pdf', 'docx', 'html'

# Ingestion config
INGESTION_CONFIG = {
    'chunk_size': 800,
    'chunk_overlap': 100,
    'embedding_model': 'text-embedding-3-small',
    'embedding_dimensions': 1536,
    'batch_size': 100,  # chunks per embedding API call
}

Security Architecture Decisions

Make security decisions upfront rather than retrofitting them later. Define: how users authenticate (JWT, API keys), what data can be retrieved per user (row-level security in pgvector queries), how prompt injection is detected, what output scanning is applied, and which actions require multi-factor confirmation. Each decision has performance implications that affect architecture choices throughout the system.

SECURITY_DECISIONS = {
    'auth': 'JWT with 24h expiry',
    'data_isolation': 'tenant_id column in all vector metadata, filter on every query',
    'injection_detection': 'rule-based pre-filter + LLM secondary check for complex inputs',
    'output_scanning': 'check for PII, system prompt leakage patterns',
    'destructive_actions': 'require confirmation token for delete operations',
    'rate_limiting': '20 queries/minute per user, 429 with Retry-After header',
    'key_storage': 'AWS Secrets Manager, rotated every 90 days'
}

Defining Success Metrics

Define what success looks like for your system before you build it. A set of concrete, measurable success criteria keeps development focused and gives you clear go/no-go criteria for deployment. Include metrics for: quality (eval score on test set), latency (p95 TTFT and total), cost (per-query cost target), and reliability (uptime SLA). Post these on the project README so all contributors share the same target.

SUCCESS_METRICS = {
    # Quality
    'min_correctness_score': 4.0,       # out of 5, LLM-as-judge
    'min_retrieval_hit_rate': 0.85,     # top-5 chunk contains answer
    # Latency
    'p95_ttft_ms': 800,                 # time to first token
    'p95_total_latency_ms': 8000,       # full response
    # Cost
    'max_cost_per_query_usd': 0.05,     # $0.05 per Q&A
    # Reliability
    'target_uptime': 0.999,             # 99.9%
    'cache_hit_rate_target': 0.25,      # 25% queries served from cache
}

Documenting Architecture Trade-offs

Every architecture decision involves trade-offs. Document them explicitly in an Architecture Decision Record (ADR): what decision was made, what alternatives were considered, and why this option was chosen. For example: 'Chose pgvector over Pinecone because we already run PostgreSQL, reducing operational overhead. Trade-off: maximum scale is limited to ~10M vectors without sharding.' Future team members will thank you for this transparency.

# Architecture Decision Records (ADRs):
#
# ADR-001: Use pgvector for vector storage
# Decision: pgvector in existing PostgreSQL
# Alternatives: Pinecone, Weaviate, Qdrant
# Reason: No new infra, row-level security native, familiar operations
# Trade-offs: Limited to ~5M vectors before performance degrades
#
# ADR-002: GPT-4o as primary model
# Decision: gpt-4o for all user-facing queries
# Alternatives: gpt-4o-mini (cheaper), Claude (alternative)
# Reason: Highest quality for use case, Claude as fallback
# Trade-offs: $0.04/query vs $0.002 for gpt-4o-mini

Sketching the Deployment Topology

Define how your system will be deployed before writing any infrastructure code. Map each component to a deployment unit: the FastAPI backend as a Docker container, the ingestion pipeline as a separate worker service, pgvector as a managed PostgreSQL instance, Redis as a managed cache. Specify which components are stateless (can be scaled horizontally) versus stateful (require careful scaling strategies). A deployment diagram saves hours of rework later.

# Deployment topology:
#
# Internet -> Load Balancer (AWS ALB)
#                |
#            FastAPI API  (stateless, 2-4 containers, auto-scale)
#                |
#         +------+------+
#         |             |
#    pgvector        Redis Cache
#  (AWS RDS Postgres) (Elasticache)
#         |
#    Ingestion Worker  (separate container, manual trigger)
#         |
#    LangSmith (external SaaS, traces only)
#
# All containers in same VPC, no public access to DB/cache

Quick Check

Test your understanding of production AI system architecture design.

Lesson Recap

In this lesson you learned: component inventory and data-flow diagrams create a shared architectural vision before coding begins, API contracts defined upfront enable parallel development and make requirements explicit, and success metrics defined in advance keep the team aligned on quality, latency, cost, and reliability targets. Next up we implement the core RAG and agent features.

Часто задаваемые вопросы

Урок «Проектирование архитектуры для промышленной эксплуатации» бесплатный?

Да — полный текст урока «Проектирование архитектуры для промышленной эксплуатации» бесплатно доступен здесь в веб-версии. Чтобы практиковать его интерактивно (встроенный редактор кода и ИИ-репетитор 24/7) и разблокировать остальной курс AI Engineering Academy, подпишись на CoddyKit PRO. Курс AI Engineering Academy содержит 4 уроков всего.

Чему я научусь в уроке «Проектирование архитектуры для промышленной эксплуатации»?

Выберите итоговый проект, набросайте полную архитектуру, включая конвейер RAG, слой агента, кэширование, наблюдаемость и API, а затем задокументируйте проектные решения и компромиссы. Ты практикуешь AI Engineering Academy с помощью реального кода, который запускаешь прямо в браузере, и ИИ-репетитор 24/7 отвечает на твои вопросы во время урока.

Нужен ли мне опыт, чтобы начать AI Engineering Academy?

Предыдущий опыт не требуется. AI Engineering Academy на CoddyKit структурирован для всех уровней — от новичков до продвинутых, поэтому ты можешь начать отсюда или с самого начала и учиться в своем темпе. Это урок 1 из 4.

Сколько времени занимает урок «Проектирование архитектуры для промышленной эксплуатации»?

Большинство уроков CoddyKit занимают около 5–10 минут. Каждый из них компактный и интерактивный, поэтому ты постоянно делаешь прогресс и продолжаешь с того же места в веб-версии и приложении.

Можно ли писать и запускать код в этом уроке AI Engineering Academy?

Да. Каждый урок AI Engineering Academy включает встроенный редактор кода, поэтому ты пишешь и запускаешь реальный код прямо в браузере и получаешь моментальную обратную связь от AI — локальная установка не требуется.

Все уроки этого курса

Проектирование архитектуры для промышленной эксплуатации
Реализация основных возможностей RAG и агента
Повышение надёжности: безопасность, кэширование и устойчивость
Оценка, развёртывание и ретроспектива

← Назад к AI Engineering Academy