LLM Apps in Production (RAG + Vector DB + Caching) · 강의

오류 처리와 복원력 패턴

LLM 애플리케이션의 장애 내성을 높이도록 견고한 오류 처리, 재시도 메커니즘, 회로 차단기를 설계합니다.

오류 처리와 복원력 패턴은(는) CoddyKit의 무료 LLM Apps in Production (RAG + Vector DB + Caching) 강의입니다. 이것은 4개 중 3번째 강의입니다. 아래에서 전체 강의를 무료로 읽을 수 있으며, 내장 코드 에디터와 24/7 AI 튜터와 함께 브라우저에서 직접 실습할 수 있습니다. 이 강의는 LLM Apps in Production (RAG + Vector DB + Caching) 학습 경로의 일부이며, 진행 상황이 웹과 CoddyKit 앱에 동기화됩니다. LLM Apps in Production (RAG + Vector DB + Caching) 강의에는 총 4개의 강의가 포함되어 있습니다.

이 강의의 일부는 아직 번역되지 않았으며 영어로 표시됩니다.

Build Robust LLM Apps

LLM applications, especially those interacting with external APIs, need to be tough!

Resilience is about designing systems that can recover from failures gracefully, without crashing or providing a bad user experience.

In this lesson, we'll learn patterns to make your LLM apps more fault-tolerant.

Typical Failures

What kind of errors can an LLM application face?

API Rate Limits: Too many requests at once.
Network Issues: Temporary connection drops.
LLM Service Unavailability: The LLM provider is down.
Bad LLM Responses: Model returns invalid JSON or hallucinates.
Dependency Failures: Vector DB or other services fail.

Standard Try-Catch

The first line of defense is standard error handling using try-catch blocks. This prevents your entire application from crashing when an expected error occurs.

It allows you to log the error, inform the user, or attempt a fallback.

public class Main {
  public static void main(String[] args) {
    try {
      // Simulate an LLM API call that might fail
      callLlmApi();
      System.out.println("API call successful.");
    } catch (Exception e) {
      System.out.println("Error: " + e.getMessage());
      // Log the error, notify user, etc.
    }
  }

  public static void callLlmApi() throws Exception {
    // In a real app, this would make an actual API call
    if (Math.random() < 0.5) { // 50% chance of failure
      throw new RuntimeException("LLM service unavailable.");
    }
  }
}

Why Just Catching Isn't Enough

Some errors are transient, meaning they're temporary and might resolve if you just try again. Think of a brief network glitch or a momentary rate limit.

A simple try-catch just fails immediately. For transient errors, a retry mechanism can significantly improve reliability without user intervention.

Simple Retry Logic

We can implement a basic retry loop. If an error occurs, we wait a bit and try again, up to a maximum number of attempts.

public class Main {
  public static void main(String[] args) {
    int maxRetries = 3;
    int currentRetry = 0;
    boolean success = false;

    while (currentRetry < maxRetries && !success) {
      try {
        System.out.println("Attempt " + (currentRetry + 1));
        callLlmApi();
        System.out.println("API call successful.");
        success = true;
      } catch (Exception e) {
        System.out.println("Error: " + e.getMessage());
        currentRetry++;
        if (currentRetry < maxRetries) {
          System.out.println("Retrying in 1 second...");
          try { Thread.sleep(1000); } catch (InterruptedException ie) {}
        }
      }
    }
    if (!success) {
      System.out.println("All retries failed.");
    }
  }

  public static void callLlmApi() throws Exception {
    // Simulate an LLM API call with 70% chance of failure
    if (Math.random() < 0.7) {
      throw new RuntimeException("Transient network error.");
    }
  }
}

Smart Retries: Exponential Backoff

Constant retry delays can overwhelm a struggling service. Exponential backoff is a strategy where the delay between retries increases exponentially.

This gives the remote service more time to recover and prevents your app from hammering it with requests.

Initial delay: 1s
Second delay: 2s
Third delay: 4s
And so on...

Circuit Breaker Pattern

What if a service is truly down, not just experiencing transient errors? Retrying repeatedly only wastes resources and delays failure detection.

The Circuit Breaker pattern prevents an application from repeatedly trying to invoke a service that is likely to fail, saving resources and allowing the service time to recover.

Circuit Breaker States

A circuit breaker has three main states:

Closed: Operations proceed normally. If errors exceed a threshold, it trips to Open.
Open: All requests fail immediately without trying the service. After a timeout, it transitions to Half-Open.
Half-Open: A limited number of requests are allowed to pass through to test if the service has recovered. If successful, it goes back to Closed; otherwise, back to Open.

Preventing Hung Requests with Timeouts

LLM API calls can sometimes hang indefinitely, waiting for a response that never comes. This can exhaust resources and degrade user experience.

Always configure timeouts for your API calls. This sets a maximum duration your application will wait for a response before giving up and throwing an error.

import java.util.concurrent.TimeUnit;

public class Main {
  public static void main(String[] args) {
    long startTime = System.nanoTime();
    long timeoutMillis = 2000; // 2 seconds timeout

    try {
      System.out.println("Calling LLM API with a timeout...");
      callLlmApiWithTimeout(timeoutMillis);
      System.out.println("API call completed successfully.");
    } catch (Exception e) {
      System.out.println("API call failed: " + e.getMessage());
    }

    long endTime = System.nanoTime();
    long duration = TimeUnit.NANOSECONDS.toMillis(endTime - startTime);
    System.out.println("Total duration: " + duration + "ms");
  }

  public static void callLlmApiWithTimeout(long timeoutMillis) throws Exception {
    // Simulate a long-running/hung API call
    long processingTime = 2500; // 2.5 seconds
    if (processingTime > timeoutMillis) {
      throw new RuntimeException("Operation timed out after " + timeoutMillis + "ms");
    }
    try {
      Thread.sleep(processingTime);
    } catch (InterruptedException e) {
      Thread.currentThread().interrupt();
      throw new RuntimeException("API call interrupted.", e);
    }
  }
}

Resilience Check

When should you use a Circuit Breaker pattern instead of just a Retry mechanism?

Recap: Building Resilient LLM Apps

We've covered key patterns for making your LLM applications fault-tolerant:

Basic Error Handling: Using try-catch for immediate failure management.
Retry Mechanisms: For handling transient errors, often with exponential backoff.
Circuit Breakers: To prevent overwhelming consistently failing services.
Timeouts: Essential for preventing hung API calls and resource exhaustion.

These patterns are crucial for robust production LLM systems!

자주 묻는 질문

“오류 처리와 복원력 패턴” 강의는 무료인가요?

네 — “오류 처리와 복원력 패턴” 전체 내용을 이 웹사이트에서 무료로 읽을 수 있습니다. 인터랙티브하게 실습하려면(내장 코드 에디터와 24/7 AI 튜터), CoddyKit PRO로 업그레이드하면 LLM Apps in Production (RAG + Vector DB + Caching) 강의 전체를 잠금 해제할 수 있습니다. LLM Apps in Production (RAG + Vector DB + Caching) 강의에는 총 4개의 강의가 포함되어 있습니다.

“오류 처리와 복원력 패턴”에서 뭘 배우나요?

LLM 애플리케이션의 장애 내성을 높이도록 견고한 오류 처리, 재시도 메커니즘, 회로 차단기를 설계합니다. 브라우저에서 직접 실행하는 실습 코드로 LLM Apps in Production (RAG + Vector DB + Caching)을(를) 배우며, 24/7 AI 튜터가 강의를 진행하면서 질문에 답변해줍니다.

LLM Apps in Production (RAG + Vector DB + Caching)을(를) 시작하는 데 경험이 필요한가요?

사전 경험은 필요하지 않습니다. CoddyKit의 LLM Apps in Production (RAG + Vector DB + Caching)은(는) 초급자부터 고급 학습자까지를 위해 구성되어 있으므로, 여기서 시작하거나 처음부터 시작할 수 있으며 자신의 속도대로 진행할 수 있습니다. 이것은 4개 중 3번째 강의입니다.

“오류 처리와 복원력 패턴” 강의는 얼마나 걸리나요?

대부분의 CoddyKit 강의는 약 5~10분이 소요됩니다. 각 강의는 간결하고 인터랙티브하여 꾸준한 진행이 가능하며, 웹과 앱에서 중단한 부분부터 바로 시작할 수 있습니다.

이 LLM Apps in Production (RAG + Vector DB + Caching) 강의에서 코드를 작성하고 실행할 수 있나요?

네. 모든 LLM Apps in Production (RAG + Vector DB + Caching) 강의에는 내장 코드 에디터가 포함되어 있으므로, 브라우저에서 바로 실제 코드를 작성하고 실행한 후 즉시 AI 피드백을 받을 수 있습니다 — 로컬 설정이 필요 없습니다.

이 강의의 모든 강의

← LLM Apps in Production (RAG + Vector DB + Caching)(으)로 돌아가기