LLM Apps in Production (RAG + Vector DB + Caching) · Pelajaran

Penanganan Galat dan Pola Ketahanan

Rancang penanganan galat, mekanisme percobaan ulang, dan pemutus sirkuit yang tangguh agar aplikasi LLM Anda lebih tahan terhadap kegagalan.

Pelajaran 3 dari 411 langkah

Penanganan Galat dan Pola Ketahanan adalah pelajaran LLM Apps in Production (RAG + Vector DB + Caching) gratis di CoddyKit. Ini adalah pelajaran 3 dari 4. Kamu bisa membaca pelajaran lengkapnya di bawah secara gratis — lalu praktikkan langsung di browser dengan editor kode bawaan dan tutor AI 24/7. Ini adalah bagian dari jalur belajar LLM Apps in Production (RAG + Vector DB + Caching), dan progresmu tersinkronisasi di web dan aplikasi CoddyKit. Kursus LLM Apps in Production (RAG + Vector DB + Caching) mencakup 4 pelajaran total.

Bagian dari pelajaran ini belum diterjemahkan dan ditampilkan dalam bahasa Inggris.

Build Robust LLM Apps

LLM applications, especially those interacting with external APIs, need to be tough!

Resilience is about designing systems that can recover from failures gracefully, without crashing or providing a bad user experience.

In this lesson, we'll learn patterns to make your LLM apps more fault-tolerant.

Typical Failures

What kind of errors can an LLM application face?

API Rate Limits: Too many requests at once.
Network Issues: Temporary connection drops.
LLM Service Unavailability: The LLM provider is down.
Bad LLM Responses: Model returns invalid JSON or hallucinates.
Dependency Failures: Vector DB or other services fail.

Standard Try-Catch

The first line of defense is standard error handling using try-catch blocks. This prevents your entire application from crashing when an expected error occurs.

It allows you to log the error, inform the user, or attempt a fallback.

public class Main {
  public static void main(String[] args) {
    try {
      // Simulate an LLM API call that might fail
      callLlmApi();
      System.out.println("API call successful.");
    } catch (Exception e) {
      System.out.println("Error: " + e.getMessage());
      // Log the error, notify user, etc.
    }
  }

  public static void callLlmApi() throws Exception {
    // In a real app, this would make an actual API call
    if (Math.random() < 0.5) { // 50% chance of failure
      throw new RuntimeException("LLM service unavailable.");
    }
  }
}

Why Just Catching Isn't Enough

Some errors are transient, meaning they're temporary and might resolve if you just try again. Think of a brief network glitch or a momentary rate limit.

A simple try-catch just fails immediately. For transient errors, a retry mechanism can significantly improve reliability without user intervention.

Simple Retry Logic

We can implement a basic retry loop. If an error occurs, we wait a bit and try again, up to a maximum number of attempts.

public class Main {
  public static void main(String[] args) {
    int maxRetries = 3;
    int currentRetry = 0;
    boolean success = false;

    while (currentRetry < maxRetries && !success) {
      try {
        System.out.println("Attempt " + (currentRetry + 1));
        callLlmApi();
        System.out.println("API call successful.");
        success = true;
      } catch (Exception e) {
        System.out.println("Error: " + e.getMessage());
        currentRetry++;
        if (currentRetry < maxRetries) {
          System.out.println("Retrying in 1 second...");
          try { Thread.sleep(1000); } catch (InterruptedException ie) {}
        }
      }
    }
    if (!success) {
      System.out.println("All retries failed.");
    }
  }

  public static void callLlmApi() throws Exception {
    // Simulate an LLM API call with 70% chance of failure
    if (Math.random() < 0.7) {
      throw new RuntimeException("Transient network error.");
    }
  }
}

Smart Retries: Exponential Backoff

Constant retry delays can overwhelm a struggling service. Exponential backoff is a strategy where the delay between retries increases exponentially.

This gives the remote service more time to recover and prevents your app from hammering it with requests.

Initial delay: 1s
Second delay: 2s
Third delay: 4s
And so on...

Circuit Breaker Pattern

What if a service is truly down, not just experiencing transient errors? Retrying repeatedly only wastes resources and delays failure detection.

The Circuit Breaker pattern prevents an application from repeatedly trying to invoke a service that is likely to fail, saving resources and allowing the service time to recover.

Circuit Breaker States

A circuit breaker has three main states:

Closed: Operations proceed normally. If errors exceed a threshold, it trips to Open.
Open: All requests fail immediately without trying the service. After a timeout, it transitions to Half-Open.
Half-Open: A limited number of requests are allowed to pass through to test if the service has recovered. If successful, it goes back to Closed; otherwise, back to Open.

Preventing Hung Requests with Timeouts

LLM API calls can sometimes hang indefinitely, waiting for a response that never comes. This can exhaust resources and degrade user experience.

Always configure timeouts for your API calls. This sets a maximum duration your application will wait for a response before giving up and throwing an error.

import java.util.concurrent.TimeUnit;

public class Main {
  public static void main(String[] args) {
    long startTime = System.nanoTime();
    long timeoutMillis = 2000; // 2 seconds timeout

    try {
      System.out.println("Calling LLM API with a timeout...");
      callLlmApiWithTimeout(timeoutMillis);
      System.out.println("API call completed successfully.");
    } catch (Exception e) {
      System.out.println("API call failed: " + e.getMessage());
    }

    long endTime = System.nanoTime();
    long duration = TimeUnit.NANOSECONDS.toMillis(endTime - startTime);
    System.out.println("Total duration: " + duration + "ms");
  }

  public static void callLlmApiWithTimeout(long timeoutMillis) throws Exception {
    // Simulate a long-running/hung API call
    long processingTime = 2500; // 2.5 seconds
    if (processingTime > timeoutMillis) {
      throw new RuntimeException("Operation timed out after " + timeoutMillis + "ms");
    }
    try {
      Thread.sleep(processingTime);
    } catch (InterruptedException e) {
      Thread.currentThread().interrupt();
      throw new RuntimeException("API call interrupted.", e);
    }
  }
}

Resilience Check

When should you use a Circuit Breaker pattern instead of just a Retry mechanism?

Jawab di pelajaran interaktif

Recap: Building Resilient LLM Apps

We've covered key patterns for making your LLM applications fault-tolerant:

Basic Error Handling: Using try-catch for immediate failure management.
Retry Mechanisms: For handling transient errors, often with exponential backoff.
Circuit Breakers: To prevent overwhelming consistently failing services.
Timeouts: Essential for preventing hung API calls and resource exhaustion.

These patterns are crucial for robust production LLM systems!

Gratis untuk memulai

Belajar LLM Apps in Production (RAG + Vector DB + Caching) dengan tutor AI — gratis

Tulis dan jalankan kode asli di browser kamu, dapatkan bantuan instan dari tutor AI 24/7, dan lanjutkan di mana kamu tinggalkan di web atau aplikasi.

Kursus: 12
Pelajaran: 48

Mulai belajar gratis Lihat kursus lengkapnya

Pertanyaan yang Sering Diajukan

Apakah pelajaran “Penanganan Galat dan Pola Ketahanan” gratis?

Ya — teks lengkap “Penanganan Galat dan Pola Ketahanan” gratis dibaca di sini di web. Untuk praktiknya secara interaktif (editor kode bawaan dan tutor AI 24/7) dan buka sisa kursus LLM Apps in Production (RAG + Vector DB + Caching), upgrade ke CoddyKit PRO. Kursus LLM Apps in Production (RAG + Vector DB + Caching) mencakup 4 pelajaran total.

Apa yang akan aku pelajari di “Penanganan Galat dan Pola Ketahanan”?

Rancang penanganan galat, mekanisme percobaan ulang, dan pemutus sirkuit yang tangguh agar aplikasi LLM Anda lebih tahan terhadap kegagalan. Kamu berlatih LLM Apps in Production (RAG + Vector DB + Caching) dengan kode praktik yang langsung kamu jalankan di browser, dan tutor AI 24/7 menjawab pertanyaanmu saat kamu mengerjakan pelajaran ini.

Apakah aku perlu pengalaman untuk memulai LLM Apps in Production (RAG + Vector DB + Caching)?

Tidak diperlukan pengalaman sebelumnya. LLM Apps in Production (RAG + Vector DB + Caching) di CoddyKit dirancang untuk pemula hingga pelajar tingkat lanjut, jadi kamu bisa memulai di sini atau dari awal dan belajar sesuai kecepatan kamu sendiri. Ini adalah pelajaran 3 dari 4.

Berapa lama pelajaran “Penanganan Galat dan Pola Ketahanan” memakan waktu?

Sebagian besar pelajaran CoddyKit memakan waktu sekitar 5–10 menit. Setiap pelajaran ringkas dan interaktif, jadi kamu membuat kemajuan stabil dan melanjutkan dari tempat kamu tinggalkan di web dan aplikasi.

Bisakah aku menulis dan menjalankan kode dalam pelajaran LLM Apps in Production (RAG + Vector DB + Caching) ini?

Ya. Setiap pelajaran LLM Apps in Production (RAG + Vector DB + Caching) menyertakan editor kode bawaan, jadi kamu menulis dan menjalankan kode nyata langsung di browser dan mendapatkan umpan balik AI instan — tidak diperlukan penyiapan lokal.

Semua pelajaran dalam kursus ini

Mengamankan Kunci API LLM dan Data Sensitif
Pembatasan Laju dan Pencegahan Penyalahgunaan
Penanganan Galat dan Pola Ketahanan
Melindungi dari Injeksi Prompt

← Kembali ke LLM Apps in Production (RAG + Vector DB + Caching)