Prompt Engineering & LLM Optimization for Developers · 강의

휴먼 인 더 루프 피드백 시스템

LLM의 성능을 지속적으로 개선하고 오류를 수정할 수 있도록 사람의 피드백을 통합하는 시스템을 설계하고 구현합니다.

휴먼 인 더 루프 피드백 시스템은(는) CoddyKit의 무료 Prompt Engineering & LLM Optimization for Developers 강의입니다. 이것은 4개 중 2번째 강의입니다. 아래에서 전체 강의를 무료로 읽을 수 있으며, 내장 코드 에디터와 24/7 AI 튜터와 함께 브라우저에서 직접 실습할 수 있습니다. 이 강의는 Prompt Engineering & LLM Optimization for Developers 학습 경로의 일부이며, 진행 상황이 웹과 CoddyKit 앱에 동기화됩니다. Prompt Engineering & LLM Optimization for Developers 강의에는 총 4개의 강의가 포함되어 있습니다.

이 강의의 일부는 아직 번역되지 않았으며 영어로 표시됩니다.

What is Human-in-the-Loop?

When working with Large Language Models (LLMs), sometimes their outputs aren't quite right. This is where Human-in-the-Loop (HITL) systems come in.

HITL means humans actively review, correct, or provide feedback on an LLM's output. This feedback then helps the model learn and improve over time, making it smarter and more reliable.

Why Humans Help LLMs

Even powerful LLMs have limitations. They can:

Hallucinate: Make up facts.
Be biased: Reflect biases from their training data.
Lack up-to-date info: Not know about recent events.
Misunderstand complex requests: Struggle with nuanced instructions.

Human oversight helps catch and fix these issues, ensuring higher quality and safer outputs.

How HITL Systems Work

A typical HITL system involves a few key parts:

LLM Generation: The model creates an output.
Human Review: A person evaluates the output.
Feedback Collection: The human's judgment is captured.
Model Improvement: This feedback is used to refine the LLM, either by adjusting prompts or fine-tuning the model itself.

It's a continuous cycle of creation, evaluation, and learning.

Two Ways to Give Feedback

Human feedback can be broadly categorized into two types:

Explicit Feedback: Direct ratings or corrections from users or annotators. Think "thumbs up/down" or editing a generated text.
Implicit Feedback: Inferred from user behavior without direct input. For example, how long a user spends on a response or if they click on certain links.

Both types are valuable for different reasons!

Getting Direct Feedback

Explicit feedback is clear and intentional. Common ways to collect it include:

Thumbs Up/Down: Simple binary feedback on response quality.
Star Ratings: A more granular scale (e.g., 1-5 stars).
Direct Edits: Users correcting or rewriting parts of the LLM's output.
Annotator Labels: Expert human annotators tagging specific issues like factual errors or toxicity.

This data directly tells you what worked and what didn't.

Learning from User Behavior

Implicit feedback is less direct but still powerful. It's about observing how users interact with LLM outputs:

Session Duration: Users spending more time on a response might indicate engagement.
Click-Through Rates: If an LLM suggests links, clicks show relevance.
Search Refinements: If a user immediately rephrases their query, the initial response might have been poor.
Copying Text: Users copying output might find it useful.

Analyzing these behaviors can reveal subtle preferences and issues.

Capturing Feedback in Code

Let's imagine a basic way to capture explicit feedback (like a "thumbs up" or "thumbs down") after an LLM generates a response. This simple Python code shows how you might structure the collection of feedback.

It doesn't call an LLM, but focuses on the feedback part.

def collect_feedback(llm_output: str, user_rating: str):
    """
    Simulates collecting feedback for an LLM output.
    In a real system, this would store to a database.
    """
    feedback_data = {
        "output": llm_output,
        "rating": user_rating, # e.g., "good", "bad", "neutral"
        "timestamp": "2023-10-27T10:30:00Z" # For simplicity
    }
    print(f"Feedback recorded: {feedback_data}")
    # In a real app, you'd save this to a database or log file.

if __name__ == "__main__":
    generated_text = "The capital of France is Paris."
    print(f"LLM output: '{generated_text}'")

    # Simulate user giving feedback
    user_choice = input("Was this output good or bad? (good/bad): ")
    collect_feedback(generated_text, user_choice)

    generated_text_2 = "The sun revolves around the Earth."
    print(f"\nLLM output: '{generated_text_2}'")
    user_choice_2 = input("Was this output good or bad? (good/bad): ")
    collect_feedback(generated_text_2, user_choice_2)

Turning Feedback into Growth

Once feedback is collected, it's not just stored; it's used to make the LLM better:

Prompt Refinement: If many users rate an output as "bad" for a specific prompt, you can adjust the prompt to be clearer or more specific.
Data for Fine-tuning: High-quality human-corrected data can be used to fine-tune the LLM, teaching it directly from good examples.
Reinforcement Learning from Human Feedback (RLHF): Feedback can be used to train a reward model, which then guides the LLM to generate preferred outputs.

This closes the loop and improves performance.

Hurdles in Human-in-the-Loop

While HITL is powerful, it comes with challenges:

Scalability: Manually reviewing every LLM output for a large application is impossible.
Cost: Human annotators or user feedback mechanisms can be expensive to build and maintain.
Bias in Feedback: Human reviewers can also introduce their own biases, which might then be reinforced in the model.
Subjectivity: What one person considers a "good" response, another might not.

Careful design is needed to overcome these.

Understanding HITL

You've learned about Human-in-the-Loop systems. Let's test your understanding!

Recap: HITL for Better LLMs

In this lesson, we explored Human-in-the-Loop (HITL) systems. We learned that HITL integrates human judgment to continuously improve LLM performance by addressing limitations like hallucinations and biases.

You now understand the components of a HITL system, the difference between explicit and implicit feedback, and how this feedback drives model improvement. You also saw a simple code example for collecting feedback and are aware of the challenges involved.

Keep exploring how to build robust LLM applications!

자주 묻는 질문

“휴먼 인 더 루프 피드백 시스템” 강의는 무료인가요?

네 — “휴먼 인 더 루프 피드백 시스템” 전체 내용을 이 웹사이트에서 무료로 읽을 수 있습니다. 인터랙티브하게 실습하려면(내장 코드 에디터와 24/7 AI 튜터), CoddyKit PRO로 업그레이드하면 Prompt Engineering & LLM Optimization for Developers 강의 전체를 잠금 해제할 수 있습니다. Prompt Engineering & LLM Optimization for Developers 강의에는 총 4개의 강의가 포함되어 있습니다.

“휴먼 인 더 루프 피드백 시스템”에서 뭘 배우나요?

LLM의 성능을 지속적으로 개선하고 오류를 수정할 수 있도록 사람의 피드백을 통합하는 시스템을 설계하고 구현합니다. 브라우저에서 직접 실행하는 실습 코드로 Prompt Engineering & LLM Optimization for Developers을(를) 배우며, 24/7 AI 튜터가 강의를 진행하면서 질문에 답변해줍니다.

Prompt Engineering & LLM Optimization for Developers을(를) 시작하는 데 경험이 필요한가요?

사전 경험은 필요하지 않습니다. CoddyKit의 Prompt Engineering & LLM Optimization for Developers은(는) 초급자부터 고급 학습자까지를 위해 구성되어 있으므로, 여기서 시작하거나 처음부터 시작할 수 있으며 자신의 속도대로 진행할 수 있습니다. 이것은 4개 중 2번째 강의입니다.

“휴먼 인 더 루프 피드백 시스템” 강의는 얼마나 걸리나요?

대부분의 CoddyKit 강의는 약 5~10분이 소요됩니다. 각 강의는 간결하고 인터랙티브하여 꾸준한 진행이 가능하며, 웹과 앱에서 중단한 부분부터 바로 시작할 수 있습니다.

이 Prompt Engineering & LLM Optimization for Developers 강의에서 코드를 작성하고 실행할 수 있나요?

네. 모든 Prompt Engineering & LLM Optimization for Developers 강의에는 내장 코드 에디터가 포함되어 있으므로, 브라우저에서 바로 실제 코드를 작성하고 실행한 후 즉시 AI 피드백을 받을 수 있습니다 — 로컬 설정이 필요 없습니다.

이 강의의 모든 강의

← Prompt Engineering & LLM Optimization for Developers(으)로 돌아가기