Prompt Engineering & LLM Optimization for Developers · 课时

人在回路中的反馈系统

设计并实现融入人工反馈的系统，以持续提升 LLM 性能并纠正错误。

第 2 / 4 课11 个步骤

人在回路中的反馈系统 是 CoddyKit 上的免费 Prompt Engineering & LLM Optimization for Developers 课时。这是第 2 节课，共 4 节。你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。这是 Prompt Engineering & LLM Optimization for Developers 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 Prompt Engineering & LLM Optimization for Developers 课程共包含 4 节课。

本课时的部分内容尚未翻译，以英文显示。

What is Human-in-the-Loop?

When working with Large Language Models (LLMs), sometimes their outputs aren't quite right. This is where Human-in-the-Loop (HITL) systems come in.

HITL means humans actively review, correct, or provide feedback on an LLM's output. This feedback then helps the model learn and improve over time, making it smarter and more reliable.

Why Humans Help LLMs

Even powerful LLMs have limitations. They can:

Hallucinate: Make up facts.
Be biased: Reflect biases from their training data.
Lack up-to-date info: Not know about recent events.
Misunderstand complex requests: Struggle with nuanced instructions.

Human oversight helps catch and fix these issues, ensuring higher quality and safer outputs.

How HITL Systems Work

A typical HITL system involves a few key parts:

LLM Generation: The model creates an output.
Human Review: A person evaluates the output.
Feedback Collection: The human's judgment is captured.
Model Improvement: This feedback is used to refine the LLM, either by adjusting prompts or fine-tuning the model itself.

It's a continuous cycle of creation, evaluation, and learning.

Two Ways to Give Feedback

Human feedback can be broadly categorized into two types:

Explicit Feedback: Direct ratings or corrections from users or annotators. Think "thumbs up/down" or editing a generated text.
Implicit Feedback: Inferred from user behavior without direct input. For example, how long a user spends on a response or if they click on certain links.

Both types are valuable for different reasons!

Getting Direct Feedback

Explicit feedback is clear and intentional. Common ways to collect it include:

Thumbs Up/Down: Simple binary feedback on response quality.
Star Ratings: A more granular scale (e.g., 1-5 stars).
Direct Edits: Users correcting or rewriting parts of the LLM's output.
Annotator Labels: Expert human annotators tagging specific issues like factual errors or toxicity.

This data directly tells you what worked and what didn't.

Learning from User Behavior

Implicit feedback is less direct but still powerful. It's about observing how users interact with LLM outputs:

Session Duration: Users spending more time on a response might indicate engagement.
Click-Through Rates: If an LLM suggests links, clicks show relevance.
Search Refinements: If a user immediately rephrases their query, the initial response might have been poor.
Copying Text: Users copying output might find it useful.

Analyzing these behaviors can reveal subtle preferences and issues.

Capturing Feedback in Code

Let's imagine a basic way to capture explicit feedback (like a "thumbs up" or "thumbs down") after an LLM generates a response. This simple Python code shows how you might structure the collection of feedback.

It doesn't call an LLM, but focuses on the feedback part.

def collect_feedback(llm_output: str, user_rating: str):
    """
    Simulates collecting feedback for an LLM output.
    In a real system, this would store to a database.
    """
    feedback_data = {
        "output": llm_output,
        "rating": user_rating, # e.g., "good", "bad", "neutral"
        "timestamp": "2023-10-27T10:30:00Z" # For simplicity
    }
    print(f"Feedback recorded: {feedback_data}")
    # In a real app, you'd save this to a database or log file.

if __name__ == "__main__":
    generated_text = "The capital of France is Paris."
    print(f"LLM output: '{generated_text}'")

    # Simulate user giving feedback
    user_choice = input("Was this output good or bad? (good/bad): ")
    collect_feedback(generated_text, user_choice)

    generated_text_2 = "The sun revolves around the Earth."
    print(f"\nLLM output: '{generated_text_2}'")
    user_choice_2 = input("Was this output good or bad? (good/bad): ")
    collect_feedback(generated_text_2, user_choice_2)

Turning Feedback into Growth

Once feedback is collected, it's not just stored; it's used to make the LLM better:

Prompt Refinement: If many users rate an output as "bad" for a specific prompt, you can adjust the prompt to be clearer or more specific.
Data for Fine-tuning: High-quality human-corrected data can be used to fine-tune the LLM, teaching it directly from good examples.
Reinforcement Learning from Human Feedback (RLHF): Feedback can be used to train a reward model, which then guides the LLM to generate preferred outputs.

This closes the loop and improves performance.

Hurdles in Human-in-the-Loop

While HITL is powerful, it comes with challenges:

Scalability: Manually reviewing every LLM output for a large application is impossible.
Cost: Human annotators or user feedback mechanisms can be expensive to build and maintain.
Bias in Feedback: Human reviewers can also introduce their own biases, which might then be reinforced in the model.
Subjectivity: What one person considers a "good" response, another might not.

Careful design is needed to overcome these.

Understanding HITL

You've learned about Human-in-the-Loop systems. Let's test your understanding!

在交互式课程中回答

Recap: HITL for Better LLMs

In this lesson, we explored Human-in-the-Loop (HITL) systems. We learned that HITL integrates human judgment to continuously improve LLM performance by addressing limitations like hallucinations and biases.

You now understand the components of a HITL system, the difference between explicit and implicit feedback, and how this feedback drives model improvement. You also saw a simple code example for collecting feedback and are aware of the challenges involved.

Keep exploring how to build robust LLM applications!

免费开始

用 AI 导师学习 Prompt Engineering & LLM Optimization for Developers — 免费

在浏览器中编写并运行真实代码，获得全天候 AI 导师的即时帮助，并在网页或应用中继续学习。

课程: 12
课程: 48

免费开始学习查看完整课程

常见问题解答

「人在回路中的反馈系统」课时是免费的吗？

是的 — 「人在回路中的反馈系统」的完整文本可在网页上免费阅读。要进行交互式练习（内置代码编辑器和全天候 AI 导师）并解锁 Prompt Engineering & LLM Optimization for Developers 课程的其余内容，请升级到 CoddyKit PRO。 Prompt Engineering & LLM Optimization for Developers 课程共包含 4 节课。

「人在回路中的反馈系统」这节课中我会学到什么？

设计并实现融入人工反馈的系统，以持续提升 LLM 性能并纠正错误。你通过在浏览器中直接运行的动手代码来练习 Prompt Engineering & LLM Optimization for Developers，全天候 AI 导师会在你学习这节课的过程中回答你的问题。

学习 Prompt Engineering & LLM Optimization for Developers 需要有经验吗？

无需任何先前经验。CoddyKit 上的 Prompt Engineering & LLM Optimization for Developers 课程适合初学者到高级学习者，你可以从这里开始或从头开始，按照自己的节奏学习。这是第 2 节课，共 4 节。

「人在回路中的反馈系统」课时需要多长时间？

大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动，所以你能稳步进步，并在网页和应用中从离开的地方继续。

我能在这节 Prompt Engineering & LLM Optimization for Developers 课中编写并运行代码吗？

能。每节 Prompt Engineering & LLM Optimization for Developers 课都包含内置代码编辑器，你可以在浏览器中直接编写并运行真实代码，并获得即时 AI 反馈 — 无需本地设置。

此课程中的所有课时

← 返回 Prompt Engineering & LLM Optimization for Developers