视觉问答
针对图像内容、数量和属性提出具体问题
视觉问答 是 CoddyKit 上的免费 AI Prompt Engineering 课时。 这是第 2 节课,共 4 节。 你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。 这是 AI Prompt Engineering 学习路径的一部分,你的进度在网页和 CoddyKit 应用中同步。 AI Prompt Engineering 课程共包含 4 节课。
视觉问答
视觉问答(VQA)是回答关于图片的自然语言问题的任务。与描述所有内容的图片描述不同,VQA 会让模型专注于回答特定问题。
VQA 提示词精确、直接,通常需要对视觉内容进行计数、识别、比较或推理。提示词的质量决定了您得到的是精确且有用的答案,还是模糊笼统的回答。
基本 VQA 提示词结构
VQA 提示词会将一张图片与一个具体问题配对。关键在于让问题足够精确,从而生成直接且可用的答案:
import anthropic, base64
client = anthropic.Anthropic(api_key='YOUR_API_KEY')
def ask_about_image(image_path, question, answer_format='Direct answer. No extra explanation.'):
with open(image_path, 'rb') as f:
img_b64 = base64.standard_b64encode(f.read()).decode('utf-8')
prompt = f'{question}\n\n{answer_format}'
r = client.messages.create(
model='claude-opus-4-5', max_tokens=150,
messages=[{'role': 'user', 'content': [
{'type': 'image', 'source': {'type': 'base64', 'media_type': 'image/jpeg', 'data': img_b64}},
{'type': 'text', 'text': prompt}
]}]
)
return r.content[0].text
# Example VQA calls (replace image.jpg with actual image)
print('VQA function defined. Ready for image questions.')计数问题
计数是常见的 VQA 任务。与模糊的提示词相比,精确的计数提示词可以生成更准确的结果:
# Vague (bad):
vague_prompt = 'How many people are there?'
# Precise (good): specifies what counts and what does not
counting_prompt = '''
How many people are visible in this image?
Count only: people whose faces OR bodies are at least 50% visible.
Do NOT count: people who are heavily cropped, cut off at the edge, or only partially visible.
Return a single number.
'''
# Even more precise: handles partial visibility explicitly
precise_count = '''
Count the number of distinct individuals visible in this image.
If a person is partially obscured, count them if more than half their body is visible.
Return JSON: {"count": integer, "partially_visible": integer, "notes": "string or null"}
'''
print('Counting prompts: vague vs precise.')
print('Precise prompts define edge cases explicitly.')品牌与标志识别
识别图片中的品牌标志是常见的产品分析任务。提示词必须明确指定要查找的内容以及返回的格式:
logo_prompt = '''
Identify all visible brand logos, company names, and product labels in this image.
For each, note:
- Brand/company name
- Where it appears in the image (top-left, center, on a product, etc.)
- Confidence: high (clearly legible) | medium (partially visible) | low (partially obscured)
Return JSON: {"brands": [{"name": str, "location": str, "confidence": str}]}
If no logos are visible, return: {"brands": []}
'''
import anthropic, base64, json
client = anthropic.Anthropic(api_key='YOUR_API_KEY')
def identify_brands(image_path):
with open(image_path, 'rb') as f:
img_b64 = base64.standard_b64encode(f.read()).decode('utf-8')
r = client.messages.create(
model='claude-opus-4-5', max_tokens=200,
messages=[{'role': 'user', 'content': [
{'type': 'image', 'source': {'type': 'base64', 'media_type': 'image/jpeg', 'data': img_b64}},
{'type': 'text', 'text': logo_prompt}
]}]
)
return json.loads(r.content[0].text)
print('Brand identification function defined.')情绪与表情识别
识别图片中的情绪表达,需要谨慎设计提示词,并承认结果可能存在不确定性:
emotion_prompt = '''
Describe the emotional expression of the person in this image.
Assess:
- Primary emotion: (happy, sad, angry, surprised, fearful, disgusted, neutral, or other)
- Intensity: (low, moderate, high)
- Confidence: (high if expression is clear, medium if subtle, low if face is obscured or turned away)
- Evidence: which specific facial features support your assessment
Return JSON:
{
"primary_emotion": str,
"intensity": str,
"confidence": str,
"evidence": str,
"secondary_emotion": str or null
}
If no person or face is clearly visible, return: {"primary_emotion": null, "confidence": "none", "reason": str}
'''
print(emotion_prompt)空间关系问题
询问物体相互位置的问题,需要在提示词中明确使用空间关系词汇:
spatial_prompt = '''
Answer questions about the spatial relationships of objects in this image.
Use these spatial terms consistently:
- Position in frame: top-left, top-center, top-right, middle-left, center, middle-right, bottom-left, bottom-center, bottom-right
- Relative position: in front of, behind, to the left of, to the right of, above, below, overlapping
- Distance: in the foreground, in the midground, in the background
Question: {question}
Answer in one or two sentences using the spatial vocabulary above.
'''
# Example questions:
questions = [
'Where is the red cup relative to the laptop?',
'Is the plant in the foreground or background?',
'What object is to the left of the person?'
]
for q in questions:
print(spatial_prompt.replace('{question}', q)[:200])
print('---')质量与状态评估
评估图片中物体的质量或状态——适用于产品检查、房地产评估和质量控制:
condition_prompt = '''
Assess the condition of the main subject in this image.
Rate on these dimensions (1-5 scale, 5=excellent):
- Physical condition: (1=heavily damaged, 5=like new)
- Cleanliness: (1=very dirty, 5=spotless)
- Completeness: (1=major parts missing, 5=fully intact)
For each rating, provide one-sentence evidence.
Return JSON:
{
"physical_condition": {"score": int, "evidence": str},
"cleanliness": {"score": int, "evidence": str},
"completeness": {"score": int, "evidence": str},
"overall_grade": "excellent|good|fair|poor",
"recommendation": str
}
'''
print('Condition assessment prompt defined.')
print('Useful for: product inspection, real estate, equipment maintenance.')是/否 VQA 问题
二元是/否问题需要使用适当的提示词,避免模型在您需要简单布尔值时给出含糊其辞的自然语言回答:
def yes_no_question(image_path, question):
with open(image_path, 'rb') as f:
img_b64 = base64.standard_b64encode(f.read()).decode('utf-8')
prompt = f'''
Answer this yes/no question about the image.
Return JSON: {{"answer": "yes|no", "confidence": "high|medium|low", "reason": str}}
Do NOT answer with maybe, possibly, or a hedged statement.
If you genuinely cannot determine the answer, return {{"answer": "unclear", "confidence": "low", "reason": str}}
Question: {question}
'''
r = client.messages.create(
model='claude-opus-4-5', max_tokens=100,
messages=[{'role': 'user', 'content': [
{'type': 'image', 'source': {'type': 'base64', 'media_type': 'image/jpeg', 'data': img_b64}},
{'type': 'text', 'text': prompt}
]}]
)
return json.loads(r.content[0].text)
# Example: 'Is there a safety helmet visible in the image?'
print('Yes/no VQA function defined.')串联 VQA 问题
可以在单个提示词中串联关于同一张图片的多个 VQA 问题,从而减少应用程序接口调用次数:
multi_question_prompt = '''
Answer all of the following questions about this image.
Return a JSON object where each key is the question ID.
Questions:
1. How many people are visible?
2. What is the approximate age range of the youngest person?
3. Is there any food visible in the image?
4. What is the dominant color in the image?
5. Is the setting indoors or outdoors?
Return JSON:
{
"q1": {"answer": str},
"q2": {"answer": str},
"q3": {"answer": "yes|no", "details": str or null},
"q4": {"answer": str},
"q5": {"answer": "indoors|outdoors|unclear"}
}
'''
print('Multi-question VQA prompt — answers 5 questions in one API call.')处理 VQA 不确定性
VQA 问题有时无法得到确定答案——图片可能模糊,相关元素可能部分被遮挡,或者答案本身确实存在歧义。请提示模型明确表达不确定性,而不是强行猜测:
uncertainty_vqa_prompt = '''
Answer this question about the image as precisely as possible.
If the answer is not clearly visible or is ambiguous, say so explicitly.
Question: {question}
Return JSON:
{
"answer": str,
"confidence": "high|medium|low|cannot_determine",
"limitation": str or null
}
For confidence levels:
- high: Answer is clearly visible and unambiguous
- medium: Visible but some uncertainty
- low: Partially visible or requires inference
- cannot_determine: Not enough visual information
Question: What brand is printed on the water bottle?
'''
print(uncertainty_vqa_prompt)特定领域的 VQA 提示词
不同领域需要使用不同的 VQA 词汇和测量标准。特定领域的提示词可以生成更准确、更具可操作性的答案:
# Manufacturing quality control VQA
qc_prompt = '''
Inspect this product image for quality defects.
Answer each question:
1. Are there any visible scratches or surface damage? (yes/no + location)
2. Is the product alignment within expected tolerance? (yes/no)
3. Are all required labels/markings present? (yes/no + list missing ones)
4. Overall QC result: PASS or FAIL?
Return JSON:
{"scratches": {"present": bool, "location": str or null},
"alignment_ok": bool,
"labels_complete": bool, "missing_labels": [str],
"qc_result": "PASS|FAIL",
"fail_reasons": [str]}
'''
# Food safety VQA
food_prompt = '''
Inspect this food preparation image.
1. Are gloves being worn? 2. Is hair covered? 3. Any visible contamination risk?
Return JSON: {"gloves": bool, "hair_covered": bool, "contamination_risk": bool, "details": str}
'''
print("Domain-specific QC and food safety VQA prompts defined.")快速检查
在对图片中的物体进行计数时,以下哪种 VQA 提示词最有可能生成精确且可用的答案?
VQA 提示词——要点总结
高效的视觉问答需要经过精心设计的提示词:
- 让问题具体、直接——避免使用某些或各种等含糊的词语
- 明确说明计数问题中的边界情况(部分可见时是否计入?)
- 指定确切的输出格式——对象表示法、单个数字、是/否——以避免得到含糊其辞的叙述性回答
- 为所有回答提供置信度等级,以便标记不确定的输出
- 将关于同一张图片的多个问题合并到一次调用中,以降低应用程序接口成本
- 对于二元是/否问题,应明确禁止含糊回答,并提供不明确这一兜底选项
- 特定领域的 VQA(医疗、法律、产品)需要在提示词中使用该领域的词汇
用 AI 导师学习 AI Prompt Engineering — 免费
在浏览器中编写并运行真实代码,获得全天候 AI 导师的即时帮助,并在网页或应用中继续学习。
- 课程
- 53
- 课程
- 199
常见问题解答
「视觉问答」课时是免费的吗?
是的 — 「视觉问答」的完整文本可在网页上免费阅读。要进行交互式练习(内置代码编辑器和全天候 AI 导师)并解锁 AI Prompt Engineering 课程的其余内容,请升级到 CoddyKit PRO。 AI Prompt Engineering 课程共包含 4 节课。
「视觉问答」这节课中我会学到什么?
针对图像内容、数量和属性提出具体问题 你通过在浏览器中直接运行的动手代码来练习 AI Prompt Engineering,全天候 AI 导师会在你学习这节课的过程中回答你的问题。
学习 AI Prompt Engineering 需要有经验吗?
无需任何先前经验。CoddyKit 上的 AI Prompt Engineering 课程适合初学者到高级学习者,你可以从这里开始或从头开始,按照自己的节奏学习。 这是第 2 节课,共 4 节。
「视觉问答」课时需要多长时间?
大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动,所以你能稳步进步,并在网页和应用中从离开的地方继续。
我能在这节 AI Prompt Engineering 课中编写并运行代码吗?
能。每节 AI Prompt Engineering 课都包含内置代码编辑器,你可以在浏览器中直接编写并运行真实代码,并获得即时 AI 反馈 — 无需本地设置。