Grounding Across Modalities
Referencing visual evidence in text.
What Grounding Means
Grounding is the requirement that every textual claim be traceable to specific evidence in another modality. An ungrounded multimodal answer is a fluent guess; a grounded one is a defensible assertion with a pointer back to the pixels (or audio frame) that justify it.
- Grounding converts opaque output into auditable output.
- It is the single most effective lever against confident visual hallucination.
Evidence-First Reasoning Order
Force the model to extract evidence before it concludes. If the conclusion is generated first, the 'evidence' becomes post-hoc rationalization that matches the (possibly wrong) answer.
Structure the response so observed regions come first, interpretation second, and final answer last.
schema = {
'observations': '[{region: str, text_read: str}]',
'inference': 'str — reasoning over observations only',
'answer': 'str'
}
# Order in the prompt enforces order in generation.All lessons in this course
- Combining Text and Images
- Grounding Across Modalities
- Audio, Text and Vision Together
- Multimodal Output Control