0Pricing
AI Agents · Lesson

Query Engines and Response Synthesis

Query engines orchestrate retrieval + synthesis with strategies like refine, compact, and tree_summarize.

Query Engines and Response Synthesis is a free AI Agents lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Agents learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

Query Engine = Retrieval + Synthesis

A LlamaIndex query engine has two stages:

  1. Retrieval — find relevant chunks
  2. Synthesis — generate an answer from those chunks

Both are configurable.

The Default Query Engine

query_engine = index.as_query_engine()
response = query_engine.query('What is our refund policy?')
print(response.response)         # the answer
print(response.source_nodes)     # the chunks used

Response Modes

How the engine assembles the final answer from retrieved chunks:

  • refine — pass chunks one by one, refining the answer (most accurate, slowest)
  • compact — fit as many chunks as possible into the context per call (default)
  • tree_summarize — recursive summarisation
  • simple_summarize — single call with all chunks

Setting Response Mode

query_engine = index.as_query_engine(response_mode='tree_summarize')

Configuring Retrieval

query_engine = index.as_query_engine(
    similarity_top_k=5,
    response_mode='compact'
)

Custom Prompts

Inject your own prompt templates:

from llama_index.core.prompts import PromptTemplate

qa_template = PromptTemplate(
    'Context:\n{context_str}\n\nQuestion: {query_str}\nAnswer:'
)
query_engine.update_prompts({'response_synthesizer:text_qa_template': qa_template})

Streaming Responses

query_engine = index.as_query_engine(streaming=True)
response = query_engine.query('Hello')
for token in response.response_gen:
    print(token, end='', flush=True)

Citations

Use CitationQueryEngine to get LLM-generated citations:

from llama_index.core.query_engine import CitationQueryEngine

engine = CitationQueryEngine.from_args(index, similarity_top_k=4)
response = engine.query('What is our refund policy?')
print(response.response)
for i, node in enumerate(response.source_nodes):
    print(f'[{i+1}] {node.metadata}')

Post-Processors

Re-rank or filter retrieved nodes before synthesis:

from llama_index.postprocessor.cohere_rerank import CohereRerank

rerank = CohereRerank(api_key=COHERE_KEY, top_n=3)
query_engine = index.as_query_engine(
    similarity_top_k=20,           # retrieve 20
    node_postprocessors=[rerank]   # rerank to top 3
)

Metadata Filters

from llama_index.core.vector_stores import MetadataFilters, ExactMatchFilter

filters = MetadataFilters(filters=[
    ExactMatchFilter(key='department', value='engineering')
])
query_engine = index.as_query_engine(filters=filters)

Async Queries

response = await query_engine.aquery('What is our refund policy?')
print(response.response)

Chat Engine

For multi-turn conversations with memory:

chat_engine = index.as_chat_engine(chat_mode='context')
response = chat_engine.chat('What is our refund policy?')
followup = chat_engine.chat('And for opened items?')

Pipelines and Workflows

For complex multi-step agents, LlamaIndex Workflows give you a Pythonic event-driven flow — similar in spirit to LangGraph.

Response Mode

Which response mode is slowest but most accurate?

Recap

Configure top_k, response_mode, post-processors, and metadata filters to shape your query engine. Use CitationQueryEngine for trustable answers.

Frequently asked questions

Is the “Query Engines and Response Synthesis” lesson free?

Yes — the full text of “Query Engines and Response Synthesis” is free to read here on the web, and the AI Agents course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Agents course, upgrade to CoddyKit PRO.

What will I learn in “Query Engines and Response Synthesis”?

Query engines orchestrate retrieval + synthesis with strategies like refine, compact, and tree_summarize. You practise AI Agents with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start AI Agents?

No prior experience is required. AI Agents on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Query Engines and Response Synthesis” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this AI Agents lesson?

Yes. Every AI Agents lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. Document Loaders and Parsers
  2. The Index Hierarchy: Vector, Tree, Keyword
  3. Query Engines and Response Synthesis
  4. Sub-Question Decomposition Strategy
← Back to AI Agents