ChatClient, Prompts und strukturierte Ausgaben

Spring AI gives you a fluent, high-level API for talking to LLMs: the ChatClient. Instead of hand-building HTTP requests to OpenAI, Anthropic, or Ollama, you describe what you want and let the framework handle transport, retries, and message assembly.

• ChatClient — fluent builder for one-shot or streaming calls.
• Prompt — a list of messages (system, user, assistant) plus options.
• Structured output — map the model's text reply directly into a typed Java object.

It is portable: swap the underlying ChatModel (OpenAI → Anthropic) and your ChatClient code stays the same.

Add a Spring AI model starter and Spring Boot auto-configures a ChatModel bean plus a ChatClient.Builder you can inject.

• The starter (e.g. spring-ai-starter-model-openai) reads your API key and model from properties.
• You never construct ChatClient directly — you inject the builder and call .build().

Typical configuration in application.yml:

Inject the auto-configured ChatClient.Builder and build a client once, usually in your service constructor. You can attach default system prompts or options here so every call inherits them.

Building per-service (not per-request) keeps configuration in one place and is cheap.

The fluent chain reads like a sentence. Start with prompt(), add a user(...) message, then terminate with call() for a blocking response and content() to get the plain text.

• call() — synchronous request/response.
• content() — extracts the assistant's text.
• chatResponse() — returns metadata (token usage, finish reason) instead.

An LLM prompt is a sequence of messages with roles:

• System — instructions and persona; sets behavior.
• User — the actual request from the end user.
• Assistant — prior model replies (for multi-turn context).

You can override the default system message per call. Keep untrusted user input in user(...), never inside the system instruction, to reduce prompt-injection risk.

Hard-coding strings does not scale. Spring AI uses template placeholders (default {name} syntax via StringTemplate) that you fill with param(...). The framework substitutes values before the request is sent.

• Define the template text once with {placeholders}.
• Bind values with .user(u -> u.text(...).param(...)).

This separates wording from data and keeps user values clearly bound.

Under the hood Spring AI builds a PromptTemplate and renders it. You can also use the template directly when you want to reuse it or load it from a resource file.

Here is the same idea as a plain Java program you can run to see substitution — no Spring needed, just string formatting that mirrors what the renderer does:

Often you do not want prose — you want a typed object. Spring AI's structured output converters do three things: inject a format instruction into the prompt, receive the model's text, and deserialize it into your type.

Define a plain Java record, then call .entity(MyType.class) instead of .content().

For collections or generic containers, Java erases the type parameter at runtime, so List.class is not enough. Pass a ParameterizedTypeReference so Spring AI knows the element type and generates the right JSON schema instruction.

It is worth understanding the mechanism: entity() uses a BeanOutputConverter that generates a JSON Schema from your record and appends a format instruction telling the model to reply with matching JSON only.

• The model returns JSON text.
• The converter parses it (via Jackson) into your record.
• If the model adds stray prose, parsing can fail — lower temperature and keep records flat for reliability.

You can call the converter yourself to inspect the injected instruction:

For long answers, stream tokens as they arrive using stream() instead of call(), which returns a reactive Flux<String>. For observability, grab the full ChatResponse to read token usage and the finish reason.

• .stream().content() — Flux<String> of incremental chunks.
• .call().chatResponse().getMetadata().getUsage() — prompt/completion tokens.

You need the model's reply mapped into a List<Actor>. Which terminal call is correct?

You learned to call LLMs the Spring way with ChatClient:

• Inject ChatClient.Builder, set defaultSystem, and build() once per service.
• Use the fluent chain: prompt().system(...).user(...).call().content().
• Keep system instructions and untrusted user input in separate messages.
• Use {placeholder} templates with .param(...) to separate wording from data.
• Map replies to typed records with .entity(Type.class), and use ParameterizedTypeReference for generics like List<T>.
• Stream with .stream().content() and inspect token usage via chatResponse().getMetadata().

For reliable structured output, keep records flat and lower the temperature.