Function-Calling Open Models (Hermes, Functionary)
Most base open models can't tool-call well — Hermes and Functionary are fine-tuned for it.
Function-Calling Open Models (Hermes, Functionary) is a free AI Agents lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Agents learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Base Models Cannot Tool-Call Reliably
Vanilla Llama, Mistral, or Qwen base models can be prompted to "return JSON like {...}" but often produce malformed output, miss arguments, or hallucinate tools.
Function-calling fine-tunes solve this.
Hermes Family
NousResearch Hermes models are top-tier function-calling fine-tunes:
- Hermes 3 Llama 3.1 8B / 70B
- Strong tool-call accuracy
- Apache-style permissive license
Functionary
MeetKai Functionary models are explicitly trained for OpenAI-compatible function calling:
- Functionary v3 (Llama 3.1 based)
- Supports parallel tool calls
- Available on HuggingFace and Together AI
Llama 3.1+ Native Tool Use
Recent Llama 3.1, Llama 3.3, and Llama 4 instruct variants have built-in tool-calling capabilities. For many tasks they're enough; for hard cases, use Hermes / Functionary.
Qwen 2.5 Tool Use
Qwen 2.5 has strong native tool-calling — competitive with closed models. Often the default choice for self-hosted multilingual agents.
Calling Tool-Capable Open Models
Any OpenAI-compatible server (Ollama, vLLM, Together AI) accepts the same tools parameter:
from openai import OpenAI
client = OpenAI(base_url='http://localhost:11434/v1', api_key='ollama')
tools = [{'type': 'function', 'function': {
'name': 'get_weather',
'description': 'Current weather',
'parameters': {'type': 'object', 'properties': {'city': {'type': 'string'}}, 'required': ['city']}
}}]
response = client.chat.completions.create(
model='hermes3:8b',
messages=[{'role': 'user', 'content': 'Weather in Paris?'}],
tools=tools
)Validate Outputs Carefully
Open-model tool calls fail more often than closed-model ones. Always:
- Parse JSON in try/except
- Validate with Pydantic
- Add a repair loop
Force Tool Calls with Outlines
For guaranteed structured output on open models, use Outlines for grammar-constrained decoding:
import outlines
model = outlines.models.transformers('meta-llama/Llama-3.1-8B-Instruct')
generator = outlines.generate.json(model, ToolCall)
result = generator(prompt)Hugging Face TGI Tool Use
TGI supports OpenAI-style tool use for many open models — drop-in replacement.
Benchmark Tool-Use Quality
Tool calling is benchmarked by:
- Berkeley Function Calling Leaderboard (BFCL) — definitive ranking
- ToolBench — wider coverage
- API-Bank — multi-tool scenarios
A Reasonable Default Stack
Combination most teams settle on:
- Llama 3.1 8B Instruct for general agent work
- Qwen 2.5 Coder for code agents
- Hermes 3 for tool-heavy agents
- Outlines or Instructor for structured-output guarantees
Run via Hosted Provider
For zero infra: Together AI, Fireworks, Anyscale host Hermes, Functionary, Llama, Qwen with OpenAI-compatible APIs. Cheaper than OpenAI for similar quality.
Cost Comparison Example
Approximate per-million-tokens cost (mid-2025):
- GPT-4o: $2.50 in, $10 out
- Llama 3.1 70B (Together AI): $0.88 in, $0.88 out
- Self-hosted Llama 8B: $0 (your electricity)
Why Use Function-Calling Fine-Tunes?
Why use Hermes or Functionary instead of base Llama for tool calling?
Recap
Recent Llama / Qwen have decent native tool use. For higher reliability, use Hermes or Functionary. Combine with Outlines/Instructor for guaranteed structure.
Frequently asked questions
Is the “Function-Calling Open Models (Hermes, Functionary)” lesson free?
Yes — the full text of “Function-Calling Open Models (Hermes, Functionary)” is free to read here on the web, and the AI Agents course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Agents course, upgrade to CoddyKit PRO.
What will I learn in “Function-Calling Open Models (Hermes, Functionary)”?
Most base open models can't tool-call well — Hermes and Functionary are fine-tuned for it. You practise AI Agents with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start AI Agents?
No prior experience is required. AI Agents on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Function-Calling Open Models (Hermes, Functionary)” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this AI Agents lesson?
Yes. Every AI Agents lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Llama, Mistral and Qwen Overview
- Running Local Models with Ollama and llama.cpp
- Function-Calling Open Models (Hermes, Functionary)
- Trade-offs: Latency, Cost, Capability