RAG or fine-tuning? The practical difference for business agents
When retrieval-augmented generation is the answer, when fine-tuning is, and how to decide based on how fast your data changes.
· 7 min read
The real question: what changes?
Reduce the decision to one question: are you teaching the model facts or behaviour? Facts change: a price moved yesterday, a product sold out today. Behaviour is comparatively fixed: reply tersely, or always emit JSON in a given shape.
Fine-tuning bakes information into the weights. Excellent for behaviour, disastrous for facts: changing one price means a whole training run, and the old figure stays latent in the weights regardless.
What RAG is and is not good at
RAG searches your knowledge at question time and hands the relevant chunks to the model. Its decisive advantage is auditability: you can see which document produced which answer, and that alone makes it the only acceptable option in regulated contexts.
It does not fix style, though. If the agent answers at tiresome length or in the wrong register, the problem is the instructions, not the retrieval.
The hybrid, and when it earns its keep
Combining them makes sense in one recurring case: a domain with heavily specialised vocabulary (medical, legal, industrial) where the model needs training to understand the terms and RAG to know the current facts. Outside that, the hybrid is complexity without a return.
Frequently asked
- Which is cheaper?
- RAG is cheaper to set up and slightly costlier per message, since every question carries retrieved context. Fine-tuning is the inverse. At ordinary business volumes RAG stays cheaper overall.
- Does RAG eliminate hallucination?
- Not entirely, but it shrinks it dramatically and makes it detectable. With a similarity floor and an explicit instruction not to answer without support, hallucination becomes a rare, observable event.
Read next
How to build an Arabic AI agent for customer service
A practical guide to an agent that understands Modern Standard Arabic and dialects, answers only from your knowledge, and hands off to a human when needed.
A security checklist before launching an AI agent
What to verify before your agent talks to a real customer: prompt injection, tenant isolation, data leakage, and audit.
Designing the handoff from agent to human
When an agent should hand off, what must travel with it, and why a zero handoff rate is a bad sign rather than a good one.