When teams want a language model to 'know' their business, two options come up immediately: retrieval-augmented generation (RAG) and fine-tuning. They're often presented as alternatives. In practice they address different needs.
What RAG is good at
RAG retrieves relevant content from your documents or data at query time and passes it to the model as context. It's the right choice when answers depend on knowledge that changes, when you need citations, and when different users should see different information.
- Knowledge that updates daily or weekly
- Answers that must cite sources
- Per-user or per-team access control
- Large corpora that would never fit in a prompt
What fine-tuning is good at
Fine-tuning adjusts the model's weights using examples. It's best for teaching behavior — a consistent output format, a domain-specific style, a classification task — or for making a smaller, cheaper model perform like a larger one on a narrow task.
A simple decision framework
Ask whether the problem is about what the model knows or how the model behaves. Knowledge problems point to RAG. Behavior problems point to better prompting first, then fine-tuning if prompting can't get you there at acceptable cost or latency.
Why most teams should start with RAG
RAG is faster to build, easier to update and easier to debug — you can inspect exactly what was retrieved. It also forces you to get your content pipeline and permissions right, which you'll need regardless. Fine-tuning becomes attractive later, once you have real usage data and a clear, stable task to optimize.
Retrieval gives the model the right facts. Fine-tuning teaches it the right habits.
Whatever you choose, evaluate
Build an evaluation set of real questions with expected answers before you build the system. Measure retrieval quality and answer quality separately. Without this, you can't tell whether a change made things better or worse.