What is Fine-tuning?
Fine-tuning is further training an existing model on a specific dataset so it performs better on a narrower task or domain.
Also called: model fine-tuning, supervised fine-tuning
In more detail
Rather than training from scratch, an existing model's weights are adjusted — often through lightweight adapters such as LoRA — using examples of the target task. The result is a model specialised to that domain but frozen at the data it was trained on.
Why it matters
Fine-tuning helps with specialised language, strict output formats, or when latency and offline operation rule out calling an API. It does not add knowledge that changes, which is retrieval's job.
How iLeaf approaches it
iLeaf benchmarks prompting and retrieval first and often recommends against fine-tuning, because it is more expensive and must be redone as data changes. Where it is warranted, the evaluation set is built before the model.