# Fine-tuning

> Fine-tuning is the process of continuing the training of a pre-trained model on a smaller, task-specific dataset to specialise its behaviour.

Category: Architecture
Source: https://impetora.com/glossary/fine-tuning
Part of: Impetora AI consulting glossary (https://impetora.com/glossary)

## What is Fine-tuning?

Fine-tuning updates some or all of a model's weights using supervised examples. Variants include full fine-tuning, parameter-efficient fine-tuning (LoRA, QLoRA), instruction tuning, and reinforcement learning from human feedback. Fine-tuning is most useful for fixed style, format, or domain vocabulary that prompting cannot reliably capture. It is rarely the right answer for adding new factual knowledge, where retrieval-augmented generation is cheaper, more auditable, and easier to update.

## How does Fine-tuning apply to enterprise AI?

Enterprises fine-tune to enforce tone-of-voice, regulated output formats, multilingual code-switching, or domain jargon. Fine-tuning data must be governed under GDPR and the EU AI Act, including consent, retention, and the right to erasure.

## Related terms

- [Foundation Model](https://impetora.com/glossary/foundation-model) - A foundation model is a large neural network pre-trained on broad data and designed to be adapted to many downstream tasks.
- [RAG (Retrieval-Augmented Generation)](https://impetora.com/glossary/rag) - Retrieval-Augmented Generation (RAG) is an architecture pattern that grounds a language model's output in retrieved source documents rather than relying on the model's parametric memory alone.
- [Prompt Engineering](https://impetora.com/glossary/prompt-engineering) - Prompt engineering is the practice of designing, testing, and versioning the instructions given to a language model to elicit reliable, evaluable outputs.
- [Data Residency](https://impetora.com/glossary/data-residency) - Data residency is the requirement that personal or regulated data stays within a specified geographic region throughout processing, storage, and backup.

## External references

- [Hu et al., LoRA](https://arxiv.org/abs/2106.09685)

---

Impetora is a custom AI consultancy and solutions partner for enterprises in regulated industries. Submit a project at https://impetora.com/intake.