Glossary

Fine-Tuning

Fine-tuning is the process of taking a pre-trained large language model and further training it on a smaller, domain-specific dataset to specialize its behavior, knowledge, or output style. The model weights are updated to reflect the new training data, creating a customized version of the base model.

How it works

Pre-trained LLMs are generalists — they know a lot about many topics but are not experts in any specific domain. Fine-tuning narrows this generality:

1. **Start with a pre-trained model** (e.g., GPT-5, Llama) that already understands language 2. **Provide domain-specific training examples** — typically hundreds to thousands of input-output pairs showing desired behavior 3. **Train for a few epochs** — the model adjusts its weights to perform better on your specific task 4. **Result**: A specialized model that retains general language ability but excels at your domain

Fine-tuning is commonly used for: adapting tone and style (matching brand voice), teaching specific output formats (JSON, structured responses), improving performance on niche domains (medical, legal, financial), and reducing latency by using smaller fine-tuned models instead of larger general ones.

Why it matters

Fine-tuning creates models that are faster, cheaper, and more consistent for specific tasks. However, it has significant trade-offs: it requires curated training data, is expensive to run, creates static knowledge (no live updates), and needs re-training when information changes. For most customer support use cases, RAG is more practical than fine-tuning because support content changes frequently.

How Chatsy uses fine-tuning

Chatsy primarily uses RAG rather than fine-tuning for customer support, because knowledge base content changes frequently and RAG reflects updates immediately. However, fine-tuning is used internally to optimize model behavior for support-specific tasks like tone calibration, response formatting, and escalation detection — areas where consistent behavior patterns matter more than factual content.

Real-world examples

Brand voice adaptation

A luxury brand fine-tunes a model on 5,000 examples of their customer communications to match their formal, elegant tone. The fine-tuned model consistently produces responses in the brand voice without needing extensive tone instructions in every prompt — reducing token usage and latency.

Medical terminology specialization

A healthcare company fine-tunes a model on medical literature and patient communication examples. The resulting model correctly uses medical terminology, understands symptom descriptions, and generates clinically appropriate responses — outperforming the base model on medical support tasks by 30%.

Structured output format training

A ticketing system fine-tunes a model to always output responses in a specific JSON format with fields for category, priority, summary, and suggested_action. The fine-tuned model produces valid JSON 99.5% of the time vs 85% for the base model with prompt-only instructions.

Key takeaways

Fine-tuning further trains a pre-trained model on domain-specific data to specialize its behavior
It excels at adapting tone, style, output format, and domain-specific language patterns
RAG is generally preferred over fine-tuning for customer support because content changes frequently
Fine-tuning creates static knowledge that requires re-training to update, while RAG updates instantly
The most effective approach often combines both: fine-tuning for behavior and RAG for factual content

Frequently asked questions

When should I use fine-tuning instead of RAG?

Use fine-tuning when you need consistent behavior changes (tone, style, format) rather than factual knowledge updates. Fine-tuning is better for teaching the model how to respond, while RAG is better for providing what to respond with. Most customer support use cases are better served by RAG or a combination of both.

How much training data does fine-tuning require?

Effective fine-tuning typically requires 500-5,000 high-quality input-output examples. More data generally improves results, but quality matters more than quantity. Poorly curated training data produces a model that is confidently wrong, which is worse than the base model.

How much does fine-tuning cost?

Fine-tuning costs include training compute ($50-$500+ per training run depending on model size and data volume), hosting the fine-tuned model ($100-$1,000+/month for inference), and data curation time (often the largest hidden cost). RAG on a base model is typically 5-10x cheaper for customer support use cases.

Can I fine-tune any LLM?

Not all LLMs support fine-tuning. OpenAI offers fine-tuning for GPT-4o and GPT-4o-mini. Open-source models like Llama and Mistral can be fine-tuned freely. Anthropic Claude and Google Gemini have more limited fine-tuning access. Check each provider for current availability and pricing.

Related terms

Large Language Model (LLM)

A Large Language Model (LLM) is a type of AI model trained on enormous amounts of text data to understand and generate h...

Retrieval-Augmented Generation (RAG)

Retrieval-Augmented Generation (RAG) is an AI architecture that enhances large language model responses by first retriev...

AI Hallucination

AI hallucination is a phenomenon where a large language model generates text that is fluent, confident, and plausible-so...

Prompt Engineering

Prompt engineering is the practice of designing, structuring, and refining the instructions (prompts) given to large lan...