5 When Is Fine-Tuning Necessary?

This is an important question because modern large language models are already highly capable and may not actually require fine-tuning.

We can often achieve our goals through RAG or prompt engineering.

Analysis of Scenarios Requiring Fine-Tuning and Their Alternatives

Fine-tuning is not necessary in every scenario, especially given the powerful foundational capabilities of modern large language models (LLMs).

A modern large language model (LLM) is like a university graduate who has received a comprehensive education. However, not every application requires this “graduate” to undergo additional fine-tuning. Let us use a few straightforward examples to understand when fine-tuning is necessary and when it is not.

Scenario Analysis

Scenario 1: Standard Tasks — No Fine-Tuning Needed, Just Optimize the Prompt

Suppose you ask the model to perform a standard task that it is already capable of handling, such as writing a “standard procedure for clocking in and out of work.” In my view, this is something a university graduate should already know how to do. So why might the result be unsatisfactory? The problem is usually not that the model lacks the necessary capabilities, but that the instructions are not clear enough.

Solution: Prompt engineering

Provide a clear document structure and requirements
Specify the writing style, format, and key content
Provide examples or templates for reference

Fine-tuning is unnecessary in this situation because the model already understands the basic logic of the task. You simply need to write a better prompt.

Scenario 2: Lack of Knowledge — No Fine-Tuning Needed; Use RAG or Add Context

When the model needs to handle domain-specific knowledge, especially proprietary terminology used within a company, such as internal system code names like “abc” and “bbc,” what the model lacks is information rather than capability. The model does not inherently know what these terms mean, so it cannot write about them correctly.

Solutions:

Add context: Provide the necessary background information directly in the prompt
Retrieval-Augmented Generation (RAG): Build a knowledge base that allows the model to retrieve relevant information automatically when answering

In this situation, fine-tuning is unnecessarily expensive because the model only needs access to the meanings and relationships associated with these proprietary terms; it does not need to learn a new reasoning pattern. It simply needs the support of a knowledge base.

Scenario 3: Unique Logic — Fine-Tuning Is Necessary

Suppose you want to build something for highly specialized internal company use. It has its own unique logic, operating procedures, and custom formulas. For example, the company may have its own distinctive way of interpreting a particular business process, supported by a complex set of internal rules. In this case, fine-tuning is necessary.

Why fine-tuning is necessary:

The model needs to learn a domain-specific way of thinking, including unique and complex decision-making processes and evaluation criteria
The model must maintain a consistent response pattern in specific situations that differs from its usual behavior

Fine-tuning is effective in this situation because you are teaching the model a new professional capability rather than merely providing it with information.

A New Question

In practice, however, Scenarios 2 and 3 often overlap because both approaches can produce similar results. Knowledge-base content can also be taught to the model through fine-tuning, while unique logical knowledge can likewise be provided to the model through a knowledge base. The final results may be similar.

Therefore, in practice, we do not always use one fixed method. We need to choose an approach based on the actual circumstances. I will discuss this situation in the next article.