An Overview of Parameter Efficient Fine-Tuning (PEFT)

May 23, 2024

Summary

This post is all about Parameter Efficient Fine-Tuning (PEFT), which are a broad range of techniques that are used to fine-tune custom LLMs while keeping compute resources/costs, training time, and inference time in mind.

We’ll specifically cover four topics in this overview:

What is Parameter Efficient Fine-Tuning (PEFT)?
What are the Benefits of Fine-Tuning LLMs with PEFT?
How Does PEFT Impacts LLM Training, Deployment, and Your ROI?

Get Organized with Notion

Support Torchstack with our affiliate link.

Do you have Post It Notes everywhere with your to-dos scribbled in semi-legible writing? Are you still doing market research in a spreadsheet (or worse, in a Word document)?

If you want to become more productive, get more work done, and collaborate with your team in an easier and better way, you should try out Notion. The Torchstack team uses it for everything, including:

Writing blog posts, business plans, technical documentation, or just brainstorming with image, videos, and links easily embedded into the page.

Tracking projects with To Do Lists, Kan Ban boards, Timelines, and Notion Calendar, which easily syncs and integrates with other calendar clients (e.g. Google Calendar).

Research: easily store links and documents, notes, and other tags/categories in easy-to-use tables that can interconnect with other tables for more complex analyses.

And much more. Whatever work, creative, or personal organization tools you need, Notion already has. Additionally, there are so many free templates that Notion provides so you can get started immediately.

Finally, Notion AI uses LLMs to do a lot of heavy lifting within the app itself, whether you need to summarize a page, draft out some text, or just make a list of action items from your meeting notes.

Get organized with Notion today, and support Torchstack with our affiliate link.

What is Parameter Efficient Fine-Tuning (PEFT)?

Parameter Efficient Fine-Tuning Overview

Parameter Efficient Fine-Tuning (PEFT) are methods that allow us to fine-tune a small number of model parameters for new tasks and with new domain knowledge, while retaining the performance and knowledge from the original pre-trained model. In many cases, this is done by freezing (or not training) the weights of the pre-trained LLM, while only training these new parameters.

PEFT provides several benefits compared to full fine-tuning, where we fine-tune the entire model on the new data, which include reducing training costs and mitigating the risks of ineffective model training.

Reducing training costs

When fine-tuning an LLM, we need to consider the following aspects to develop a cost model.

Storage: During training, we checkpoint or store intermediate versions of the model. Larger models consume more storage.
Instance size (compute power): We need powerful compute processors to train larger models. Instances with more powerful GPUs or TPUs incur higher costs. However, there is a balance between instance cost per hour and the amount of training time → longer training times = more costs per hour. Practically, the instance size and time allocated for training needs to be optimized.
Memory usage: We need to consider how much RAM and VRAM we’re consuming for the dataset, batch, model parameters, gradients and weights from the forward/backward passes, and other parameters that we’re storing during each training cycle or epoch.

Since PEFT reduces the number of trained parameters, this lowers the number of parameters that need to be stored, reduces the instance size, and decreases the memory needed for training. All of these factors reduce the time and costs associated with LLM fine-tuning.

Mitigating risks of ineffective model training

The whole point of fine-tuning an LLM is to train it to perform a specific task (e.g. summarize personal finance topics) really well. The main benefit of PEFT is that it reduces the two main risk associated with full fine-tuning: overfitting and catastrophic forgetting.

Overfitting: LLMs and AI models that update all their weights on a smaller amount of specialized data run the risk of overfitting (or memorizing) on the new data points.
- PEFT reduces the risk of overfitting because the smaller number of trained parameters are used with the original pre-trained weights to preserve the original knowledge while injecting some aspects of the new task or domain into the model.
Catastrophic forgetting: Overfitting on new data points can lead to catastrophic forgetting, which is forgetting the good things that the pre-trained model already learned, and reducing its performance on those previously learned tasks.
- PEFT reduces the risk of catastrophic forgetting because the core knowledge from the pre-trained model is retained, while the new knowledge from the new parameters will have a smaller negative impact on model performance.

Fine-Tuning with PEFT and LLMs: Benefits and Risks

There are many other benefits and risks associated with PEFT and the practical considerations associated with model fine-tuning and training. This includes costs, speed, scalability, and ultimately if the fine-tuned LLM impacts your bottom line.

We summarize these considerations in the table below.

Fine-Tuning Considerations	Benefits of PEFT	Risks with PEFT and Fine-Tuning
Cost Efficiency	Reduced training and deployment costs from smaller number of parameter updates.	Experimentation to find the optimal configuration can drive costs up.
Performance	Can have targeted improvements on specific tasks.	May not have generalized performance improvements on all tasks.
Deployment speed	Less training time means faster deployment time.	Additional optimization may be needed, which may reduce deployment time.
Inference speed	Can lead to faster inference due to updating fewer parameters and limited additional computational overhead.	Complex layers, high dimensional prompts, integration overhead, and complex routing can slow down inference.
Resource utilization	Utilizes less computational resources.	Additional optimization may be needed, which may increase resource utilization.
Scalability	PEFT is scalable and can better support incremental updates.	There may be limits to an LLM’s ability to adapt to new data patterns.
Maintainability	It is simpler to maintain a smaller set of updated parameters.	Continual tuning and updating are needed, which means ongoing updates.
Data fitting	There is a lower likelihood of model overfitting.	The PEFT layer can underperform on broad tasks.
ROI	You can have a targeted impact for fine-tuning, which can have a profound impact on ROI.	There can be indirect benefits or risks that make it difficult to evaluate the ROI of fine-tuning.

The general principle is that PEFT tends to lower resource costs and increase performance on new tasks, which would have a positive return-on-investment (ROI) for your business.

However, as with any research and development endeavor, additional unforeseen experimentation, optimization, and task complexity will add on costs.

Conclusions

Parameter Efficient Fine-Tuning (PEFT) are a set of methods that allow you to fine-tune custom large language models (LLMs) while reducing the training time and compute costs.
PEFT not only allows you to develop custom LLMs for your specific business, but it reduces the negative impact fine-tuning can have on a pre-trained LLM, including model overfitting and catastrophic forgetting.
While PEFT is a powerful tool for generating custom LLMs in a cost efficient way, there are always risks with fine-tuning LLMs that result from unexpected experimentation, performance limitations, and changing data/model requirements.

🔥 Develop Custom LLMs With Torchstack 🔥

Overwhelmed with PEFT and fine-tuning LLMs? Torchstack specializes in developing custom AI solutions with our expert team of AI/ML Researchers, Engineers, and Data Scientists. Schedule a call to get started working with us.

Back to blog

Item added to your cart