Fine-tuning (deep learning)

In <a href="/facts/Deep_learning/JLuwD3ea">deep learning</a>, fine-tuning is an approach to <a href="/facts/Transfer_learning/pNz4P2KP">transfer learning</a> in which the parameters of a pre-trained <a href="/facts/Neural_network_(machine_learning)/6V1jMlkx">neural network</a> model are trained on new data. Fine-tuning can be done on the entire neural network, or on only a subset of its <a href="/facts/Hidden_layers/uKzUskIX">layers</a>, in which case the layers that are not being fine-tuned are "frozen" (i.e., not changed during <a href="/facts/Backpropagation/lCsIdKHc">backpropagation</a>). A model may also be augmented with "adapters" that consist of far fewer parameters than the original model, and fine-tuned in a parameter-efficient way by tuning the weights of the adapters and leaving the rest of the model's weights frozen.
For some architectures, such as <a href="/facts/Convolutional_neural_network/kHEdnmGU">convolutional neural networks</a>, it is common to keep the earlier layers (those closest to the input layer) frozen, as they capture lower-level <a href="/facts/Feature_(computer_vision)/GdfWRp5U">features</a>, while later layers often discern high-level features that can be more related to the task that the model is trained on.
Models that are pre-trained on large, general corpora are usually fine-tuned by reusing their parameters as a starting point and adding a task-specific layer trained from scratch. Fine-tuning the full model is also common and often yields better results, but is more computationally expensive.
Fine-tuning is typically accomplished via <a href="/facts/Supervised_learning/JlNFNPFt">supervised learning</a>, but there are also techniques to fine-tune a model using <a href="/facts/Semi-supervised_learning/4mzcqFjI">weak supervision</a>. Fine-tuning can be combined with a <a href="/facts/Reinforcement_learning_from_human_feedback/WaWzn0x1">reinforcement learning from human feedback</a>-based <a href="/facts/Loss_function/xv5ozuhl">objective</a> to produce language models such as <a href="/facts/ChatGPT/ptB1y6J5">ChatGPT</a> (a fine-tuned version of <a href="/facts/Generative_pre-trained_transformer/myah4Tjj">GPT models</a>) and <a href="/facts/Sparrow_(bot)/jUwESYur">Sparrow</a>.

Fine-tuning (deep learning) open-in-new

Fine-tuning (deep learning)