Foundation model

In <a href="/facts/Artificial_intelligence/lJGauQwX">artificial intelligence</a> (AI), a foundation model (FM), also known as large X model (LxM), is a <a href="/facts/Machine_learning/e0w0XJTu">machine learning</a> or <a href="/facts/Deep_learning/JLuwD3ea">deep learning</a> model trained on vast datasets so that it can be applied across a wide range of use cases. <a href="/facts/Generative_artificial_intelligence/ykT3GGyT">Generative AI</a> applications like <a href="/facts/Large_language_model/WnogWVJY">large language models</a> (LLM) are common examples of foundation models.
Building foundation models is often highly resource-intensive, with the most advanced models costing hundreds of millions of dollars to cover the expenses of acquiring, curating, and processing massive datasets, as well as the compute power required for training. These costs stem from the need for sophisticated infrastructure, extended training times, and advanced hardware, such as <a href="/facts/GPU/PTK1RQVp">GPUs</a>. In contrast, adapting an existing foundation model for a specific task or using it directly is far less costly, as it leverages pre-trained capabilities and typically requires only fine-tuning on smaller, task-specific datasets.
Early examples of foundation models are <a href="/facts/Language_models/FntSpg0j">language models</a> (LMs) like <a href="/facts/Generative_pre-trained_transformer/U6JNeDMe">OpenAI's GPT</a> series and <a href="/facts/Google/GT9Sugza">Google</a>'s <a href="/facts/BERT_(language_model)/rDSueM4E">BERT</a>. Beyond text, foundation models have been developed across a range of modalities—including <a href="/facts/DALL-E/rkWL8P49">DALL-E</a> and Flamingo for images, MusicGen for music, and RT-2 for robotic control. Foundation models are also being developed for fields like astronomy, radiology, genomics, music, coding, <a href="/facts/Time_series/fSXPR817">times-series</a> forecasting, mathematics, and chemistry.

Foundation model open-in-new

Foundation model