Foundation model

A foundation model is a machine learning model that is trained on broad data such that it can be applied across a wide range of use cases. Foundation models have transformed artificial intelligence (AI), powering prominent generative AI applications like ChatGPT. The Stanford Institute for Human-Centered Artificial Intelligence's (HAI) Center for Research on Foundation Models (CRFM) created and popularized the term.

Foundation models are general-purpose technologies that can support a diverse range of use cases. Building foundation models is often highly resource-intensive, with the most expensive models costing hundreds of millions of dollars to pay for the underlying data and compute required. In contrast, adapting an existing foundation model for a specific use case or using it directly is much less expensive.

Early examples of foundation models were language models (LMs) like Google's BERT and OpenAI's "GPT-n" series. Beyond text, foundation models have been developed across a range of modalities—including DALL-E and Flamingo for images, MusicGen for music, and RT-2 for robotic control. Foundation models constitute a broad shift in AI development: foundation models are being built for astronomy, radiology, genomics, music, coding, and mathematics.