Restricted Boltzmann machine

A restricted Boltzmann machine (RBM) (also called a restricted Sherrington–Kirkpatrick model with external field or restricted stochastic Ising–Lenz–Little model) is a <a href="/facts/Generative_model/JUgExNIP">generative</a> <a href="/facts/Stochastic_neural_network/BKu7GCU3">stochastic</a> <a href="/facts/Artificial_neural_network/6V1jMlkx">artificial neural network</a> that can learn a <a href="/facts/Probability_distribution/EpsKKVRu">probability distribution</a> over its set of inputs.
RBMs were initially proposed under the name Harmonium by <a href="/facts/Paul_Smolensky/JrsCWlwT">Paul Smolensky</a> in 1986, and rose to prominence after <a href="/facts/Geoffrey_Hinton/HJU6lC2H">Geoffrey Hinton</a> and collaborators used fast learning algorithms for them in the mid-2000s. RBMs have found applications in <a href="/facts/Dimensionality_reduction/XoRpcrB0">dimensionality reduction</a>, <a href="/facts/Statistical_classification/jXXHRkXR">classification</a>, <a href="/facts/Collaborative_filtering/NhSFypjN">collaborative filtering</a>, <a href="/facts/Feature_learning/6GYV0mux">feature learning</a>, <a href="/facts/Topic_model/aCAuIjwe">topic modelling</a>, <a href="/facts/Immunology/vylm0L8e">immunology</a>, and even <a href="/facts/Many-body_problem/n2rIxgwv">many‑body quantum mechanics</a>.

They can be trained in either <a href="/facts/Supervised_learning/JlNFNPFt">supervised</a> or <a href="/facts/Unsupervised_learning/8a6f61MC">unsupervised</a> ways, depending on the task.
As their name implies, RBMs are a variant of <a href="/facts/Boltzmann_machine/2wyLI0pI">Boltzmann machines</a>, with the restriction that their <a href="/facts/Artificial_neural_network/6V1jMlkx">neurons</a> must form a <a href="/facts/Bipartite_graph/xWcXV9MB">bipartite graph</a>:

<ul><li>a pair of nodes from each of the two groups of units (commonly referred to as the "visible" and "hidden" units respectively) may have a symmetric connection between them; and</li>
<li>there are no connections between nodes within a group.</li></ul>
By contrast, "unrestricted" Boltzmann machines may have connections between <a href="/facts/Artificial_neural_network/6V1jMlkx">hidden units</a>. This restriction allows for more efficient training <a href="/facts/Algorithms/fnl5NmRt">algorithms</a> than are available for the general class of Boltzmann machines, in particular the <a href="/facts/Gradient_descent/pFFrek0F">gradient-based</a> contrastive divergence algorithm.
Restricted Boltzmann machines can also be used in <a href="/facts/Deep_learning/JLuwD3ea">deep learning</a> networks. In particular, <a href="/facts/Deep_belief_network/aPkAjLIU">deep belief networks</a> can be formed by "stacking" RBMs and optionally fine-tuning the resulting deep network with <a href="/facts/Gradient_descent/pFFrek0F">gradient descent</a> and <a href="/facts/Backpropagation/lCsIdKHc">backpropagation</a>.

Restricted Boltzmann machine open-in-new

Restricted Boltzmann machine