Large width limits of neural networks

<p><a href="/facts/Artificial_neural_network/6V1jMlkx">Artificial neural networks</a> are a class of models used in <a href="/facts/Machine_learning/e0w0XJTu">machine learning</a>, and inspired by <a href="/facts/Neural_circuit/pgngqMLf">biological neural networks</a>. They are the core component of modern <a href="/facts/Deep_learning/JLuwD3ea">deep learning</a> algorithms. Computation in artificial neural networks is usually organized into sequential layers of <a href="/facts/Artificial_neuron/TSH6nsgg">artificial neurons</a>. The number of neurons in a layer is called the layer width. Theoretical analysis of artificial neural networks sometimes considers the limiting case that layer width becomes large or infinite. This limit enables simple analytic statements to be made about neural network predictions, training dynamics, generalization, and loss surfaces. This wide layer limit is also of practical interest, since finite width neural networks often perform strictly better as layer width is increased.
</p>

Large width limits of neural networks open-in-new

Large width limits of neural networks