Menu
Home
People
Places
Arts
History
Plants & Animals
Science
Life & Culture
Technology
Reference.org
Activation function
open-in-new
<p>The activation function of a node in an <a href="/facts/Artificial_neural_network/6V1jMlkx">artificial neural network</a> is a function that calculates the output of the node based on its individual inputs and their weights. Nontrivial problems can be solved using only a few nodes if the activation function is <i>nonlinear</i>. </p><p>Modern activation functions include the logistic (<a href="/facts/Sigmoid_function/lAXu3AwW">sigmoid</a>) function used in the 2012 <a href="/facts/Speech_recognition/z7S7Pgk6">speech recognition</a> model developed by Hinton et al; the <a href="/facts/ReLU/pwzdU9jF">ReLU</a> used in the 2012 <a href="/facts/AlexNet/FUFE1Xov">AlexNet</a> computer vision model and in the 2015 <a href="/facts/Residual_neural_network/twRvA669">ResNet</a> model; and the smooth version of the ReLU, the <a href="/facts/ReLU/pwzdU9jF">GELU</a>, which was used in the 2018 <a href="/facts/BERT_(language_model)/rDSueM4E">BERT</a> model. </p>