Convolutional neural network

A convolutional neural network (CNN) is a type of <a href="/facts/Feedforward_neural_network/CP0pPGDF">feedforward neural network</a> that learns <a href="/facts/Feature_engineering/VkNJFAFb">features</a> via <a href="/facts/Filter_(signal_processing)/vydJCNle">filter</a> (or kernel) optimization. This type of <a href="/facts/Deep_learning/JLuwD3ea">deep learning</a> network has been applied to process and make <a href="/facts/Prediction/0w4XXB33">predictions</a> from many different types of data including text, images and audio. Convolution-based networks are the de-facto standard in <a href="/facts/Deep_learning/JLuwD3ea">deep learning</a>-based approaches to <a href="/facts/Computer_vision/Tl2Yyk66">computer vision</a> and image processing, and have only recently been replaced—in some cases—by newer deep learning architectures such as the <a href="/facts/Transformer_(deep_learning_architecture)/cDbjx6a8">transformer</a>.
<a href="/facts/Vanishing_gradient_problem/3hteovPr">Vanishing gradients</a> and exploding gradients, seen during <a href="/facts/Backpropagation/lCsIdKHc">backpropagation</a> in earlier neural networks, are prevented by the <a href="/facts/Regularization_(mathematics)/K601A3y2">regularization</a> that comes from using shared weights over fewer connections. For example, for each neuron in the fully-connected layer, 10,000 weights would be required for processing an image sized 100 × 100 pixels. However, applying cascaded convolution (or cross-correlation) kernels, only 25 weights for each convolutional layer are required to process 5x5-sized tiles. Higher-layer features are extracted from wider context windows, compared to lower-layer features.
Some applications of CNNs include: 

<ul><li><a href="/facts/Computer_vision/Tl2Yyk66">image and video recognition</a>,</li>
<li><a href="/facts/Recommender_system/HjodW6nS">recommender systems</a>,</li>
<li><a href="/facts/Image_classification/Tl2Yyk66">image classification</a>,</li>
<li><a href="/facts/Image_segmentation/jofAhbxa">image segmentation</a>,</li>
<li><a href="/facts/Medical_image_computing/aVMvp2Mx">medical image analysis</a>,</li>
<li><a href="/facts/Natural_language_processing/1hjMKsSN">natural language processing</a>,</li>
<li><a href="/facts/Brain%25E2%2580%2593computer_interface/wdXWqFVr">brain–computer interfaces</a>, and</li>
<li>financial <a href="/facts/Time_series/fSXPR817">time series</a>.</li></ul>
CNNs are also known as shift invariant or space invariant artificial neural networks, based on the shared-weight architecture of the <a href="/facts/Convolution/PVPrdz9J">convolution</a> kernels or filters that slide along input features and provide translation-<a href="/facts/Equivariant_map/KkIkDwIH">equivariant</a> responses known as feature maps. Counter-intuitively, most convolutional neural networks are not <a href="/facts/Translation_invariant/PYVMaqWb">invariant to translation</a>, due to the downsampling operation they apply to the input.
<a href="/facts/Feedforward_neural_network/CP0pPGDF">Feedforward neural networks</a> are usually fully connected networks, that is, each neuron in one <a href="/facts/Layer_(deep_learning)/cKFbI4GF">layer</a> is connected to all neurons in the next <a href="/facts/Layer_(deep_learning)/cKFbI4GF">layer</a>. The "full connectivity" of these networks makes them prone to <a href="/facts/Overfitting/5xnFLcMg">overfitting</a> data. Typical ways of regularization, or preventing overfitting, include: penalizing parameters during training (such as weight decay) or trimming connectivity (skipped connections, dropout, etc.) Robust datasets also increase the probability that CNNs will learn the generalized principles that characterize a given dataset rather than the biases of a poorly-populated set.
Convolutional networks were <a href="/facts/Mathematical_biology/pV9KCBSD">inspired</a> by <a href="/facts/Biological/1M1ee3pB">biological</a> processes in that the connectivity pattern between <a href="/facts/Artificial_neuron/TSH6nsgg">neurons</a> resembles the organization of the animal <a href="/facts/Visual_cortex/qmF8F5Sb">visual cortex</a>. Individual <a href="/facts/Cortical_neuron/GpXosGHP">cortical neurons</a> respond to stimuli only in a restricted region of the <a href="/facts/Visual_field/xckE6fpm">visual field</a> known as the <a href="/facts/Receptive_field/2LIVVKfe">receptive field</a>. The receptive fields of different neurons partially overlap such that they cover the entire visual field.
CNNs use relatively little pre-processing compared to other <a href="/facts/Image_classification/Tl2Yyk66">image classification algorithms</a>. This means that the network learns to optimize the <a href="/facts/Filter_(signal_processing)/vydJCNle">filters</a> (or kernels) through automated learning, whereas in traditional algorithms these filters are <a href="/facts/Feature_engineering/VkNJFAFb">hand-engineered</a>. This simplifies and automates the process, enhancing efficiency and scalability overcoming human-intervention bottlenecks.

Convolutional neural network open-in-new

Convolutional neural network