A basic algorithm for pruning is as follows:34
Most work on neural network pruning focuses on removing weights, namely, setting their values to zero. Early work suggested to also change the values of non-pruned weights.5
Blalock, Davis; Ortiz, Jose Javier Gonzalez; Frankle, Jonathan; Guttag, John (2020-03-06). "What is the State of Neural Network Pruning?". arXiv:2003.03033 [cs.LG]. /wiki/ArXiv_(identifier) ↩
Chechik, Gal; Meilijson, Isaac; Ruppin, Eytan (October 1998). "Synaptic Pruning in Development: A Computational Account". Neural Computation. 10 (7): 1759–1777. doi:10.1162/089976698300017124. ISSN 0899-7667. PMID 9744896. S2CID 14629275. https://ieeexplore.ieee.org/document/6790725 ↩
Molchanov, P., Tyree, S., Karras, T., Aila, T., & Kautz, J. (2016). Pruning convolutional neural networks for resource efficient inference. arXiv preprint arXiv:1611.06440. ↩
Gildenblat, Jacob (2017-06-23). "Pruning deep neural networks to make them fast and small". Github. Retrieved 2024-02-04. http://jacobgil.github.io/deeplearning/pruning-deep-learning ↩
Chechik, Gal; Meilijson, Isaac; Ruppin, Eytan (April 2001). "Effective Neuronal Learning with Ineffective Hebbian Learning Rules". Neural Computation. 13 (4): 817–840. doi:10.1162/089976601300014367. ISSN 0899-7667. PMID 11255571. S2CID 133186. https://ieeexplore.ieee.org/document/6789989 ↩