Lazy learning

<h2 id="advantages">Advantages</h2>
<p>The main advantage gained in employing a lazy learning method is that the target function will be approximated locally, such as in the <a href="/facts/K-nearest_neighbor_algorithm/Bs1UaAmE">k-nearest neighbor algorithm</a>. Because the target function is approximated locally for each query to the system, lazy learning systems can simultaneously solve multiple problems and deal successfully with changes in the problem domain. At the same time they can reuse a lot of theoretical and applied results from linear regression modelling (notably <a href="/facts/PRESS_statistic/u0ksCavz">PRESS statistic</a>) and control.<a class="footnote-ref" id="fnref:3" href="#fn:3"><sup>3</sup></a> It is said that the advantage of this system is achieved if the predictions using a single training set are only developed for few objects.<a class="footnote-ref" id="fnref:4" href="#fn:4"><sup>4</sup></a> This can be demonstrated in the case of the k-NN technique, which is instance-based and function is only estimated locally.<a class="footnote-ref" id="fnref:5" href="#fn:5"><sup>5</sup></a><a class="footnote-ref" id="fnref:6" href="#fn:6"><sup>6</sup></a>
</p>
<h2 id="disadvantages">Disadvantages</h2>
<p>Theoretical disadvantages with lazy learning include:
</p>
<ul><li>The large space requirement to store the entire training dataset. In practice, this is not an issue because of advances in hardware and the relatively small number of attributes (e.g., as co-occurrence frequency) that need to be stored.</li>
<li>Particularly noisy training data increases the case base unnecessarily, because no abstraction is made during the training phase. In practice, as stated earlier, lazy learning is applied to situations where any learning performed in advance soon becomes obsolete because of changes in the data. Also, for the problems for which lazy learning is optimal, "noisy" data does not really occur - the purchaser of a book has either bought another book or hasn't.</li>
<li>Lazy learning methods are usually slower to evaluate. In practice, for very large databases with high concurrency loads, the queries are <i>not</i> postponed until actual query time, but recomputed in advance on a periodic basis - e.g., nightly, in anticipation of future queries, and the answers stored. This way, the next time  new queries are asked about existing entries in the database, the answers are merely looked up rapidly instead of having to be computed on the fly, which would almost certainly bring a high-concurrency multi-user system to its knees.</li>
<li>Larger training data also entail increased cost. Particularly, there is the fixed amount of computational cost,  where a processor can only process a limited amount of training data points.<a class="footnote-ref" id="fnref:7" href="#fn:7"><sup>7</sup></a></li></ul>
<p>There are standard techniques to improve re-computation efficiency so that a particular answer is not recomputed unless the data that impact this answer has changed (e.g., new items, new purchases, new views). In other words, the stored answers are updated incrementally.
</p><p>This approach, used by large e-commerce or media sites, has long been used in the <a href="/facts/Entrez/1b3UvdGY">Entrez</a> portal of the <a href="/facts/National_Center_for_Biotechnology_Information/6Xj5wCbu">National Center for Biotechnology Information</a> (NCBI) to precompute similarities between the different items in its large datasets: biological sequences, 3-D protein structures, published-article abstracts, etc. Because "find similar" queries are asked so frequently, the NCBI uses highly parallel hardware to perform nightly recomputation. The recomputation is performed only for new entries in the datasets against each other and against existing entries: the similarity between two existing entries need not be recomputed.
</p>
<h2 id="examples-of-lazy-learning-methods">Examples of Lazy Learning Methods</h2>
<ul><li><a href="/facts/K-nearest_neighbors/Bs1UaAmE">K-nearest neighbors</a>, which is a special case of instance-based learning.</li>
<li><a href="/facts/Local_regression/0kvMY73g">Local regression</a>.</li>
<li>Lazy <a href="/facts/Naive_Bayes/FmNHI688">naive Bayes</a> rules, which are extensively used in commercial spam detection software. Here, the spammers keep getting smarter and revising their spamming strategies, and therefore the learning rules must also be continually updated.</li></ul>

<h2 id="further-reading">Further reading</h2>
<ul><li><a href="https://cran.r-project.org/web/packages/lazy/">lazy: Lazy Learning for Local Regression</a>, <a href="/facts/R_(programming_language)/LSrkr8K8">R</a> package with reference manual</li>
<li><a href="https://web.archive.org/web/20120216183916/http://iridia0.ulb.ac.be/~lazy/">"The Lazy Learning Package"</a>. Archived from <a href="http://iridia0.ulb.ac.be/~lazy/">the original</a> on 16 February 2012.</li>
<li>Webb G.I. (2011) Lazy Learning. In: Sammut C., Webb G.I. (eds) Encyclopedia of Machine Learning. Springer, Boston, MA</li>
<li>David W. Aha: Lazy learning. Kluwer Academic Publishers, Norwell 1997, ISBN 0-7923-4584-3.</li>
<li>Atkeson, Christopher G.; Moore, Andrew W.; Schaal, Stefan (1 February 1997). "Locally Weighted Learning for Control". <i>Artificial Intelligence Review</i>. 11 (1): 75–113. <a href="/facts/Doi_(identifier)/muM9Etpq">doi</a>:<a href="https://doi.org/10.1023%2FA%3A1006511328852">10.1023/A:1006511328852</a>. <a href="/facts/S2CID_(identifier)/ldJsHa2Y">S2CID</a> <a href="https://api.semanticscholar.org/CorpusID:3694612">3694612</a>.</li>
<li>Bontempi, Gianluca; Birattari, Mauro; Bersini, Hugues (1999). <a href="https://www.tandfonline.com/doi/abs/10.1080/002071799220830">"Lazy Learning for Local Modeling and Control Design"</a>. <i>International Journal of Control</i>. 72 (7): 643–658. <a href="/facts/Doi_(identifier)/muM9Etpq">doi</a>:<a href="https://doi.org/10.1080%2F002071799220830">10.1080/002071799220830</a>.</li>
<li>Aha, David W.; Kibler, Dennis; Albert, Marc K. (1 January 1991). <a href="https://doi.org/10.1007/BF00153759">"Instance-based learning algorithms"</a>. <i>Machine Learning</i>. 6 (1): 37–66. <a href="/facts/Doi_(identifier)/muM9Etpq">doi</a>:<a href="https://doi.org/10.1007%2FBF00153759">10.1007/BF00153759</a>.</li></ul>

<h2 id="references">References</h2>

<ol>
<li id="fn:1"><p>Aha, David (29 June 2013). Lazy Learning (illustrated ed.). Springer Science & Business Media, 2013. p. 424. ISBN 978-9401720533. Retrieved 30 September 2021. <a href="978-9401720533" target="_blank">978-9401720533</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li>
<li id="fn:2"><p>Tamrakar, Preeti; Roy, Siddharth Singha; Satapathy, Biswajit; Ibrahim, S. P. Syed (2019). Integration of lazy learning associative classification with kNN algorithm. pp. 1–4. doi:10.1109/ViTECoN.2019.8899415. ISBN 978-1-5386-9353-7. <a href="978-1-5386-9353-7" target="_blank">978-1-5386-9353-7</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li>
<li id="fn:3"><p>Bontempi, Gianluca; Birattari, Mauro; Bersini, Hugues (1 January 1999). "Lazy learning for local modelling and control design". International Journal of Control. 72 (7–8): 643–658. doi:10.1080/002071799220830. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li>
<li id="fn:4"><p>Sammut, Claude; Webb, Geoffrey I. (2011). Encyclopedia of Machine Learning. New York: Springer Science & Business Media. p. 572. ISBN 9780387307688. <a href="9780387307688" target="_blank">9780387307688</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></p></li>
<li id="fn:5"><p>Pal, Saurabh (2017-11-02). Data Mining Applications. A Comparative Study for Predicting Student's Performance. GRIN Verlag. ISBN 9783668561458. <a href="9783668561458" target="_blank">9783668561458</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></p></li>
<li id="fn:6"><p>Loncarevic, Zvezdan; Simonic, Mihael; Ude, Ales; Gams, Andrej (2022). Combining Reinforcement Learning and Lazy Learning for Faster Few-Shot Transfer Learning. pp. 285–290. doi:10.1109/Humanoids53995.2022.10000095. ISBN 979-8-3503-0979-9. <a href="979-8-3503-0979-9" target="_blank">979-8-3503-0979-9</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></p></li>
<li id="fn:7"><p>Aha, David W. (2013). Lazy Learning. Berlin: Springer Science & Business Media. p. 106. ISBN 9789401720533. <a href="9789401720533" target="_blank">9789401720533</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></p></li>
</ol>

Lazy learning open-in-new

Lazy learning