Menu
Home
People
Places
Arts
History
Plants & Animals
Science
Life & Culture
Technology
Reference.org
CatBoost
open-in-new
<h2 id="features">Features</h2> <p>CatBoost has gained popularity compared to other gradient boosting algorithms primarily due to the following features<a class="footnote-ref" id="fnref:11" href="#fn:11"><sup>11</sup></a> </p> <ul><li>Native handling for categorical features<a class="footnote-ref" id="fnref:12" href="#fn:12"><sup>12</sup></a></li> <li>Fast GPU training<a class="footnote-ref" id="fnref:13" href="#fn:13"><sup>13</sup></a></li> <li>Visualizations and tools for model and feature analysis</li> <li>Using <a href="/facts/Oblivious_data_structure/1AGRfhVB">oblivious</a> trees or symmetric trees for faster execution</li> <li>Ordered boosting to overcome overfitting<a class="footnote-ref" id="fnref:14" href="#fn:14"><sup>14</sup></a></li></ul> <h2 id="history">History</h2> <p>In 2009 Andrey Gulin developed <a href="/facts/MatrixNet/k4UEEyuN">MatrixNet</a>, a proprietary gradient boosting library that was used in Yandex to rank search results. Since 2009 MatrixNet has been used in different projects in Yandex, including recommendation systems and weather prediction. </p><p>In 2014–2015 Andrey Gulin with a team of researchers has started a new project called Tensornet that was aimed at solving the problem of "how to work with <a href="/facts/Categorical_variable/3UQNkLqs">categorical data</a>". It resulted in several proprietary Gradient Boosting libraries with different approaches to handling categorical data. </p><p>In 2016 Machine Learning Infrastructure team led by Anna Dorogush started working on Gradient Boosting in Yandex, including Matrixnet and Tensornet. They implemented and open-sourced the next version of Gradient Boosting library called CatBoost, which has support of categorical and text data, GPU training, model analysis, visualization tools. </p><p>CatBoost was open-sourced in July 2017 and is under active development in Yandex and the open-source community. </p> <h2 id="application">Application</h2> <ul><li><a href="/facts/JetBrains/1YcNFxgv">JetBrains</a> uses CatBoost for code completion<a class="footnote-ref" id="fnref:15" href="#fn:15"><sup>15</sup></a></li> <li><a href="/facts/Cloudflare/CmQr5dy4">Cloudflare</a> uses CatBoost for bot detection<a class="footnote-ref" id="fnref:16" href="#fn:16"><sup>16</sup></a></li> <li><a href="/facts/Careem/7cR8JG8V">Careem</a> uses CatBoost to predict future destinations of the rides<a class="footnote-ref" id="fnref:17" href="#fn:17"><sup>17</sup></a></li></ul> <h2 id="see-also">See also</h2> <ul> <li>Free and open-source software portal</li></ul> <ul><li><a href="/facts/Machine_learning/e0w0XJTu">Machine learning</a></li> <li><a href="/facts/Gradient_boosting/rm99gPyH">Gradient boosting</a></li> <li><a href="/facts/XGBoost/YupgXmfH">XGBoost</a></li> <li><a href="/facts/LightGBM/gGX9MIwv">LightGBM</a></li> <li><a href="/facts/Scikit-learn/HqxDHRMJ">scikit-learn</a></li></ul> <h2 id="external-links">External links</h2> <ul><li><a href="https://catboost.ai/">CatBoost</a></li> <li><a href="https://github.com/catboost/catboost">GitHub - catboost/catboost</a></li> <li><a href="https://tech.yandex.com/catboost/">CatBoost - Yandex Technology</a></li></ul> <h2 id="references">References</h2> <ol> <li id="fn:1"><p>"catboost/catboost". August 30, 2020 – via GitHub. <a href="https://github.com/catboost/catboost" target="_blank">https://github.com/catboost/catboost</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li> <li id="fn:2"><p>Prokhorenkova, Liudmila; Gusev, Gleb; Vorobev, Aleksandr; Dorogush, Anna Veronika; Gulin, Andrey (2019-01-20). "CatBoost: unbiased boosting with categorical features". arXiv:1706.09516 [cs.LG]. <a href="/wiki/ArXiv_(identifier)" target="_blank">/wiki/ArXiv_(identifier)</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li> <li id="fn:3"><p>"Python Package Index PYPI: catboost". Retrieved 2020-08-20. <a href="https://pypi.python.org/pypi/catboost/" target="_blank">https://pypi.python.org/pypi/catboost/</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li> <li id="fn:4"><p>"Conda force package catboost-r". Retrieved 2020-08-30. <a href="https://anaconda.org/conda-forge/r-catboost" target="_blank">https://anaconda.org/conda-forge/r-catboost</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></p></li> <li id="fn:5"><p>"Maven Repository: ai.catboost » catboost-prediction". mvnrepository.com. Retrieved 2020-08-30. <a href="https://mvnrepository.com/artifact/ai.catboost/catboost-prediction" target="_blank">https://mvnrepository.com/artifact/ai.catboost/catboost-prediction</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></p></li> <li id="fn:6"><p>"catboost/catboost". August 30, 2020 – via GitHub. <a href="https://github.com/catboost/catboost" target="_blank">https://github.com/catboost/catboost</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></p></li> <li id="fn:7"><p>staff, InfoWorld (27 September 2017). "Bossie Awards 2017: The best machine learning tools". InfoWorld. <a href="https://www.infoworld.com/article/3228224/bossie-awards-2017-the-best-machine-learning-tools.html" target="_blank">https://www.infoworld.com/article/3228224/bossie-awards-2017-the-best-machine-learning-tools.html</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></p></li> <li id="fn:8"><p>"State of Data Science and Machine Learning 2020". <a href="https://www.kaggle.com/kaggle-survey-2020" target="_blank">https://www.kaggle.com/kaggle-survey-2020</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></p></li> <li id="fn:9"><p>"State of Data Science and Machine Learning 2021". <a href="https://www.kaggle.com/kaggle-survey-2021" target="_blank">https://www.kaggle.com/kaggle-survey-2021</a> <a href="#fnref:9" class="footnote-back-ref">↩</a></p></li> <li id="fn:10"><p>"PyPI Stats catboost". PyPI Stats. <a href="https://pypistats.org/packages/catboost" target="_blank">https://pypistats.org/packages/catboost</a> <a href="#fnref:10" class="footnote-back-ref">↩</a></p></li> <li id="fn:11"><p>Joseph, Manu (2020-02-29). "The Gradient Boosters V: CatBoost". Deep & Shallow. Retrieved 2020-08-30. <a href="https://deep-and-shallow.com/2020/02/29/the-gradient-boosters-v-catboost/" target="_blank">https://deep-and-shallow.com/2020/02/29/the-gradient-boosters-v-catboost/</a> <a href="#fnref:11" class="footnote-back-ref">↩</a></p></li> <li id="fn:12"><p>Dorogush, Anna Veronika; Ershov, Vasily; Gulin, Andrey (2018-10-24). "CatBoost: gradient boosting with categorical features support". arXiv:1810.11363 [cs.LG]. <a href="/wiki/ArXiv_(identifier)" target="_blank">/wiki/ArXiv_(identifier)</a> <a href="#fnref:12" class="footnote-back-ref">↩</a></p></li> <li id="fn:13"><p>"CatBoost Enables Fast Gradient Boosting on Decision Trees Using GPUs". NVIDIA Developer Blog. 2018-12-13. Retrieved 2020-08-30. <a href="https://developer.nvidia.com/blog/catboost-fast-gradient-boosting-decision-trees/" target="_blank">https://developer.nvidia.com/blog/catboost-fast-gradient-boosting-decision-trees/</a> <a href="#fnref:13" class="footnote-back-ref">↩</a></p></li> <li id="fn:14"><p>Prokhorenkova, Liudmila; Gusev, Gleb; Vorobev, Aleksandr; Dorogush, Anna Veronika; Gulin, Andrey (2019-01-20). "CatBoost: unbiased boosting with categorical features". arXiv:1706.09516 [cs.LG]. <a href="/wiki/ArXiv_(identifier)" target="_blank">/wiki/ArXiv_(identifier)</a> <a href="#fnref:14" class="footnote-back-ref">↩</a></p></li> <li id="fn:15"><p>"Code Completion, Episode 4: Model Training". JetBrains Developer Blog. 2021-08-20. <a href="https://blog.jetbrains.com/blog/2021/08/20/code-completion-episode-4-model-training/" target="_blank">https://blog.jetbrains.com/blog/2021/08/20/code-completion-episode-4-model-training/</a> <a href="#fnref:15" class="footnote-back-ref">↩</a></p></li> <li id="fn:16"><p>"Stop the Bots: Practical Lessons in Machine Learning". The Cloudflare Blog. 2019-02-20. <a href="https://blog.cloudflare.com/stop-the-bots-practical-lessons-in-machine-learning/" target="_blank">https://blog.cloudflare.com/stop-the-bots-practical-lessons-in-machine-learning/</a> <a href="#fnref:16" class="footnote-back-ref">↩</a></p></li> <li id="fn:17"><p>"How Careem's Destination Prediction Service speeds up your ride". Careem. 2019-02-19. <a href="https://blog.careem.com/en/careems-destination-prediction-service/" target="_blank">https://blog.careem.com/en/careems-destination-prediction-service/</a> <a href="#fnref:17" class="footnote-back-ref">↩</a></p></li> </ol>