Zero-shot learning

<h2 id="background-and-history">Background and history</h2>
The first paper on zero-shot learning in natural language processing appeared in a 2008 paper by Chang, Ratinov, Roth, and Srikumar, at the <a href="/facts/Association_for_the_Advancement_of_Artificial_Intelligence/BjScke2s">AAAI’08</a>, but the name given to the learning paradigm there was dataless classification.<a class="footnote-ref" id="fnref:3" href="#fn:3">3</a> The first paper on zero-shot learning in computer vision appeared at the same conference, under the name zero-data learning.<a class="footnote-ref" id="fnref:4" href="#fn:4">4</a> The term zero-shot learning itself first appeared in the literature in a 2009 paper from Palatucci, Hinton, Pomerleau, and Mitchell at <a href="/facts/Neural_Information_Processing_Systems/YlhI2Atl">NIPS’09</a>.<a class="footnote-ref" id="fnref:5" href="#fn:5">5</a> This terminology was repeated later in another computer vision paper<a class="footnote-ref" id="fnref:6" href="#fn:6">6</a> and the term zero-shot learning caught on, as a take-off on <a href="/facts/One-shot_learning_in_computer_vision/CgIw6JSz">one-shot learning that was introduced in computer vision</a> years earlier.<a class="footnote-ref" id="fnref:7" href="#fn:7">7</a> 
In computer vision, zero-shot learning models learned parameters for seen classes along with their class representations and rely on representational similarity among class labels so that, during inference, instances can be classified into new classes.
In natural language processing, the key technical direction developed builds on the ability to "understand the labels"—represent the labels in the same semantic space as that of the documents to be classified. This supports the classification of a single example without observing any annotated data, the purest form of zero-shot classification. The original paper<a class="footnote-ref" id="fnref:8" href="#fn:8">8</a> made use of the <a href="/facts/Explicit_semantic_analysis/UGQztsMB">Explicit Semantic Analysis</a> (ESA) representation but later papers made use of other representations, including dense representations. This approach was also extended to multilingual domains,<a class="footnote-ref" id="fnref:9" href="#fn:9">9</a><a class="footnote-ref" id="fnref:10" href="#fn:10">10</a> fine entity typing<a class="footnote-ref" id="fnref:11" href="#fn:11">11</a> and other problems. Moreover, beyond relying solely on representations, the computational approach has been extended to depend on transfer from other tasks, such as <a href="/facts/Textual_entailment/6c93bq9J">textual entailment</a><a class="footnote-ref" id="fnref:12" href="#fn:12">12</a> and question answering.<a class="footnote-ref" id="fnref:13" href="#fn:13">13</a>
The original paper<a class="footnote-ref" id="fnref:14" href="#fn:14">14</a> also points out that, beyond the ability to classify a single example, when a collection of examples is given, with the assumption that they come from the same distribution, it is possible to bootstrap the performance in a semi-supervised like manner (or <a href="/facts/Transduction_(machine_learning)/YBMCKGfo">transductive learning</a>). 
Unlike standard <a href="/facts/Generalization_error/dy5YSgB8">generalization</a> in machine learning, where classifiers are expected to correctly classify new samples to classes they have already observed during training, in ZSL, no samples from the classes have been given during training the classifier. It can therefore be viewed as an extreme case of <a href="/facts/Domain_adaptation/eq5cbR9K">domain adaptation</a>.

<h2 id="prerequisite-information-for-zero-shot-classes">Prerequisite information for zero-shot classes</h2>
Naturally, some form of auxiliary information has to be given about these zero-shot classes, and this type of information can be of several types. 

<ul><li>Learning with attributes: classes are accompanied by pre-defined structured description. For example, for bird descriptions, this could include "red head", "long beak".<a class="footnote-ref" id="fnref:15" href="#fn:15">15</a><a class="footnote-ref" id="fnref:16" href="#fn:16">16</a> These attributes are often organized in a structured compositional way, and taking that structure into account improves learning.<a class="footnote-ref" id="fnref:17" href="#fn:17">17</a> While this approach was used mostly in computer vision, there are some examples for it also in natural language processing.<a class="footnote-ref" id="fnref:18" href="#fn:18">18</a></li>
<li>Learning from textual description. As pointed out above, this has been the key direction pursued in natural language processing. Here class labels are taken to have a meaning and are often augmented with definitions or free-text natural-language description. This could include for example a wikipedia description of the class.<a class="footnote-ref" id="fnref:19" href="#fn:19">19</a><a class="footnote-ref" id="fnref:20" href="#fn:20">20</a><a class="footnote-ref" id="fnref:21" href="#fn:21">21</a></li>
<li>Class-class similarity. Here, classes are embedded in a continuous space. A zero-shot classifier can predict that a sample corresponds to some position in that space, and the nearest embedded class is used as a predicted class, even if no such samples were observed during training.<a class="footnote-ref" id="fnref:22" href="#fn:22">22</a></li></ul>
<h2 id="generalized-zero-shot-learning">Generalized zero-shot learning</h2>
The above ZSL setup assumes that at test time, only zero-shot samples are given, namely, samples from new unseen classes. In generalized zero-shot learning, samples from both new and known classes, may appear at test time. This poses new challenges for classifiers at test time, because it is very challenging to estimate if a given sample is new or known. Some approaches to handle this include: 

<ul><li>a gating module, which is first trained to decide if a given sample comes from a new class or from an old one, and then, at inference time, outputs either a hard decision,<a class="footnote-ref" id="fnref:23" href="#fn:23">23</a> or a soft probabilistic decision<a class="footnote-ref" id="fnref:24" href="#fn:24">24</a></li>
<li>a generative module, which is trained to generate feature representation of the unseen classes--a standard classifier can then be trained on samples from all classes, seen and unseen.<a class="footnote-ref" id="fnref:25" href="#fn:25">25</a></li></ul>
<h2 id="domains-of-application">Domains of application</h2>
Zero shot learning has been applied to the following fields:

<ul><li><a href="/facts/Image_classification/Tl2Yyk66">image classification</a></li>
<li><a href="/facts/Semantic_segmentation/jofAhbxa">semantic segmentation</a></li>
<li><a href="/facts/Artificial_intelligence_art/d591uR6T">image generation</a></li>
<li><a href="/facts/Object_detection/sCx3FK44">object detection</a></li>
<li><a href="/facts/Natural_language_processing/1hjMKsSN">natural language processing</a></li>
<li><a href="/facts/Computational_biology/GxgNfwcX">computational biology</a><a class="footnote-ref" id="fnref:26" href="#fn:26">26</a></li></ul>
<h2 id="see-also">See also</h2>
<ul><li><a href="/facts/One-shot_learning_in_computer_vision/CgIw6JSz">One-shot learning in computer vision</a></li>
<li><a href="/facts/Transfer_learning/pNz4P2KP">Transfer learning</a></li>
<li><a href="/facts/Fast_mapping/gl7WHWWx">Fast mapping</a></li>
<li><a href="/facts/Explanation-based_learning/c6N3kEhE">Explanation-based learning</a></li></ul>

<h2 id="references">References</h2>

<ol>
<li id="fn:1">Xian, Yongqin; Lampert, Christoph H.; Schiele, Bernt; Akata, Zeynep (2020-09-23). "Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly". arXiv:1707.00600 [cs.CV]. <a href="/wiki/ArXiv_(identifier)" target="_blank">/wiki/ArXiv_(identifier)</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></li>
<li id="fn:2">Xian, Yongqin; Schiele, Bernt; Akata, Zeynep (2017). "Zero-shot learning-the good, the bad and the ugly". Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition: 4582–4591. arXiv:1703.04394. Bibcode:2017arXiv170304394X. <a href="/wiki/ArXiv_(identifier)" target="_blank">/wiki/ArXiv_(identifier)</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></li>
<li id="fn:3">Chang, M.W. (2008). "Importance of Semantic Representation: Dataless Classification". AAAI. <a href="https://citeseerx.ist.psu.edu/document?doi=ee0a332b4fc1e82a9999acd6cebceb165dc8645b" target="_blank">https://citeseerx.ist.psu.edu/document?doi=ee0a332b4fc1e82a9999acd6cebceb165dc8645b</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></li>
<li id="fn:4">Larochelle, Hugo (2008). "Zero-data Learning of New Tasks" (PDF). <a href="https://www.aaai.org/Papers/AAAI/2008/AAAI08-103.pdf" target="_blank">https://www.aaai.org/Papers/AAAI/2008/AAAI08-103.pdf</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></li>
<li id="fn:5">Palatucci, Mark (2009). "Zero-Shot Learning with Semantic Output Codes" (PDF). NIPS. <a href="https://www.cs.toronto.edu/~hinton/absps/palatucci.pdf" target="_blank">https://www.cs.toronto.edu/~hinton/absps/palatucci.pdf</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></li>
<li id="fn:6">Lampert, C.H. (2009). "Learning to detect unseen object classes by between-class attribute transfer". IEEE Conference on Computer Vision and Pattern Recognition: 951–958. CiteSeerX 10.1.1.165.9750. <a href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.165.9750&rep=rep1&type=pdf" target="_blank">http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.165.9750&rep=rep1&type=pdf</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></li>
<li id="fn:7">Miller, E. G. (2000). "Learning from One Example Through Shared Densities on Transforms" (PDF). CVPR. <a href="https://people.cs.umass.edu/~elm/papers/cvpr2000.pdf" target="_blank">https://people.cs.umass.edu/~elm/papers/cvpr2000.pdf</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></li>
<li id="fn:8">Chang, M.W. (2008). "Importance of Semantic Representation: Dataless Classification". AAAI. <a href="https://citeseerx.ist.psu.edu/document?doi=ee0a332b4fc1e82a9999acd6cebceb165dc8645b" target="_blank">https://citeseerx.ist.psu.edu/document?doi=ee0a332b4fc1e82a9999acd6cebceb165dc8645b</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></li>
<li id="fn:9">Song, Yangqiu (2019). "Toward any-language zero-shot topic classification of textual documents". Artificial Intelligence. 274: 133–150. doi:10.1016/j.artint.2019.02.002. <a href="https://doi.org/10.1016%2Fj.artint.2019.02.002" target="_blank">https://doi.org/10.1016%2Fj.artint.2019.02.002</a> <a href="#fnref:9" class="footnote-back-ref">↩</a></li>
<li id="fn:10">Song, Yangqiu (2016). "Cross-Lingual Dataless Classification for Many Languages" (PDF). IJCAI. <a href="https://www.ijcai.org/Proceedings/16/Papers/412.pdf" target="_blank">https://www.ijcai.org/Proceedings/16/Papers/412.pdf</a> <a href="#fnref:10" class="footnote-back-ref">↩</a></li>
<li id="fn:11">Zhou, Ben (2018). "Zero-Shot Open Entity Typing as Type-Compatible Grounding" (PDF). EMNLP. arXiv:1907.03228. <a href="https://www.aclweb.org/anthology/D18-1231.pdf" target="_blank">https://www.aclweb.org/anthology/D18-1231.pdf</a> <a href="#fnref:11" class="footnote-back-ref">↩</a></li>
<li id="fn:12">Yin, Wenpeng (2019). "Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach" (PDF). EMNLP. arXiv:1909.00161. <a href="https://www.aclweb.org/anthology/D19-1404.pdf" target="_blank">https://www.aclweb.org/anthology/D19-1404.pdf</a> <a href="#fnref:12" class="footnote-back-ref">↩</a></li>
<li id="fn:13">Levy, Omer (2017). "Zero-Shot Relation Extraction via Reading Comprehension" (PDF). CoNLL. arXiv:1706.04115. <a href="https://www.aclweb.org/anthology/K17-1034.pdf" target="_blank">https://www.aclweb.org/anthology/K17-1034.pdf</a> <a href="#fnref:13" class="footnote-back-ref">↩</a></li>
<li id="fn:14">Chang, M.W. (2008). "Importance of Semantic Representation: Dataless Classification". AAAI. <a href="https://citeseerx.ist.psu.edu/document?doi=ee0a332b4fc1e82a9999acd6cebceb165dc8645b" target="_blank">https://citeseerx.ist.psu.edu/document?doi=ee0a332b4fc1e82a9999acd6cebceb165dc8645b</a> <a href="#fnref:14" class="footnote-back-ref">↩</a></li>
<li id="fn:15">Lampert, C.H. (2009). "Learning to detect unseen object classes by between-class attribute transfer". IEEE Conference on Computer Vision and Pattern Recognition: 951–958. CiteSeerX 10.1.1.165.9750. <a href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.165.9750&rep=rep1&type=pdf" target="_blank">http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.165.9750&rep=rep1&type=pdf</a> <a href="#fnref:15" class="footnote-back-ref">↩</a></li>
<li id="fn:16">Romera-Paredes, Bernardino; Torr, Phillip (2015). "An embarrassingly simple approach to zero-shot learning" (PDF). International Conference on Machine Learning: 2152–2161. <a href="http://www.jmlr.org/proceedings/papers/v37/romera-paredes15.pdf" target="_blank">http://www.jmlr.org/proceedings/papers/v37/romera-paredes15.pdf</a> <a href="#fnref:16" class="footnote-back-ref">↩</a></li>
<li id="fn:17">Atzmon, Yuval; Chechik, Gal (2018). "Probabilistic AND-OR Attribute Grouping for Zero-Shot Learning" (PDF). Uncertainty in Artificial Intelligence. arXiv:1806.02664. Bibcode:2018arXiv180602664A. <a href="http://auai.org/uai2018/proceedings/papers/151.pdf" target="_blank">http://auai.org/uai2018/proceedings/papers/151.pdf</a> <a href="#fnref:17" class="footnote-back-ref">↩</a></li>
<li id="fn:18">Roth, Dan (2009). "Aspect Guided Text Categorization with Unobserved Labels". ICDM. CiteSeerX 10.1.1.148.9946. <a href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.148.9946&rep=rep1&type=pdf" target="_blank">http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.148.9946&rep=rep1&type=pdf</a> <a href="#fnref:18" class="footnote-back-ref">↩</a></li>
<li id="fn:19">Zhou, Ben (2018). "Zero-Shot Open Entity Typing as Type-Compatible Grounding" (PDF). EMNLP. arXiv:1907.03228. <a href="https://www.aclweb.org/anthology/D18-1231.pdf" target="_blank">https://www.aclweb.org/anthology/D18-1231.pdf</a> <a href="#fnref:19" class="footnote-back-ref">↩</a></li>
<li id="fn:20">Hu, R Lily; Xiong, Caiming; Socher, Richard (2018). "Zero-Shot Image Classification Guided by Natural Language Descriptions of Classes: A Meta-Learning Approach" (PDF). NeurIPS. <a href="https://nips2018vigil.github.io/static/papers/accepted/28.pdf" target="_blank">https://nips2018vigil.github.io/static/papers/accepted/28.pdf</a> <a href="#fnref:20" class="footnote-back-ref">↩</a></li>
<li id="fn:21">Srivastava, Shashank; Labutov, Igor; Mitchelle, Tom (2018). "Zero-shot Learning of Classifiers from Natural Language Quantification". Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 306–316. doi:10.18653/v1/P18-1029. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:21" class="footnote-back-ref">↩</a></li>
<li id="fn:22">Frome, Andrea; et, al (2013). "Devise: A deep visual-semantic embedding model" (PDF). Advances in Neural Information Processing Systems: 2121–2129. <a href="https://papers.nips.cc/paper/5204-devise-a-deep-visual-semantic-embedding-model.pdf" target="_blank">https://papers.nips.cc/paper/5204-devise-a-deep-visual-semantic-embedding-model.pdf</a> <a href="#fnref:22" class="footnote-back-ref">↩</a></li>
<li id="fn:23">Socher, R; Ganjoo, M; Manning, C.D.; Ng, A. (2013). "Zero-shot learning through cross-modal transfer". Neural Information Processing Systems. arXiv:1301.3666. Bibcode:2013arXiv1301.3666S. <a href="/wiki/ArXiv_(identifier)" target="_blank">/wiki/ArXiv_(identifier)</a> <a href="#fnref:23" class="footnote-back-ref">↩</a></li>
<li id="fn:24">Atzmon, Yuval (2019). "Adaptive Confidence Smoothing for Generalized Zero-Shot Learning". The IEEE Conference on Computer Vision and Pattern Recognition: 11671–11680. arXiv:1812.09903. Bibcode:2018arXiv181209903A. <a href="/wiki/ArXiv_(identifier)" target="_blank">/wiki/ArXiv_(identifier)</a> <a href="#fnref:24" class="footnote-back-ref">↩</a></li>
<li id="fn:25">Felix, R; et, al (2018). "Multi-modal cycle-consistent generalized zero-shot learning". Proceedings of the European Conference on Computer Vision: 21–37. arXiv:1808.00136. Bibcode:2018arXiv180800136F. <a href="/wiki/ArXiv_(identifier)" target="_blank">/wiki/ArXiv_(identifier)</a> <a href="#fnref:25" class="footnote-back-ref">↩</a></li>
<li id="fn:26">Wittmann, Bruce J.; Yue, Yisong; Arnold, Frances H. (2020-12-04). "Machine Learning-Assisted Directed Evolution Navigates a Combinatorial Epistatic Fitness Landscape with Minimal Screening Burden": 2020.12.04.408955. doi:10.1101/2020.12.04.408955. S2CID 227914824. {{cite journal}}: Cite journal requires |journal= (help) <a href="https://www.biorxiv.org/content/10.1101/2020.12.04.408955v1" target="_blank">https://www.biorxiv.org/content/10.1101/2020.12.04.408955v1</a> <a href="#fnref:26" class="footnote-back-ref">↩</a></li>
</ol>

Zero-shot learning open-in-new

Zero-shot learning