Query understanding

<h2 id="methods">Methods</h2>
<h3>Stemming and lemmatization</h3>
<p>Many languages <a href="/facts/Inflection/mhXSXpmJ">inflect</a> words to reflect their role in the utterance they appear in. The variation between various forms of a word is likely to be of little importance for the relatively coarse-grained model of meaning involved in a retrieval system, and for this reason the task of conflating the various forms of a word is a potentially useful technique to increase recall of a retrieval system.<a class="footnote-ref" id="fnref:2" href="#fn:2"><sup>2</sup></a>
</p><p><a href="/facts/Stemming/lqtKp4X7">Stemming</a> algorithms, also known as stemmers, typically use a collection of simple rules to remove <a href="/facts/Suffix/Nlnft3NK">suffixes</a> intended to model the language’s inflection rules.<a class="footnote-ref" id="fnref:3" href="#fn:3"><sup>3</sup></a>
</p><p>For some languages, there are simple <a href="/facts/Lemmatisation/eGz6L9yE">lemmatisation</a> methods to reduce a word in query to its <a href="/facts/Lemma_(morphology)/W4d8jVmI">lemma</a> or <a href="/facts/Root_(linguistics)/VwkvmJxn">root</a> form or its <a href="/facts/Word_stem/7Aq8G5fP">stem</a>; for others, this operation involves non-trivial string processing and may require recognizing the word's <a href="/facts/Part_of_speech/WbQ89MHX">part of speech</a> or referencing a  <a href="/facts/Lexical_database/yCwD4rqz">lexical database</a>.
</p><p>The effectiveness of stemming and lemmatization varies across languages.
<a class="footnote-ref" id="fnref:4" href="#fn:4"><sup>4</sup></a>
<a class="footnote-ref" id="fnref:5" href="#fn:5"><sup>5</sup></a>
</p>
<h3>Query Segmentation</h3>
<p>Query segmentation is a key component of query understanding, aiming to divide a query into meaningful segments. Traditional approaches, such as the <a href="/facts/Bag-of-words_model/TlhIx600">bag-of-words model</a>, treat individual words as independent units, which can limit interpretative accuracy. For languages like Chinese, where words are not separated by spaces, segmentation is essential, as individual characters often lack standalone meaning. Even in English, the BOW model may not capture the full meaning, as certain phrases—such as "New York"—carry significance as a whole rather than as isolated terms. By identifying phrases or entities within queries, query segmentation enhances interpretation, enabling search engines to apply proximity and ordering constraints, ultimately improving search accuracy and user satisfaction.<a class="footnote-ref" id="fnref:6" href="#fn:6"><sup>6</sup></a>
</p>
<h3>Entity recognition</h3>
<p>Entity recognition is the process of locating and classifying entities within a text string. <a href="/facts/Named-entity_recognition/ko6GRQQd">Named-entity recognition</a> specifically focuses on <a href="/facts/Named_entity/pqC6pucA">named entities</a>, such as names of people, places, and organizations. In addition, entity recognition includes identifying concepts in queries that may be represented by multi-word phrases. Entity recognition systems typically use grammar-based linguistic techniques or statistical <a href="/facts/Machine_learning/e0w0XJTu">machine learning</a> models.<a class="footnote-ref" id="fnref:7" href="#fn:7"><sup>7</sup></a>
</p>
<h3>Query rewriting</h3>
<p>Query rewriting is the process of automatically reformulating a search query to more accurately capture its intent. <a href="/facts/Query_expansion/poSUBUTD">Query expansion</a> adds additional query terms, such as synonyms, in order to retrieve more documents and thereby increase recall. Query relaxation removes query terms to reduce the requirements for a document to match the query, thereby also increasing <a href="/facts/Precision_and_recall/Jnmmpl8H">recall</a>. Other forms of query rewriting, such as automatically converting consecutive query terms into <a href="/facts/Phrase/mbnD0ksH">phrases</a> and restricting query terms to specific <a href="/facts/Field_(computer_science)/jt622EDb">fields</a>, aim to increase <a href="/facts/Precision_and_recall/Jnmmpl8H">precision</a>.
</p>
<h3>Spelling Correction</h3>
<p>Automatic <a href="/facts/Spell_checker/Dg3GUjUd">spelling correction</a> is a critical feature of modern search engines, designed to address common spelling errors in user queries. Such errors are especially frequent as users often search for unfamiliar topics. By correcting misspelled queries, search engines enhance their understanding of user intent, thereby improving the relevance and quality of search results and overall user experience.<a class="footnote-ref" id="fnref:8" href="#fn:8"><sup>8</sup></a>
</p>
<h2 id="external-links">External links</h2>
<ul><li><a href="https://web.archive.org/web/20180710071222/https://pdfs.semanticscholar.org/97ae/21a9f3b2781420adaba8e637b9f911511ab2.pdf">Proceedings of ACM SIGIR 2011 Workshop on Query Representation and Understanding</a></li>
<li><a href="https://www.springer.com/gp/book/9783030583330">Query Understanding for Search Engines (Yi Chang and Hongbo Deng, Eds.)</a></li></ul>

<h2 id="references">References</h2>

<ol>
<li id="fn:1"><p>"Association for Computing Machinery (ACM) Special Interest Group on Information Retrieval (SIGIR) 2010 Workshop on Query Representation and Understanding" (PDF). <a href="http://www.sigir.org/files/forum/2010D/sigirwksp/2010d_sigirforum_croft.pdf" target="_blank">http://www.sigir.org/files/forum/2010D/sigirwksp/2010d_sigirforum_croft.pdf</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li>
<li id="fn:2"><p>Lowe, Thomas; Roberts, David; Kurtz, Peterdate=1973. Additional Text Processing for On-Line Retrieval (The RADCOL System). Volume 1. DTIC Document.{{cite book}}:  CS1 maint: numeric names: authors list (link)
Lennon, Martin; Peirce, David; Tarry, Brian D; Willett, Peter (1981). "An evaluation of some conflation algorithms for information retrieval". Information Scientist. 3 (4). SAGE. <a href="/wiki/Template:Cite_book" target="_blank">/wiki/Template:Cite_book</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li>
<li id="fn:3"><p>Lovins, Julie (1968). Development of a stemming algorithm. MIT Information Processing Group. <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li>
<li id="fn:4"><p>Harman, Donna (1991). "How Effective is Suffixing?". Journal of the American Society for Information Science. 42 (1): 7–15. doi:10.1002/(SICI)1097-4571(199101)42:1<7::AID-ASI2>3.0.CO;2-P. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></p></li>
<li id="fn:5"><p>Popovic, Mirkoc; Willett, Peter (1981). "The effectiveness of stemming for natural-language access to Slovene textual data". Information Scientist. 3 (4). SAGE. <a href="#fnref:5" class="footnote-back-ref">↩</a></p></li>
<li id="fn:6"><p>Li, Hang; Xu, Jun; Zhang, Min (2021). Query Understanding for Search Engines. Springer. <a href="#fnref:6" class="footnote-back-ref">↩</a></p></li>
<li id="fn:7"><p>"A Survey of Named Entity Recognition and Classification" (PDF). <a href="http://nlp.cs.nyu.edu/sekine/papers/li07.pdf" target="_blank">http://nlp.cs.nyu.edu/sekine/papers/li07.pdf</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></p></li>
<li id="fn:8"><p>Li, Hang; Xu, Jun; Zhang, Min (2021). Query Understanding for Search Engines. Springer. <a href="#fnref:8" class="footnote-back-ref">↩</a></p></li>
</ol>

Query understanding open-in-new

Query understanding