Menu
Home
People
Places
Arts
History
Plants & Animals
Science
Life & Culture
Technology
Reference.org
Speech processing
open-in-new
<h2 id="history">History</h2> <p>Early attempts at speech processing and recognition were primarily focused on understanding a handful of simple <a href="/facts/Phonetics/ljAVT8fv">phonetic</a> elements such as vowels. In 1952, three researchers at Bell Labs, Stephen. Balashek, R. Biddulph, and K. H. Davis, developed a system that could recognize digits spoken by a single speaker.<a class="footnote-ref" id="fnref:2" href="#fn:2"><sup>2</sup></a> Pioneering works in field of speech recognition using analysis of its spectrum were reported in the 1940s.<a class="footnote-ref" id="fnref:3" href="#fn:3"><sup>3</sup></a> </p><p><a href="/facts/Linear_predictive_coding/2Bl1xdSN">Linear predictive coding</a> (LPC), a speech processing algorithm, was first proposed by <a href="/facts/Fumitada_Itakura/RnbyhGNI">Fumitada Itakura</a> of <a href="/facts/Nagoya_University/0MABJXSj">Nagoya University</a> and Shuzo Saito of <a href="/facts/Nippon_Telegraph_and_Telephone/a3gI457X">Nippon Telegraph and Telephone</a> (NTT) in 1966.<a class="footnote-ref" id="fnref:4" href="#fn:4"><sup>4</sup></a> Further developments in LPC technology were made by <a href="/facts/Bishnu_S._Atal/bI2098fu">Bishnu S. Atal</a> and <a href="/facts/Manfred_R._Schroeder/0AQPXT8L">Manfred R. Schroeder</a> at <a href="/facts/Bell_Labs/fKP5Gk0H">Bell Labs</a> during the 1970s.<a class="footnote-ref" id="fnref:5" href="#fn:5"><sup>5</sup></a> LPC was the basis for <a href="/facts/Voice-over-IP/5QGBbXYg">voice-over-IP</a> (VoIP) technology,<a class="footnote-ref" id="fnref:6" href="#fn:6"><sup>6</sup></a> as well as <a href="/facts/Speech_synthesizer/ceTczTv9">speech synthesizer</a> chips, such as the <a href="/facts/Texas_Instruments_LPC_Speech_Chips/PCAjLofL">Texas Instruments LPC Speech Chips</a> used in the <a href="/facts/Speak_%26_Spell_(toy)/Fn2LGVxK">Speak & Spell</a> toys from 1978.<a class="footnote-ref" id="fnref:7" href="#fn:7"><sup>7</sup></a> </p><p>One of the first commercially available speech recognition products was Dragon Dictate, released in 1990. In 1992, technology developed by <a href="/facts/Lawrence_Rabiner/Sse3XfHx">Lawrence Rabiner</a> and others at Bell Labs was used by <a href="/facts/AT%26T/y3Lo9uhk">AT&T</a> in their Voice Recognition Call Processing service to route calls without a human operator. By this point, the vocabulary of these systems was larger than the average human vocabulary.<a class="footnote-ref" id="fnref:8" href="#fn:8"><sup>8</sup></a> </p><p>By the early 2000s, the dominant speech processing strategy started to shift away from <a href="/facts/Hidden_Markov_model/ur1zTAhP">Hidden Markov Models</a> towards more modern <a href="/facts/Artificial_neural_network/6V1jMlkx">neural networks</a> and <a href="/facts/Deep_learning/JLuwD3ea">deep learning</a>.<a class="footnote-ref" id="fnref:9" href="#fn:9"><sup>9</sup></a> </p><p>In 2012, <a href="/facts/Geoffrey_Hinton/HJU6lC2H">Geoffrey Hinton</a> and his team at the <a href="/facts/University_of_Toronto/FpGjQNm9">University of Toronto</a> demonstrated that deep neural networks could significantly outperform traditional HMM-based systems on large vocabulary continuous speech recognition tasks. This breakthrough led to widespread adoption of deep learning techniques in the industry.<a class="footnote-ref" id="fnref:10" href="#fn:10"><sup>10</sup></a><a class="footnote-ref" id="fnref:11" href="#fn:11"><sup>11</sup></a> </p><p>By the mid-2010s, companies like <a href="/facts/Google/GT9Sugza">Google</a>, <a href="/facts/Microsoft/nGIDDXdx">Microsoft</a>, <a href="/facts/Amazon_(company)/YzUTUVE8">Amazon</a>, and <a href="/facts/Apple_Inc./oTJYta3k">Apple</a> had integrated advanced speech recognition systems into their virtual assistants such as <a href="/facts/Google_Assistant/ahQuotGz">Google Assistant</a>, <a href="/facts/Cortana_(virtual_assistant)/du1CmE4u">Cortana</a>, <a href="/facts/Amazon_Alexa/deexPRqM">Alexa</a>, and <a href="/facts/Siri/wfkBV2dF">Siri</a>.<a class="footnote-ref" id="fnref:12" href="#fn:12"><sup>12</sup></a> These systems utilized deep learning models to provide more natural and accurate voice interactions. </p><p>The development of Transformer-based models, like Google's BERT (Bidirectional Encoder Representations from Transformers) and OpenAI's GPT (Generative Pre-trained Transformer), further pushed the boundaries of natural language processing and speech recognition. These models enabled more context-aware and semantically rich understanding of speech.<a class="footnote-ref" id="fnref:13" href="#fn:13"><sup>13</sup></a><a class="footnote-ref" id="fnref:14" href="#fn:14"><sup>14</sup></a> In recent years, end-to-end speech recognition models have gained popularity. These models simplify the speech recognition pipeline by directly converting audio input into text output, bypassing intermediate steps like feature extraction and acoustic modeling. This approach has streamlined the development process and improved performance.<a class="footnote-ref" id="fnref:15" href="#fn:15"><sup>15</sup></a> </p> <h2 id="techniques">Techniques</h2> <h3>Dynamic time warping</h3> <p class="note">Main article: <a href="/facts/Dynamic_time_warping/guEHSMw4">Dynamic time warping</a></p><p>Dynamic time warping (DTW) is an <a href="/facts/Algorithm/fnl5NmRt">algorithm</a> for measuring similarity between two <a href="/facts/Time_series/fSXPR817">temporal sequences</a>, which may vary in speed. In general, DTW is a method that calculates an <a href="/facts/Optimal_matching/wdPMohRC">optimal match</a> between two given sequences (e.g. time series) with certain restriction and rules. The optimal match is denoted by the match that satisfies all the restrictions and the rules and that has the minimal cost, where the cost is computed as the sum of absolute differences, for each matched pair of indices, between their values. </p><h3>Hidden Markov models</h3> <p class="note">Main article: <a href="/facts/Hidden_Markov_model/ur1zTAhP">Hidden Markov model</a></p><p>A hidden Markov model can be represented as the simplest <a href="/facts/Dynamic_Bayesian_network/Gm1jeuh2">dynamic Bayesian network</a>. The goal of the algorithm is to estimate a hidden variable x(t) given a list of observations y(t). By applying the <a href="/facts/Markov_property/g1Fs9GFS">Markov property</a>, the <a href="/facts/Conditional_probability_distribution/0eGm3P9W">conditional probability distribution</a> of the hidden variable <i>x</i>(<i>t</i>) at time <i>t</i>, given the values of the hidden variable <i>x</i> at all times, depends <i>only</i> on the value of the hidden variable <i>x</i>(<i>t</i> − 1). Similarly, the value of the observed variable <i>y</i>(<i>t</i>) only depends on the value of the hidden variable <i>x</i>(<i>t</i>) (both at time <i>t</i>). </p><h3>Artificial neural networks</h3> <p class="note">Main article: <a href="/facts/Artificial_neural_network/6V1jMlkx">Artificial neural network</a></p><p>An artificial neural network (ANN) is based on a collection of connected units or nodes called <a href="/facts/Artificial_neuron/TSH6nsgg">artificial neurons</a>, which loosely model the <a href="/facts/Neuron/G7CyTVdH">neurons</a> in a biological <a href="/facts/Brain/EKla9NkJ">brain</a>. Each connection, like the <a href="/facts/Synapse/vdMRJCUn">synapses</a> in a biological <a href="/facts/Brain/EKla9NkJ">brain</a>, can transmit a signal from one artificial neuron to another. An artificial neuron that receives a signal can process it and then signal additional artificial neurons connected to it. In common ANN implementations, the signal at a connection between artificial neurons is a <a href="/facts/Real_number/R02gw5Pb">real number</a>, and the output of each artificial neuron is computed by some non-linear function of the sum of its inputs. </p><h3>Phase-aware processing</h3> <p>Phase is usually supposed to be random uniform variable and thus useless. This is due wrapping of phase:<a class="footnote-ref" id="fnref:16" href="#fn:16"><sup>16</sup></a> result of <a href="/facts/Arctangent/bVwU9FPG">arctangent</a> function is not continuous due to periodical jumps on 2 π {\displaystyle 2\pi } . After phase unwrapping (see,<a class="footnote-ref" id="fnref:17" href="#fn:17"><sup>17</sup></a> Chapter 2.3; <a href="/facts/Instantaneous_phase_and_frequency/fJ17zuan">Instantaneous phase and frequency</a>), it can be expressed as:<a class="footnote-ref" id="fnref:18" href="#fn:18"><sup>18</sup></a><a class="footnote-ref" id="fnref:19" href="#fn:19"><sup>19</sup></a> ϕ ( h , l ) = ϕ l i n ( h , l ) + Ψ ( h , l ) {\displaystyle \phi (h,l)=\phi _{lin}(h,l)+\Psi (h,l)} , where ϕ l i n ( h , l ) = ω 0 ( l ′ ) Δ t {\displaystyle \phi _{lin}(h,l)=\omega _{0}(l'){}_{\Delta }t} is linear phase ( Δ t {\displaystyle {}_{\Delta }t} is temporal shift at each frame of analysis), Ψ ( h , l ) {\displaystyle \Psi (h,l)} is phase contribution of the vocal tract and phase source.<a class="footnote-ref" id="fnref:20" href="#fn:20"><sup>20</sup></a> Obtained phase estimations can be used for noise reduction: temporal smoothing of instantaneous phase <a class="footnote-ref" id="fnref:21" href="#fn:21"><sup>21</sup></a> and its derivatives by time (<a href="/facts/Instantaneous_phase_and_frequency/fJ17zuan">instantaneous frequency</a>) and frequency (<a href="/facts/Group_delay_and_phase_delay/gKWFV26n">group delay</a>),<a class="footnote-ref" id="fnref:22" href="#fn:22"><sup>22</sup></a> smoothing of phase across frequency.<a class="footnote-ref" id="fnref:23" href="#fn:23"><sup>23</sup></a> Joined amplitude and phase estimators can recover speech more accurately basing on assumption of von Mises distribution of phase.<a class="footnote-ref" id="fnref:24" href="#fn:24"><sup>24</sup></a> </p> <h2 id="applications">Applications</h2> <ul><li><a href="/facts/Interactive_voice_response/UJvnBdL7">Interactive voice response</a></li> <li><a href="/facts/Virtual_assistant/BuouwL2U">Virtual Assistants</a></li> <li><a href="/facts/Speaker_recognition/9Td6nPY7">Voice Identification</a></li> <li><a href="/facts/Emotion_recognition/yWGzqfhz">Emotion Recognition</a></li> <li>Call Center Automation</li> <li><a href="/facts/Robotics/wkmvodJz">Robotics</a></li></ul> <h2 id="see-also">See also</h2> <ul><li><a href="/facts/Computational_audiology/9mcAGnTP">Computational audiology</a></li> <li><a href="/facts/Neurocomputational_speech_processing/v68ddgfr">Neurocomputational speech processing</a></li> <li><a href="/facts/Speech_coding/xs5gvMXq">Speech coding</a></li> <li><a href="/facts/Speech_technology/Whm4qE7E">Speech technology</a></li> <li><a href="/facts/Natural_language_processing/1hjMKsSN">Natural Language Processing</a></li></ul> <h2 id="references">References</h2> <ol> <li id="fn:1"><p>Sahidullah, Md; Patino, Jose; Cornell, Samuele; Yin, Ruiking; Sivasankaran, Sunit; Bredin, Herve; Korshunov, Pavel; Brutti, Alessio; Serizel, Romain; Vincent, Emmanuel; Evans, Nicholas; Marcel, Sebastien; Squartini, Stefano; Barras, Claude (2019-11-06). "The Speed Submission to DIHARD II: Contributions & Lessons Learned". arXiv:1911.02388 [eess.AS]. <a href="/wiki/ArXiv_(identifier)" target="_blank">/wiki/ArXiv_(identifier)</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li> <li id="fn:2"><p>Juang, B.-H.; Rabiner, L.R. (2006), "Speech Recognition, Automatic: History", Encyclopedia of Language & Linguistics, Elsevier, pp. 806–819, doi:10.1016/b0-08-044854-2/00906-8, ISBN 9780080448541 <a href="9780080448541" target="_blank">9780080448541</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li> <li id="fn:3"><p>Myasnikov, L. L.; Myasnikova, Ye. N. (1970). Automatic recognition of sound pattern (in Russian). Leningrad: Energiya. <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li> <li id="fn:4"><p>Gray, Robert M. (2010). "A History of Realtime Digital Speech on Packet Networks: Part II of Linear Predictive Coding and the Internet Protocol" (PDF). Found. Trends Signal Process. 3 (4): 203–303. doi:10.1561/2000000036. ISSN 1932-8346. <a href="https://ee.stanford.edu/~gray/lpcip.pdf" target="_blank">https://ee.stanford.edu/~gray/lpcip.pdf</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></p></li> <li id="fn:5"><p>Gray, Robert M. (2010). "A History of Realtime Digital Speech on Packet Networks: Part II of Linear Predictive Coding and the Internet Protocol" (PDF). Found. Trends Signal Process. 3 (4): 203–303. doi:10.1561/2000000036. ISSN 1932-8346. <a href="https://ee.stanford.edu/~gray/lpcip.pdf" target="_blank">https://ee.stanford.edu/~gray/lpcip.pdf</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></p></li> <li id="fn:6"><p>Gray, Robert M. (2010). "A History of Realtime Digital Speech on Packet Networks: Part II of Linear Predictive Coding and the Internet Protocol" (PDF). Found. Trends Signal Process. 3 (4): 203–303. doi:10.1561/2000000036. ISSN 1932-8346. <a href="https://ee.stanford.edu/~gray/lpcip.pdf" target="_blank">https://ee.stanford.edu/~gray/lpcip.pdf</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></p></li> <li id="fn:7"><p>"VC&G - VC&G Interview: 30 Years Later, Richard Wiggins Talks Speak & Spell Development". <a href="http://www.vintagecomputing.com/index.php/archives/528" target="_blank">http://www.vintagecomputing.com/index.php/archives/528</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></p></li> <li id="fn:8"><p>Huang, Xuedong; Baker, James; Reddy, Raj (2014-01-01). "A historical perspective of speech recognition". Communications of the ACM. 57 (1): 94–103. doi:10.1145/2500887. ISSN 0001-0782. S2CID 6175701. <a href="/wiki/Doi_(identifier)" target="_blank">/wiki/Doi_(identifier)</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></p></li> <li id="fn:9"><p>Furui, Sadaoki (2005). "50 Years of Progress in Speech and Speaker Recognition Research". ECTI Transactions on Computer and Information Technology (ECTI-CIT). 1 (2): 64–74. doi:10.37936/ecti-cit.200512.51834. ISSN 2286-9131. <a href="https://ph01.tci-thaijo.org/index.php/saujournalst/www.tci-thaijo.org/index.php/ecticit/article/view/51834" target="_blank">https://ph01.tci-thaijo.org/index.php/saujournalst/www.tci-thaijo.org/index.php/ecticit/article/view/51834</a> <a href="#fnref:9" class="footnote-back-ref">↩</a></p></li> <li id="fn:10"><p>"Deep Neural Networks for Acoustic Modeling in Speech Recognition" (PDF). 2019-07-23. Retrieved 2024-11-05. <a href="https://www.cs.toronto.edu/~hinton/absps/DNN-2012-proof.pdf?form=MG0AV3" target="_blank">https://www.cs.toronto.edu/~hinton/absps/DNN-2012-proof.pdf?form=MG0AV3</a> <a href="#fnref:10" class="footnote-back-ref">↩</a></p></li> <li id="fn:11"><p>"SPEECH RECOGNITION WITH DEEP RECURRENT NEURAL NETWORKS" (PDF). 2019-07-23. Retrieved 2024-11-05. <a href="https://www.cs.toronto.edu/~hinton/absps/DRNN_speech.pdf?form=MG0AV3" target="_blank">https://www.cs.toronto.edu/~hinton/absps/DRNN_speech.pdf?form=MG0AV3</a> <a href="#fnref:11" class="footnote-back-ref">↩</a></p></li> <li id="fn:12"><p>Hoy, Matthew B. (2018). "Alexa, Siri, Cortana, and More: An Introduction to Voice Assistants". Medical Reference Services Quarterly. 37 (1): 81–88. doi:10.1080/02763869.2018.1404391. ISSN 1540-9597. PMID 29327988. <a href="https://pubmed.ncbi.nlm.nih.gov/29327988/" target="_blank">https://pubmed.ncbi.nlm.nih.gov/29327988/</a> <a href="#fnref:12" class="footnote-back-ref">↩</a></p></li> <li id="fn:13"><p>"Vbee". vbee.vn (in Vietnamese). Retrieved 2024-11-05. <a href="https://vbee.vn" target="_blank">https://vbee.vn</a> <a href="#fnref:13" class="footnote-back-ref">↩</a></p></li> <li id="fn:14"><p>"Deep Neural Networks for Acoustic Modeling in Speech Recognition" (PDF). 2019-07-23. Retrieved 2024-11-05. <a href="https://www.cs.toronto.edu/~hinton/absps/DNN-2012-proof.pdf?form=MG0AV3" target="_blank">https://www.cs.toronto.edu/~hinton/absps/DNN-2012-proof.pdf?form=MG0AV3</a> <a href="#fnref:14" class="footnote-back-ref">↩</a></p></li> <li id="fn:15"><p>Hagiwara, Masato (2021-12-21). Real-World Natural Language Processing: Practical applications with deep learning. Simon and Schuster. ISBN 978-1-63835-039-2. <a href="978-1-63835-039-2" target="_blank">978-1-63835-039-2</a> <a href="#fnref:15" class="footnote-back-ref">↩</a></p></li> <li id="fn:16"><p>Mowlaee, Pejman; Kulmer, Josef (August 2015). "Phase Estimation in Single-Channel Speech Enhancement: Limits-Potential". IEEE/ACM Transactions on Audio, Speech, and Language Processing. 23 (8): 1283–1294. doi:10.1109/TASLP.2015.2430820. ISSN 2329-9290. S2CID 13058142. Retrieved 2017-12-03. <a href="https://ieeexplore.ieee.org/document/7103305" target="_blank">https://ieeexplore.ieee.org/document/7103305</a> <a href="#fnref:16" class="footnote-back-ref">↩</a></p></li> <li id="fn:17"><p>Mowlaee, Pejman; Kulmer, Josef; Stahl, Johannes; Mayer, Florian (2017). Single channel phase-aware signal processing in speech communication: theory and practice. Chichester: Wiley. ISBN 978-1-119-23882-9. <a href="978-1-119-23882-9" target="_blank">978-1-119-23882-9</a> <a href="#fnref:17" class="footnote-back-ref">↩</a></p></li> <li id="fn:18"><p>Mowlaee, Pejman; Kulmer, Josef (August 2015). "Phase Estimation in Single-Channel Speech Enhancement: Limits-Potential". IEEE/ACM Transactions on Audio, Speech, and Language Processing. 23 (8): 1283–1294. doi:10.1109/TASLP.2015.2430820. ISSN 2329-9290. S2CID 13058142. Retrieved 2017-12-03. <a href="https://ieeexplore.ieee.org/document/7103305" target="_blank">https://ieeexplore.ieee.org/document/7103305</a> <a href="#fnref:18" class="footnote-back-ref">↩</a></p></li> <li id="fn:19"><p>Kulmer, Josef; Mowlaee, Pejman (April 2015). "Harmonic phase estimation in single-channel speech enhancement using von Mises distribution and prior SNR". Acoustics, Speech and Signal Processing (ICASSP), 2015 IEEE International Conference on. IEEE. pp. 5063–5067. <a href="#fnref:19" class="footnote-back-ref">↩</a></p></li> <li id="fn:20"><p>Kulmer, Josef; Mowlaee, Pejman (April 2015). "Harmonic phase estimation in single-channel speech enhancement using von Mises distribution and prior SNR". Acoustics, Speech and Signal Processing (ICASSP), 2015 IEEE International Conference on. IEEE. pp. 5063–5067. <a href="#fnref:20" class="footnote-back-ref">↩</a></p></li> <li id="fn:21"><p>Kulmer, Josef; Mowlaee, Pejman (May 2015). "Phase Estimation in Single Channel Speech Enhancement Using Phase Decomposition". IEEE Signal Processing Letters. 22 (5): 598–602. Bibcode:2015ISPL...22..598K. doi:10.1109/LSP.2014.2365040. ISSN 1070-9908. S2CID 15503015. Retrieved 2017-12-03. <a href="https://ieeexplore.ieee.org/document/6936313" target="_blank">https://ieeexplore.ieee.org/document/6936313</a> <a href="#fnref:21" class="footnote-back-ref">↩</a></p></li> <li id="fn:22"><p>Mowlaee, Pejman; Saeidi, Rahim; Stylianou, Yannis (July 2016). "Advances in phase-aware signal processing in speech communication". Speech Communication. 81: 1–29. doi:10.1016/j.specom.2016.04.002. ISSN 0167-6393. S2CID 17409161. Retrieved 2017-12-03. <a href="http://linkinghub.elsevier.com/retrieve/pii/S0167639316300784" target="_blank">http://linkinghub.elsevier.com/retrieve/pii/S0167639316300784</a> <a href="#fnref:22" class="footnote-back-ref">↩</a></p></li> <li id="fn:23"><p>Mowlaee, Pejman; Saeidi, Rahim; Stylianou, Yannis (July 2016). "Advances in phase-aware signal processing in speech communication". Speech Communication. 81: 1–29. doi:10.1016/j.specom.2016.04.002. ISSN 0167-6393. S2CID 17409161. Retrieved 2017-12-03. <a href="http://linkinghub.elsevier.com/retrieve/pii/S0167639316300784" target="_blank">http://linkinghub.elsevier.com/retrieve/pii/S0167639316300784</a> <a href="#fnref:23" class="footnote-back-ref">↩</a></p></li> <li id="fn:24"><p>Kulmer, Josef; Mowlaee, Pejman (April 2015). "Harmonic phase estimation in single-channel speech enhancement using von Mises distribution and prior SNR". Acoustics, Speech and Signal Processing (ICASSP), 2015 IEEE International Conference on. IEEE. pp. 5063–5067. <a href="#fnref:24" class="footnote-back-ref">↩</a></p></li> </ol>