Menu
Home
People
Places
Arts
History
Plants & Animals
Science
Life & Culture
Technology
Reference.org
Siamese neural network
open-in-new
<h2 id="learning">Learning</h2> <p>Learning in twin networks can be done with <a href="/facts/Triplet_loss/FCtHWMYt">triplet loss</a> or <a href="/facts/Contrastive_loss/FCtHWMYt">contrastive loss</a>. For learning by triplet loss a baseline vector (anchor image) is compared against a positive vector (truthy image) and a negative vector (falsy image). The negative vector will force learning in the network, while the positive vector will act like a regularizer. For learning by contrastive loss there must be a weight decay to regularize the weights, or some similar operation like a normalization. </p><p>A distance metric for a loss function may have the following properties<a class="footnote-ref" id="fnref:6" href="#fn:6"><sup>6</sup></a> </p> <ul><li>Non-negativity: δ ( x , y ) ≥ 0 {\displaystyle \delta (x,y)\geq 0} </li> <li>Identity of Non-discernibles: δ ( x , y ) = 0 ⟺ x = y {\displaystyle \delta (x,y)=0\iff x=y} </li> <li>Commutativity: δ ( x , y ) = δ ( y , x ) {\displaystyle \delta (x,y)=\delta (y,x)} </li> <li>Triangle inequality: δ ( x , z ) ≤ δ ( x , y ) + δ ( y , z ) {\displaystyle \delta (x,z)\leq \delta (x,y)+\delta (y,z)} </li></ul> <p>In particular, the triplet loss algorithm is often defined with squared Euclidean (which unlike Euclidean, does not have triangle inequality) distance at its core. </p> <h3>Predefined metrics, Euclidean distance metric</h3> <p>The common learning goal is to minimize a distance metric for similar objects and maximize for distinct ones. This gives a loss function like </p> δ ( x ( i ) , x ( j ) ) = { min ‖ f ( x ( i ) ) − f ( x ( j ) ) ‖ , i = j max ‖ f ( x ( i ) ) − f ( x ( j ) ) ‖ , i ≠ j {\displaystyle {\begin{aligned}\delta (x^{(i)},x^{(j)})={\begin{cases}\min \ \|\operatorname {f} \left(x^{(i)}\right)-\operatorname {f} \left(x^{(j)}\right)\|\,,i=j\\\max \ \|\operatorname {f} \left(x^{(i)}\right)-\operatorname {f} \left(x^{(j)}\right)\|\,,i\neq j\end{cases}}\end{aligned}}} i , j {\displaystyle i,j} are indexes into a set of vectors f ( ⋅ ) {\displaystyle \operatorname {f} (\cdot )} function implemented by the twin network <p>The most common distance metric used is <a href="/facts/Euclidean_distance/9qDoQKQe">Euclidean distance</a>, in case of which the loss function can be rewritten in matrix form as </p> δ ( x ( i ) , x ( j ) ) ≈ ( x ( i ) − x ( j ) ) T ( x ( i ) − x ( j ) ) {\displaystyle \operatorname {\delta } (\mathbf {x} ^{(i)},\mathbf {x} ^{(j)})\approx (\mathbf {x} ^{(i)}-\mathbf {x} ^{(j)})^{T}(\mathbf {x} ^{(i)}-\mathbf {x} ^{(j)})} <h3>Learned metrics, nonlinear distance metric</h3> <p>A more general case is where the output vector from the twin network is passed through additional network layers implementing non-linear distance metrics. </p> if i = j then δ [ f ( x ( i ) ) , f ( x ( j ) ) ] is small otherwise δ [ f ( x ( i ) ) , f ( x ( j ) ) ] is large {\displaystyle {\begin{aligned}{\text{if}}\,i=j\,{\text{then}}&\,\operatorname {\delta } \left[\operatorname {f} \left(x^{(i)}\right),\,\operatorname {f} \left(x^{(j)}\right)\right]\,{\text{is small}}\\{\text{otherwise}}&\,\operatorname {\delta } \left[\operatorname {f} \left(x^{(i)}\right),\,\operatorname {f} \left(x^{(j)}\right)\right]\,{\text{is large}}\end{aligned}}} i , j {\displaystyle i,j} are indexes into a set of vectors f ( ⋅ ) {\displaystyle \operatorname {f} (\cdot )} function implemented by the twin network δ ( ⋅ ) {\displaystyle \operatorname {\delta } (\cdot )} function implemented by the network joining outputs from the twin network <p>On a matrix form the previous is often approximated as a <a href="/facts/Mahalanobis_distance/UneEsDng">Mahalanobis distance</a> for a linear space as<a class="footnote-ref" id="fnref:7" href="#fn:7"><sup>7</sup></a> </p> δ ( x ( i ) , x ( j ) ) ≈ ( x ( i ) − x ( j ) ) T M ( x ( i ) − x ( j ) ) {\displaystyle \operatorname {\delta } (\mathbf {x} ^{(i)},\mathbf {x} ^{(j)})\approx (\mathbf {x} ^{(i)}-\mathbf {x} ^{(j)})^{T}\mathbf {M} (\mathbf {x} ^{(i)}-\mathbf {x} ^{(j)})} <p>This can be further subdivided in at least <a href="/facts/Unsupervised_learning/8a6f61MC">Unsupervised learning</a> and <a href="/facts/Supervised_learning/JlNFNPFt">Supervised learning</a>. </p> <h3>Learned metrics, half-twin networks</h3> <p>This form also allows the twin network to be more of a half-twin, implementing a slightly different functions </p> if i = j then δ [ f ( x ( i ) ) , g ( x ( j ) ) ] is small otherwise δ [ f ( x ( i ) ) , g ( x ( j ) ) ] is large {\displaystyle {\begin{aligned}{\text{if}}\,i=j\,{\text{then}}&\,\operatorname {\delta } \left[\operatorname {f} \left(x^{(i)}\right),\,\operatorname {g} \left(x^{(j)}\right)\right]\,{\text{is small}}\\{\text{otherwise}}&\,\operatorname {\delta } \left[\operatorname {f} \left(x^{(i)}\right),\,\operatorname {g} \left(x^{(j)}\right)\right]\,{\text{is large}}\end{aligned}}} i , j {\displaystyle i,j} are indexes into a set of vectors f ( ⋅ ) , g ( ⋅ ) {\displaystyle \operatorname {f} (\cdot ),\operatorname {g} (\cdot )} function implemented by the half-twin network δ ( ⋅ ) {\displaystyle \operatorname {\delta } (\cdot )} function implemented by the network joining outputs from the twin network <h2 id="twin-networks-for-object-tracking">Twin networks for object tracking</h2> <p>Twin networks have been used in object tracking because of its unique two tandem inputs and similarity measurement. In object tracking, one input of the twin network is user pre-selected exemplar image, the other input is a larger search image, which twin network's job is to locate exemplar inside of search image. By measuring the similarity between exemplar and each part of the search image, a map of similarity score can be given by the twin network. Furthermore, using a Fully Convolutional Network, the process of computing each sector's similarity score can be replaced with only one cross correlation layer.<a class="footnote-ref" id="fnref:8" href="#fn:8"><sup>8</sup></a> </p><p>After being first introduced in 2016, Twin fully convolutional network has been used in many High-performance Real-time Object Tracking Neural Networks. Like CFnet,<a class="footnote-ref" id="fnref:9" href="#fn:9"><sup>9</sup></a> StructSiam,<a class="footnote-ref" id="fnref:10" href="#fn:10"><sup>10</sup></a> SiamFC-tri,<a class="footnote-ref" id="fnref:11" href="#fn:11"><sup>11</sup></a> DSiam,<a class="footnote-ref" id="fnref:12" href="#fn:12"><sup>12</sup></a> SA-Siam,<a class="footnote-ref" id="fnref:13" href="#fn:13"><sup>13</sup></a> SiamRPN,<a class="footnote-ref" id="fnref:14" href="#fn:14"><sup>14</sup></a> DaSiamRPN,<a class="footnote-ref" id="fnref:15" href="#fn:15"><sup>15</sup></a> Cascaded SiamRPN,<a class="footnote-ref" id="fnref:16" href="#fn:16"><sup>16</sup></a> SiamMask,<a class="footnote-ref" id="fnref:17" href="#fn:17"><sup>17</sup></a> SiamRPN++,<a class="footnote-ref" id="fnref:18" href="#fn:18"><sup>18</sup></a> Deeper and Wider SiamRPN.<a class="footnote-ref" id="fnref:19" href="#fn:19"><sup>19</sup></a> </p> <h2 id="see-also">See also</h2> <ul><li><a href="/facts/Artificial_neural_network/6V1jMlkx">Artificial neural network</a></li> <li><a href="/facts/Triplet_loss/FCtHWMYt">Triplet loss</a></li></ul> <h2 id="further-reading">Further reading</h2> <ul><li>Chicco, Davide (2020), <a href="https://doi.org/10.1007/978-1-0716-0826-5_3">"Siamese neural networks: an overview"</a>, <i>Artificial Neural Networks</i>, Methods in Molecular Biology, vol. 2190 (3rd ed.), New York City, New York, USA: <a href="/facts/Springer-Verlag/nAesf6nT">Springer Protocols</a>, Humana Press, pp. 73–94, <a href="/facts/Doi_(identifier)/muM9Etpq">doi</a>:<a href="https://doi.org/10.1007%2F978-1-0716-0826-5_3">10.1007/978-1-0716-0826-5_3</a>, <a href="/facts/ISBN_(identifier)/15AdSPa9">ISBN</a> 978-1-0716-0826-5, <a href="/facts/PMID_(identifier)/JlHAvMHt">PMID</a> <a href="https://pubmed.ncbi.nlm.nih.gov/32804361">32804361</a>, <a href="/facts/S2CID_(identifier)/ldJsHa2Y">S2CID</a> <a href="https://api.semanticscholar.org/CorpusID:221144012">221144012</a></li></ul> <h2 id="references">References</h2> <ol> <li id="fn:1"><p>Chicco, Davide (2020), "Siamese neural networks: an overview", Artificial Neural Networks, Methods in Molecular Biology, vol. 2190 (3rd ed.), New York City, New York, USA: Springer Protocols, Humana Press, pp. 73–94, doi:10.1007/978-1-0716-0826-5_3, ISBN 978-1-0716-0826-5, PMID 32804361, S2CID 221144012 <a href="978-1-0716-0826-5" target="_blank">978-1-0716-0826-5</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li> <li id="fn:2"><p>Bromley, Jane; Guyon, Isabelle; LeCun, Yann; Säckinger, Eduard; Shah, Roopak (1994). "Signature verification using a "Siamese" time delay neural network" (PDF). Advances in Neural Information Processing Systems. 6: 737–744. <a href="https://papers.neurips.cc/paper_files/paper/1993/file/288cc0ff022877bd3df94bc9360b9c5d-Paper.pdf" target="_blank">https://papers.neurips.cc/paper_files/paper/1993/file/288cc0ff022877bd3df94bc9360b9c5d-Paper.pdf</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li> <li id="fn:3"><p>Chopra, S.; Hadsell, R.; LeCun, Y. (June 2005). "Learning a Similarity Metric Discriminatively, with Application to Face Verification". 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05). Vol. 1. pp. 539–546 vol. 1. doi:10.1109/CVPR.2005.202. ISBN 0-7695-2372-2. S2CID 5555257. <a href="0-7695-2372-2" target="_blank">0-7695-2372-2</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li> <li id="fn:4"><p>Taigman, Y.; Yang, M.; Ranzato, M.; Wolf, L. (June 2014). "DeepFace: Closing the Gap to Human-Level Performance in Face Verification". 2014 IEEE Conference on Computer Vision and Pattern Recognition. pp. 1701–1708. doi:10.1109/CVPR.2014.220. ISBN 978-1-4799-5118-5. S2CID 2814088. <a href="978-1-4799-5118-5" target="_blank">978-1-4799-5118-5</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></p></li> <li id="fn:5"><p>Taigman, Y.; Yang, M.; Ranzato, M.; Wolf, L. (June 2014). "DeepFace: Closing the Gap to Human-Level Performance in Face Verification". 2014 IEEE Conference on Computer Vision and Pattern Recognition. pp. 1701–1708. doi:10.1109/CVPR.2014.220. ISBN 978-1-4799-5118-5. S2CID 2814088. <a href="978-1-4799-5118-5" target="_blank">978-1-4799-5118-5</a> <a href="#fnref:5" class="footnote-back-ref">↩</a></p></li> <li id="fn:6"><p>Chatterjee, Moitreya; Luo, Yunan. "Similarity Learning with (or without) Convolutional Neural Network" (PDF). Retrieved 2018-12-07. <a href="http://slazebni.cs.illinois.edu/spring17/lec09_similarity.pdf" target="_blank">http://slazebni.cs.illinois.edu/spring17/lec09_similarity.pdf</a> <a href="#fnref:6" class="footnote-back-ref">↩</a></p></li> <li id="fn:7"><p>Chandra, M.P. (1936). "On the generalized distance in statistics" (PDF). Proceedings of the National Institute of Sciences of India. 1. 2: 49–55. <a href="http://library.isical.ac.in:8080/jspui/bitstream/123456789/6765/1/Vol02_1936_1_Art05-pcm.pdf" target="_blank">http://library.isical.ac.in:8080/jspui/bitstream/123456789/6765/1/Vol02_1936_1_Art05-pcm.pdf</a> <a href="#fnref:7" class="footnote-back-ref">↩</a></p></li> <li id="fn:8"><p>Fully-Convolutional Siamese Networks for Object Tracking arXiv:1606.09549 <a href="/wiki/ArXiv_(identifier)" target="_blank">/wiki/ArXiv_(identifier)</a> <a href="#fnref:8" class="footnote-back-ref">↩</a></p></li> <li id="fn:9"><p>"End-to-end representation learning for Correlation Filter based tracking". <a href="https://www.robots.ox.ac.uk/~luca/cfnet.html" target="_blank">https://www.robots.ox.ac.uk/~luca/cfnet.html</a> <a href="#fnref:9" class="footnote-back-ref">↩</a></p></li> <li id="fn:10"><p>"Structured Siamese Network for Real-Time Visual Tracking" (PDF). <a href="http://openaccess.thecvf.com/content_ECCV_2018/papers/Yunhua_Zhang_Structured_Siamese_Network_ECCV_2018_paper.pdf" target="_blank">http://openaccess.thecvf.com/content_ECCV_2018/papers/Yunhua_Zhang_Structured_Siamese_Network_ECCV_2018_paper.pdf</a> <a href="#fnref:10" class="footnote-back-ref">↩</a></p></li> <li id="fn:11"><p>"Triplet Loss in Siamese Network for Object Tracking" (PDF). <a href="http://openaccess.thecvf.com/content_ECCV_2018/papers/Xingping_Dong_Triplet_Loss_with_ECCV_2018_paper.pdf" target="_blank">http://openaccess.thecvf.com/content_ECCV_2018/papers/Xingping_Dong_Triplet_Loss_with_ECCV_2018_paper.pdf</a> <a href="#fnref:11" class="footnote-back-ref">↩</a></p></li> <li id="fn:12"><p>"Learning Dynamic Siamese Network for Visual Object Tracking" (PDF). <a href="http://openaccess.thecvf.com/content_ICCV_2017/papers/Guo_Learning_Dynamic_Siamese_ICCV_2017_paper.pdf" target="_blank">http://openaccess.thecvf.com/content_ICCV_2017/papers/Guo_Learning_Dynamic_Siamese_ICCV_2017_paper.pdf</a> <a href="#fnref:12" class="footnote-back-ref">↩</a></p></li> <li id="fn:13"><p>"A Twofold Siamese Network for Real-Time Object Tracking" (PDF). <a href="http://openaccess.thecvf.com/content_cvpr_2018/papers/He_A_Twofold_Siamese_CVPR_2018_paper.pdf" target="_blank">http://openaccess.thecvf.com/content_cvpr_2018/papers/He_A_Twofold_Siamese_CVPR_2018_paper.pdf</a> <a href="#fnref:13" class="footnote-back-ref">↩</a></p></li> <li id="fn:14"><p>"High Performance Visual Tracking with Siamese Region Proposal Network" (PDF). <a href="http://openaccess.thecvf.com/content_cvpr_2018/papers/Li_High_Performance_Visual_CVPR_2018_paper.pdf" target="_blank">http://openaccess.thecvf.com/content_cvpr_2018/papers/Li_High_Performance_Visual_CVPR_2018_paper.pdf</a> <a href="#fnref:14" class="footnote-back-ref">↩</a></p></li> <li id="fn:15"><p>Zhu, Zheng; Wang, Qiang; Li, Bo; Wu, Wei; Yan, Junjie; Hu, Weiming (2018). "Distractor-aware Siamese Networks for Visual Object Tracking". arXiv:1808.06048 [cs.CV]. <a href="/wiki/ArXiv_(identifier)" target="_blank">/wiki/ArXiv_(identifier)</a> <a href="#fnref:15" class="footnote-back-ref">↩</a></p></li> <li id="fn:16"><p>Fan, Heng; Ling, Haibin (2018). "Siamese Cascaded Region Proposal Networks for Real-Time Visual Tracking". arXiv:1812.06148 [cs.CV]. <a href="/wiki/ArXiv_(identifier)" target="_blank">/wiki/ArXiv_(identifier)</a> <a href="#fnref:16" class="footnote-back-ref">↩</a></p></li> <li id="fn:17"><p>Wang, Qiang; Zhang, Li; Bertinetto, Luca; Hu, Weiming; Torr, Philip H. S. (2018). "Fast Online Object Tracking and Segmentation: A Unifying Approach". arXiv:1812.05050 [cs.CV]. <a href="/wiki/ArXiv_(identifier)" target="_blank">/wiki/ArXiv_(identifier)</a> <a href="#fnref:17" class="footnote-back-ref">↩</a></p></li> <li id="fn:18"><p>Li, Bo; Wu, Wei; Wang, Qiang; Zhang, Fangyi; Xing, Junliang; Yan, Junjie (2018). "SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networks". arXiv:1812.11703 [cs.CV]. <a href="/wiki/ArXiv_(identifier)" target="_blank">/wiki/ArXiv_(identifier)</a> <a href="#fnref:18" class="footnote-back-ref">↩</a></p></li> <li id="fn:19"><p>Zhang, Zhipeng; Peng, Houwen (2019). "Deeper and Wider Siamese Networks for Real-Time Visual Tracking". arXiv:1901.01660 [cs.CV]. <a href="/wiki/ArXiv_(identifier)" target="_blank">/wiki/ArXiv_(identifier)</a> <a href="#fnref:19" class="footnote-back-ref">↩</a></p></li> </ol>