Sensor fusion is a term that covers a number of methods and algorithms, including:
Two example sensor fusion calculations are illustrated below.
Let
x
1
{\displaystyle {x}_{1}}
and
x
2
{\displaystyle {x}_{2}}
denote two estimates from two independent sensor measurements, with noise variances
σ
1
2
{\displaystyle \scriptstyle \sigma _{1}^{2}}
and
σ
2
2
{\displaystyle \scriptstyle \sigma _{2}^{2}}
, respectively. One way of obtaining a combined estimate
x
3
{\displaystyle {x}_{3}}
is to apply inverse-variance weighting, which is also employed within the Fraser-Potter fixed-interval smoother, namely
x
3
=
σ
3
2
(
σ
1
−
2
x
1
+
σ
2
−
2
x
2
)
{\displaystyle {x}_{3}=\sigma _{3}^{2}(\sigma _{1}^{-2}{x}_{1}+\sigma _{2}^{-2}{x}_{2})}
,
where
σ
3
2
=
(
σ
1
−
2
+
σ
2
−
2
)
−
1
{\displaystyle \scriptstyle \sigma _{3}^{2}=(\scriptstyle \sigma _{1}^{-2}+\scriptstyle \sigma _{2}^{-2})^{-1}}
is the variance of the combined estimate. It can be seen that the fused result is simply a linear combination of the two measurements weighted by their respective information.
It is worth noting that if
x
{\displaystyle {x}}
is a random variable. The estimates
x
1
{\displaystyle {x}_{1}}
and
x
2
{\displaystyle {x}_{2}}
will be correlated through common process noise, which will cause the estimate
x
3
{\displaystyle {x}_{3}}
to lose conservativeness.
By inspection, when the first measurement is noise free, the filter ignores the second measurement and vice versa. That is, the combined estimate is weighted by the quality of the measurements.
In sensor fusion, centralized versus decentralized refers to where the fusion of the data occurs. In centralized fusion, the clients simply forward all of the data to a central location, and some entity at the central location is responsible for correlating and fusing the data. In decentralized, the clients take full responsibility for fusing the data. "In this case, every sensor or platform can be viewed as an intelligent asset having some degree of autonomy in decision-making."
Multiple combinations of centralized and decentralized systems exist.
Another classification of sensor configuration refers to the coordination of information flow between sensors. These mechanisms provide a way to resolve conflicts or disagreements and to allow the development of dynamic sensing strategies.
Sensors are in redundant (or competitive) configuration if each node delivers independent measures of the same properties. This configuration can be used in error correction when comparing information from multiple nodes. Redundant strategies are often used with high level fusions in voting procedures.
Complementary configuration occurs when multiple information sources supply different information about the same features. This strategy is used for fusing information at raw data level within decision-making algorithms. Complementary features are typically applied in motion recognition tasks with neural network, hidden Markov model, support vector machine, clustering methods and other techniques.
Cooperative sensor fusion uses the information extracted by multiple independent sensors to provide information that would not be available from single sensors. For example, sensors connected to body segments are used for the detection of the angle between them. Cooperative sensor strategy gives information impossible to obtain from single nodes. Cooperative information fusion can be used in motion recognition, gait analysis, motion analysis,,.
There are several categories or levels of sensor fusion that are commonly used.
Sensor fusion level can also be defined basing on the kind of information used to feed the fusion algorithm. More precisely, sensor fusion can be performed fusing raw data coming from different sources, extrapolated features or even decision made by single nodes.
Elmenreich, W. (2002). Sensor Fusion in Time-Triggered Systems, PhD Thesis (PDF). Vienna, Austria: Vienna University of Technology. p. 173. https://mobile.aau.at/~welmenre/papers/elmenreich_Dissertation_sensorFusionInTimeTriggeredSystems.pdf
Haghighat, Mohammad Bagher Akbari; Aghagolzadeh, Ali; Seyedarabi, Hadi (2011). "Multi-focus image fusion for visual sensor networks in DCT domain". Computers & Electrical Engineering. 37 (5): 789–797. doi:10.1016/j.compeleceng.2011.04.016. S2CID 38131177. /wiki/Doi_(identifier)
Li, Wangyan; Wang, Zidong; Wei, Guoliang; Ma, Lifeng; Hu, Jun; Ding, Derui (2015). "A Survey on Multisensor Fusion and Consensus Filtering for Sensor Networks". Discrete Dynamics in Nature and Society. 2015: 1–12. doi:10.1155/2015/683701. ISSN 1026-0226. https://doi.org/10.1155%2F2015%2F683701
Badeli, Vahid; Ranftl, Sascha; Melito, Gian Marco; Reinbacher-Köstinger, Alice; Von Der Linden, Wolfgang; Ellermann, Katrin; Biro, Oszkar (2021-01-01). "Bayesian inference of multi-sensors impedance cardiography for detection of aortic dissection". COMPEL - the International Journal for Computation and Mathematics in Electrical and Electronic Engineering. 41 (3): 824–839. doi:10.1108/COMPEL-03-2021-0072. ISSN 0332-1649. S2CID 245299500. https://doi.org/10.1108/COMPEL-03-2021-0072
Ranftl, Sascha; Melito, Gian Marco; Badeli, Vahid; Reinbacher-Köstinger, Alice; Ellermann, Katrin; von der Linden, Wolfgang (2019-12-31). "Bayesian Uncertainty Quantification with Multi-Fidelity Data and Gaussian Processes for Impedance Cardiography of Aortic Dissection". Entropy. 22 (1): 58. Bibcode:2019Entrp..22...58R. doi:10.3390/e22010058. ISSN 1099-4300. PMC 7516489. PMID 33285833. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7516489
Maybeck, S. (1982). Stochastic Models, Estimating, and Control. River Edge, NJ: Academic Press.
Forsling, Robin (2023-12-15). The Dark Side of Decentralized Target Tracking : Unknown Correlations and Communication Constraints. Linköping Studies in Science and Technology. Dissertations. Vol. 2359. Linköping: Linköping University Electronic Press. doi:10.3384/9789180754101. ISBN 978-91-8075-409-5. 978-91-8075-409-5
N. Xiong; P. Svensson (2002). "Multi-sensor management for information fusion: issues and approaches". Information Fusion. p. 3(2):163–186. http://www.elsevier.com/locate/inffus
Durrant-Whyte, Hugh F. (2016). "Sensor Models and Multisensor Integration". The International Journal of Robotics Research. 7 (6): 97–113. doi:10.1177/027836498800700608. ISSN 0278-3649. S2CID 35656213. /wiki/Doi_(identifier)
Galar, Diego; Kumar, Uday (2017). eMaintenance: Essential Electronic Tools for Efficiency. Academic Press. p. 26. ISBN 9780128111543. 9780128111543
Li, Wenfeng; Bao, Junrong; Fu, Xiuwen; Fortino, Giancarlo; Galzarano, Stefano (2012). "Human Postures Recognition Based on D-S Evidence Theory and Multi-sensor Data Fusion". 2012 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (ccgrid 2012). pp. 912–917. doi:10.1109/CCGrid.2012.144. ISBN 978-1-4673-1395-7. S2CID 1571720. 978-1-4673-1395-7
Fortino, Giancarlo; Gravina, Raffaele (2015). "Fall-MobileGuard: a Smart Real-Time Fall Detection System". Proceedings of the 10th EAI International Conference on Body Area Networks. doi:10.4108/eai.28-9-2015.2261462. ISBN 978-1-63190-084-6. S2CID 38913107. 978-1-63190-084-6
Tao, Shuai; Zhang, Xiaowei; Cai, Huaying; Lv, Zeping; Hu, Caiyou; Xie, Haiqun (2018). "Gait based biometric personal authentication by using MEMS inertial sensors". Journal of Ambient Intelligence and Humanized Computing. 9 (5): 1705–1712. doi:10.1007/s12652-018-0880-6. ISSN 1868-5137. S2CID 52304214. /wiki/Doi_(identifier)
Dehzangi, Omid; Taherisadr, Mojtaba; ChangalVala, Raghvendar (2017). "IMU-Based Gait Recognition Using Convolutional Neural Networks and Multi-Sensor Fusion". Sensors. 17 (12): 2735. Bibcode:2017Senso..17.2735D. doi:10.3390/s17122735. ISSN 1424-8220. PMC 5750784. PMID 29186887. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5750784
Guenterberg, E.; Yang, A.Y.; Ghasemzadeh, H.; Jafari, R.; Bajcsy, R.; Sastry, S.S. (2009). "A Method for Extracting Temporal Parameters Based on Hidden Markov Models in Body Sensor Networks With Inertial Sensors" (PDF). IEEE Transactions on Information Technology in Biomedicine. 13 (6): 1019–1030. doi:10.1109/TITB.2009.2028421. ISSN 1089-7771. PMID 19726268. S2CID 1829011. http://www.eecs.berkeley.edu/~yang/paper/GuenterbergE-Biomedicine.pdf
Parisi, Federico; Ferrari, Gianluigi; Giuberti, Matteo; Contin, Laura; Cimolin, Veronica; Azzaro, Corrado; Albani, Giovanni; Mauro, Alessandro (2016). "Inertial BSN-Based Characterization and Automatic UPDRS Evaluation of the Gait Task of Parkinsonians". IEEE Transactions on Affective Computing. 7 (3): 258–271. doi:10.1109/TAFFC.2016.2549533. ISSN 1949-3045. S2CID 16866555. /wiki/Doi_(identifier)
Gao, Lei; Bourke, A.K.; Nelson, John (2014). "Evaluation of accelerometer based multi-sensor versus single-sensor activity recognition systems". Medical Engineering & Physics. 36 (6): 779–785. doi:10.1016/j.medengphy.2014.02.012. ISSN 1350-4533. PMID 24636448. /wiki/Doi_(identifier)
Gao, Lei; Bourke, A.K.; Nelson, John (2014). "Evaluation of accelerometer based multi-sensor versus single-sensor activity recognition systems". Medical Engineering & Physics. 36 (6): 779–785. doi:10.1016/j.medengphy.2014.02.012. ISSN 1350-4533. PMID 24636448. /wiki/Doi_(identifier)
Parisi, Federico; Ferrari, Gianluigi; Giuberti, Matteo; Contin, Laura; Cimolin, Veronica; Azzaro, Corrado; Albani, Giovanni; Mauro, Alessandro (2016). "Inertial BSN-Based Characterization and Automatic UPDRS Evaluation of the Gait Task of Parkinsonians". IEEE Transactions on Affective Computing. 7 (3): 258–271. doi:10.1109/TAFFC.2016.2549533. ISSN 1949-3045. S2CID 16866555. /wiki/Doi_(identifier)
Xu, James Y.; Wang, Yan; Barrett, Mick; Dobkin, Bruce; Pottie, Greg J.; Kaiser, William J. (2016). "Personalized Multilayer Daily Life Profiling Through Context Enabled Activity Classification and Motion Reconstruction: An Integrated System Approach". IEEE Journal of Biomedical and Health Informatics. 20 (1): 177–188. doi:10.1109/JBHI.2014.2385694. ISSN 2168-2194. PMID 25546868. S2CID 16785375. https://doi.org/10.1109%2FJBHI.2014.2385694
Chia Bejarano, Noelia; Ambrosini, Emilia; Pedrocchi, Alessandra; Ferrigno, Giancarlo; Monticone, Marco; Ferrante, Simona (2015). "A Novel Adaptive, Real-Time Algorithm to Detect Gait Events From Wearable Sensors". IEEE Transactions on Neural Systems and Rehabilitation Engineering. 23 (3): 413–422. doi:10.1109/TNSRE.2014.2337914. hdl:11311/865739. ISSN 1534-4320. PMID 25069118. S2CID 25828466. /wiki/Doi_(identifier)
Wang, Zhelong; Qiu, Sen; Cao, Zhongkai; Jiang, Ming (2013). "Quantitative assessment of dual gait analysis based on inertial sensors with body sensor network". Sensor Review. 33 (1): 48–56. doi:10.1108/02602281311294342. ISSN 0260-2288. /wiki/Doi_(identifier)
Kong, Weisheng; Wanning, Lauren; Sessa, Salvatore; Zecca, Massimiliano; Magistro, Daniele; Takeuchi, Hikaru; Kawashima, Ryuta; Takanishi, Atsuo (2017). "Step Sequence and Direction Detection of Four Square Step Test" (PDF). IEEE Robotics and Automation Letters. 2 (4): 2194–2200. doi:10.1109/LRA.2017.2723929. ISSN 2377-3766. S2CID 23410874. http://irep.ntu.ac.uk/id/eprint/33502/1/10999_Magistro.pdf
Rethinking JDL Data Fusion Levels http://www.infofusion.buffalo.edu/tm/Dr.Llinas'stuff/Rethinking%20JDL%20Data%20Fusion%20Levels_BowmanSteinberg.pdf
Blasch, E., Plano, S. (2003) “Level 5: User Refinement to aid the Fusion Process”, Proceedings of the SPIE, Vol. 5099.
J. Llinas; C. Bowman; G. Rogova; A. Steinberg; E. Waltz; F. White (2004). Revisiting the JDL data fusion model II. International Conference on Information Fusion. CiteSeerX 10.1.1.58.2996. /wiki/CiteSeerX_(identifier)
Blasch, E. (2006) "Sensor, user, mission (SUM) resource management and their interaction with level 2/3 fusion[permanent dead link]" International Conference on Information Fusion. http://www.iut-amiens.fr/~ricquebourg/these/fusion_2006/Papers/394.pdf
"Harnessing the full power of sensor fusion -". 3 April 2024. http://defensesystems.com/articles/2009/09/02/c4isr1-sensor-fusion.aspx
Blasch, E., Steinberg, A., Das, S., Llinas, J., Chong, C.-Y., Kessler, O., Waltz, E., White, F. (2013) "Revisiting the JDL model for information Exploitation," International Conference on Information Fusion.
Gravina, Raffaele; Alinia, Parastoo; Ghasemzadeh, Hassan; Fortino, Giancarlo (2017). "Multi-sensor fusion in body sensor networks: State-of-the-art and research challenges". Information Fusion. 35: 68–80. doi:10.1016/j.inffus.2016.09.005. ISSN 1566-2535. S2CID 40608207. /wiki/Doi_(identifier)
Gao, Teng; Song, Jin-Yan; Zou, Ji-Yan; Ding, Jin-Hua; Wang, De-Quan; Jin, Ren-Cheng (2015). "An overview of performance trade-off mechanisms in routing protocol for green wireless sensor networks". Wireless Networks. 22 (1): 135–157. doi:10.1007/s11276-015-0960-x. ISSN 1022-0038. S2CID 34505498. /wiki/Doi_(identifier)
Chen, Chen; Jafari, Roozbeh; Kehtarnavaz, Nasser (2015). "A survey of depth and inertial sensor fusion for human action recognition". Multimedia Tools and Applications. 76 (3): 4405–4425. doi:10.1007/s11042-015-3177-1. ISSN 1380-7501. S2CID 18112361. /wiki/Doi_(identifier)
Gravina, Raffaele; Alinia, Parastoo; Ghasemzadeh, Hassan; Fortino, Giancarlo (2017). "Multi-sensor fusion in body sensor networks: State-of-the-art and research challenges". Information Fusion. 35: 68–80. doi:10.1016/j.inffus.2016.09.005. ISSN 1566-2535. S2CID 40608207. /wiki/Doi_(identifier)
Banovic, Nikola; Buzali, Tofi; Chevalier, Fanny; Mankoff, Jennifer; Dey, Anind K. (2016). "Modeling and Understanding Human Routine Behavior". Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems - CHI '16. pp. 248–260. doi:10.1145/2858036.2858557. ISBN 9781450333627. S2CID 872756. 9781450333627
Maria, Aileni Raluca; Sever, Pasca; Carlos, Valderrama (2015). "Biomedical sensors data fusion algorithm for enhancing the efficiency of fault-tolerant systems in case of wearable electronics device". 2015 Conference Grid, Cloud & High Performance Computing in Science (ROLCG). pp. 1–4. doi:10.1109/ROLCG.2015.7367228. ISBN 978-6-0673-7040-9. S2CID 18782930. 978-6-0673-7040-9
Chen, Chen; Jafari, Roozbeh; Kehtarnavaz, Nasser (2015). "A survey of depth and inertial sensor fusion for human action recognition". Multimedia Tools and Applications. 76 (3): 4405–4425. doi:10.1007/s11042-015-3177-1. ISSN 1380-7501. S2CID 18112361. /wiki/Doi_(identifier)
Gross, Jason; Yu Gu; Matthew Rhudy; Srikanth Gururajan; Marcello Napolitano (July 2012). "Flight Test Evaluation of Sensor Fusion Algorithms for Attitude Estimation". IEEE Transactions on Aerospace and Electronic Systems. 48 (3): 2128–2139. Bibcode:2012ITAES..48.2128G. doi:10.1109/TAES.2012.6237583. S2CID 393165. /wiki/Bibcode_(identifier)
Joshi, V., Rajamani, N., Takayuki, K., Prathapaneni, N., Subramaniam, L. V. (2013). Information Fusion Based Learning for Frugal Traffic State Sensing. Proceedings of the Twenty-Third International Joint Conference on Artificial Intelligence.{{cite conference}}: CS1 maint: multiple names: authors list (link) /wiki/Template:Cite_conference
Mircea Paul, Muresan; Ion, Giosan; Sergiu, Nedevschi (2020-02-18). "Stabilization and Validation of 3D Object Position Using Multimodal Sensor Fusion and Semantic Segmentation". Sensors. 20 (4): 1110. Bibcode:2020Senso..20.1110M. doi:10.3390/s20041110. PMC 7070899. PMID 32085608. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7070899
Ran, Lingyan; Zhang, Yanning; Wei, Wei; Zhang, Qilin (2017-10-23). "A Hyperspectral Image Classification Framework with Spatial Pixel Pair Features". Sensors. 17 (10): 2421. Bibcode:2017Senso..17.2421R. doi:10.3390/s17102421. PMC 5677443. PMID 29065535. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5677443
Haghighat, Mohammad; Abdel-Mottaleb, Mohamed; Alhalabi, Wadee (2016). "Discriminant Correlation Analysis: Real-Time Feature Level Fusion for Multimodal Biometric Recognition". IEEE Transactions on Information Forensics and Security. 11 (9): 1984–1996. doi:10.1109/TIFS.2016.2569061. S2CID 15624506. https://zenodo.org/record/889881