Menu
Home Explore People Places Arts History Plants & Animals Science Life & Culture Technology
On this page
Concurrent testing

Research on concurrent computing and software testing focuses on evaluating the behavior and stability of systems that perform concurrent operations. Concurrency testing, which began developing in the 1960s following early studies of program concurrency in the 1950s, aims to identify issues such as incorrect shared memory access and unpredictable thread execution order. Key challenges addressed include resource contention, scheduling, deadlock avoidance, priority inversion, and race conditions, all critical to ensuring reliable concurrent system performance during normal operation.

We don't have any images related to Concurrent testing yet.
We don't have any YouTube videos related to Concurrent testing yet.
We don't have any PDF documents related to Concurrent testing yet.
We don't have any Books related to Concurrent testing yet.
We don't have any archived web articles related to Concurrent testing yet.

Selected history & approaches of testing concurrency

Approaches to concurrency testing may be on a limited unit test level right up to system test level.8

Some approaches to research and application of testing program/software concurrency have been:

  • Execute a test once.9: 63 
This was considered to be ineffective for testing concurrency in a non-deterministic system and was equivalent to the testing of a sequential non-concurrent program on a system
  • Execution of the same test sequence multiple times.10: 63 
Considered likely to find some issues in non-deterministic software execution. This later became called non-deterministic testing.11
  • Deterministic testing.12: 63 
This is an approach to set the system into a particular state so that code can be executed in a known order.
  • Reachability testing1314
An attempt to test synchronisation sequence combinations for a specified input (shared variable access not being corrupted, effectively testing race conditions variables). The sequence is typically derived for non-deterministic test execution.
  • Structural Approaches / Static Analysis
Analysis of code structure and static analysis tools. An example was a heuristic approach15 This led to code checker development, for example jlint.16 Research and comparison of static analysis and code checkers for concurrency bugs 17 See also List of tools for static code analysis
  • Multi-user approach
This is an approach to testing program concurrency by looking at multiple user access, either serving different users or tasks simultaneously.1819 : 745 

Testing software and system concurrency should not be confused with stress testing, which is usually associated with loading a system beyond its defined limits. Testing of concurrent programs can exhibit problems when a system is performing within its defined limits. Most of the approaches above do not rely on overloading a system. Some literature20: 745  states that testing of concurrency is a pre-requisite to stress testing.

Lessons learned from concurrency bug characteristics study

A study in 200821 analysed bug databases in a selection of open source software. It was thought to be the first real-world study of concurrency bugs. 105 bugs were classified as concurrency bugs and analysed, split as 31 being deadlock bugs and 74 non-deadlock bugs. The study had several findings, for potential follow-up and investigation:

  • Approximately one-third of the concurrency bugs cause crashes or hanging programs.
  • Most non-deadlock concurrency bugs are atomicity or order violation.
I.e. focusing on atomicity (protected use of shared data) or sequence will potentially find most non-deadlock type bugs.
  • Most concurrency bugs involve 1 or 2 threads.
I.e. Heavy simultaneous users/usage is not the trigger for these bugs. There is a suggestion that pairwise testing may be effective to catch these types of bugs.
  • Over 20% (7/31) deadlock bugs occurred with a single thread.
  • Most deadlock concurrency bugs (30/31) involved only one or two resources.
An implication that pairwise testing from a resource usage perspective could be applied to reveal deadlocks.

See also

General References

References

  1. Wang, Chao; Said, Mahmoud; Gupta, Aarti (21–28 May 2011). Coverage guided systematic concurrency testing. ICSE '11 Proceedings of the 33rd International Conference on Software Engineering. Waikiki. pp. 221–230. /wiki/Aarti_Gupta_(computer_scientist) ↩

  2. Dustin, Elfriede (28 December 2002). Effective Software Testing: 50 Ways to Improve Your Software Testing. Addison-Wesley Longman. p. 186. ISBN 0201794292. 0201794292 ↩

  3. Leiner, A.L.; Notz, W.A.; Smith, J.L.; Weinberger, A. (July 1959). "PILOT—A New Multiple Computer System". Journal of the ACM. 6 (3): 313–335. doi:10.1145/320986.320987. S2CID 19867617. https://doi.org/10.1145%2F320986.320987 ↩

  4. Dijkstra, Edsger W. (May 1968). "The structure of the "THE"-multiprogramming system". Communications of the ACM. 11 (5): 341–346. doi:10.1145/363095.363143. S2CID 2021311. https://doi.org/10.1145%2F363095.363143 ↩

  5. "Concurrent Software Testing: A Systematic Review" (PDF). Archived from the original on 24 September 2015. Retrieved 4 March 2014.{{cite web}}: CS1 maint: bot: original URL status unknown (link) https://web.archive.org/web/20150924220657/http://www.icmc.usp.br/CMS/Arquivos/arquivos_enviados/BIBLIOTECA_113_RT_359.pdf ↩

  6. Wang, Chao; Said, Mahmoud; Gupta, Aarti (21–28 May 2011). Coverage guided systematic concurrency testing. ICSE '11 Proceedings of the 33rd International Conference on Software Engineering. Waikiki. pp. 221–230. /wiki/Aarti_Gupta_(computer_scientist) ↩

  7. Binder, Robert V. (1999). Testing object-oriented systems: models, patterns, and tools. Addison-Wesley Longman. ISBN 0-201-80938-9. 0-201-80938-9 ↩

  8. Melo, Silvana Morita; Souza, Simone do Rocio Senger de; Souza, Paulo Sérgio Lopes de; Carver, Jeffrey C. (2017). How to test your concurrent software: an approach for the selection of testing techniques. Conference on Systems, Programming, Languages, and Applications: Software for Humanity - SPLASH. ↩

  9. K.C., Tai (20–22 September 1989). Testing of concurrent software. Proceedings of the Thirteenth Annual International Computer Software & Applications Conference. Orlando, FL, USA, USA. pp. 62–64. ↩

  10. K.C., Tai (20–22 September 1989). Testing of concurrent software. Proceedings of the Thirteenth Annual International Computer Software & Applications Conference. Orlando, FL, USA, USA. pp. 62–64. ↩

  11. Hwang, Gwan-Hwan; Tai, Kuo-Chung; Huang, Ting-Lu (1995). "Reachability Testing: An Approach To Testing Concurrent Software". International Journal of Software Engineering and Knowledge Engineering. 5 (4): 493–510. doi:10.1142/S0218194095000241. /wiki/Doi_(identifier) ↩

  12. K.C., Tai (20–22 September 1989). Testing of concurrent software. Proceedings of the Thirteenth Annual International Computer Software & Applications Conference. Orlando, FL, USA, USA. pp. 62–64. ↩

  13. Hwang, Gwan-Hwan; Tai, Kuo-Chung; Huang, Ting-Lu (1995). "Reachability Testing: An Approach To Testing Concurrent Software". International Journal of Software Engineering and Knowledge Engineering. 5 (4): 493–510. doi:10.1142/S0218194095000241. /wiki/Doi_(identifier) ↩

  14. Qi, Xiaofang; Li, Yueran (23–24 November 2018). Parallel Reachability Testing Based on Hadoop MapReduce. th International Conference, SATE 2018. Shenzhen, Guangdong, China. pp. 173–184. doi:10.1007/978-3-030-04272-1_11. /wiki/Doi_(identifier) ↩

  15. Lu, Shan; Park, Soyeon; Seo, Eunsoo; Zhou, Yuanyuan (1–5 March 2008). Learning from mistakes: a comprehensive study on real world concurrency bug characteristics. ASPLOS XIII Proceedings of the 13th international conference on Architectural support for programming languages and operating systems. Seattle, WA, USA. pp. 329–339. ↩

  16. Artho, Cyrille; Biere, Armin (27–28 August 2001). Applying static analysis to large-scale, multi-threaded Java programs. Proceedings 2001 Australian Software Engineering Conference. Canberra, ACT, Australia, Australia. pp. 68–75. ↩

  17. Manzoor, Numan; Munir, Hussan; Moayyed, Misagh (27–30 November 2012). Comparison of Static Analysis Tools for Finding Concurrency Bugs. 2012 IEEE 23rd International Symposium on Software Reliability Engineering Workshops. Dallas, TX, USA. pp. 129–133. ↩

  18. Dustin, Elfriede (28 December 2002). Effective Software Testing: 50 Ways to Improve Your Software Testing. Addison-Wesley Longman. p. 186. ISBN 0201794292. 0201794292 ↩

  19. Binder, Robert V. (1999). Testing object-oriented systems: models, patterns, and tools. Addison-Wesley Longman. ISBN 0-201-80938-9. 0-201-80938-9 ↩

  20. Binder, Robert V. (1999). Testing object-oriented systems: models, patterns, and tools. Addison-Wesley Longman. ISBN 0-201-80938-9. 0-201-80938-9 ↩

  21. Lu, Shan; Park, Soyeon; Seo, Eunsoo; Zhou, Yuanyuan (1–5 March 2008). Learning from mistakes: a comprehensive study on real world concurrency bug characteristics. ASPLOS XIII Proceedings of the 13th international conference on Architectural support for programming languages and operating systems. Seattle, WA, USA. pp. 329–339. ↩