Menu
Home
People
Places
Arts
History
Plants & Animals
Science
Life & Culture
Technology
Reference.org
Cognitive walkthrough
open-in-new
<h2 id="introduction">Introduction</h2> <p>A cognitive walkthrough starts with a <a href="/facts/Task_analysis/LzaFdGBP">task analysis</a> that specifies the sequence of steps or actions required by a user to accomplish a task, and the system responses to those actions. The designers and developers of the software then walk through the steps as a group, asking themselves a set of questions at each step. Data is gathered during the walkthrough, and afterwards a report of potential issues is compiled. Finally the software is redesigned to address the issues identified. </p><p>The effectiveness of methods such as cognitive walkthroughs is hard to measure in applied settings, as there is very limited opportunity for controlled experiments while developing software. Typically measurements involve comparing the number of usability problems found by applying different methods. However, Gray and Salzman called into question the validity of those studies in their dramatic 1998 paper "Damaged Merchandise", demonstrating how very difficult it is to measure the effectiveness of <a href="/facts/Usability_inspection_method/LRfWhNP0">usability inspection methods</a>. The consensus in the usability community is that the cognitive walkthrough method works well in a variety of settings and applications. </p> <h2 id="streamlined-cognitive-walkthrough-procedure">Streamlined cognitive walkthrough procedure</h2> <p>After the task analysis has been made, the participants perform the walkthrough:<a class="footnote-ref" id="fnref:1" href="#fn:1"><sup>1</sup></a> </p> <ol><li>Define inputs to the walkthrough: a usability specialist lays out the scenarios and produces an analysis of said scenarios through explanation of the actions required to accomplish the task. <ol><li>Identify <a href="/facts/End_user/7IM9qu7N">users</a></li> <li>Create a sample task for evaluation</li> <li>Create action sequences for completing the tasks</li> <li>Implementation of interface</li></ol></li> <li>Convene the walkthrough: <ol><li>What are the goals of the walkthrough?</li> <li>What will be done during the walkthrough</li> <li>What will not be done during the walkthrough</li> <li>Post ground rules <ol><li>Some common ground rules <ol><li>No designing</li> <li>No defending a design</li> <li>No debating cognitive theory</li> <li>The usability specialist is the leader of the session</li></ol></li></ol></li> <li>Assign roles</li> <li>Appeal for submission to leadership</li></ol></li> <li>Walk through the action sequences for each task <ol><li>Participants perform the walkthrough by asking themselves a set of questions for each subtask. Typically four questions are as <ul><li>Will the user try to achieve the effect that the subtask has? E.g. Does the user understand that this subtask is needed to reach the user's goal</li> <li>Will the user notice that the correct action is available? E.g. is the button visible?</li> <li>Will the user understand that the wanted subtask can be achieved by the action? E.g. the right button is visible but the user does not understand the text and will therefore not click on it.</li> <li>Does the user get appropriate feedback? Will the user know that they have done the right thing after performing the action?</li></ul></li> <li>By answering the questions for each subtask usability problems will be noticed.</li></ol></li> <li>Record any important information <ol><li>Learnability problems</li> <li>Design ideas and gaps</li> <li>Problems with analysis of the task</li></ol></li> <li>Revise the interface using what was learned in the walkthrough to improve the problems.</li></ol> <p>The CW method does not take several social attributes into account. The method can only be successful if the usability specialist takes care to prepare the team for all possibilities during the cognitive walkthrough. This tends to enhance the ground rules and avoid the pitfalls that come with an ill-prepared team. </p> <h2 id="common-shortcomings">Common shortcomings</h2> <p>In teaching people to use the walkthrough method, Lewis & Rieman have found that there are two common misunderstandings:<a class="footnote-ref" id="fnref:2" href="#fn:2"><sup>2</sup></a> </p> <ol><li>The evaluator doesn't know how to perform the task themself, so they stumble through the interface trying to discover the correct sequence of actions—and then they evaluate the stumbling process. (The user should identify and perform the <i>optimal</i> action sequence.)</li> <li>The walkthrough method does not test real users on the system. The walkthrough will often identify many more problems than you would find with a single, unique user in a single test session</li></ol> <p>There are social constraints that inhibit the cognitive walkthrough process. These include time pressure, lengthy design discussions and design defensiveness. Time pressure is caused when design iterations occur late in the development process, when a development team usually feels considerable pressure to actually implement specifications, and may not think they have the time to evaluate them properly. Many developers feel that CW's are not efficient because of the amount of time they take and the time pressures that they are facing. A design team spends their time trying to resolve the problem, during the CW instead of after the results have been formulated. Evaluation time is spent re-designing, this inhibits the effectiveness of the walkthrough and leads to lengthy design discussions. Many times, designers may feel personally offended that their work is even being evaluated. Due to the fact that a walk-through would likely lead to more work on a project that they already are under pressure to complete in the allowed time, designers will over-defend their design during the walkthrough. They are more likely to be argumentative and reject changes that seem obvious. </p> <h2 id="history">History</h2> <p>The method was developed in the early nineties by Wharton, et al., and reached a large usability audience when it was published as a chapter in <a href="/facts/Jakob_Nielsen_(usability_consultant)/faoYR7ku">Jakob Nielsen</a>'s seminal book on usability, "Usability Inspection Methods".<a class="footnote-ref" id="fnref:3" href="#fn:3"><sup>3</sup></a> The Wharton, et al. method required asking four questions at each step, along with extensive documentation of the analysis. In 2000 there was a resurgence in interest in the method in response to a CHI paper by Spencer who described modifications to the method to make it effective in a real software development setting. <a href="https://facweb.cdm.depaul.edu/cmiller/eval/p353-spencer.pdf">Spencer's streamlined method</a> required asking only two questions at each step, and involved creating less documentation. Spencer's paper followed the example set by Rowley, et al. who described the modifications to the method that they made based on their experience applying the methods in their 1992 CHI paper "The Cognitive Jogthrough".<a class="footnote-ref" id="fnref:4" href="#fn:4"><sup>4</sup></a> </p><p>Originally designed as a tool to evaluate interactive systems, such as postal kiosks, automated teller machines (ATMs), and interactive museum exhibits, where users would have little to no experience with using this new technology. However, since its creation, the method has been applied with success to complex systems like CAD software and some software development tools to understand the first experience of new users. </p> <h2 id="see-also">See also</h2> <ul><li><a href="/facts/Cognitive_dimensions/QS2s13sr">Cognitive dimensions</a>, a framework for identifying and evaluating elements that affect the usability of an interface</li> <li><a href="/facts/Comparison_of_usability_evaluation_methods/VKr271fC">Comparison of usability evaluation methods</a></li></ul> <h2 id="further-reading">Further reading</h2> <ul><li>Blackmon, M. H. Polson, P.G. Muneo, K & Lewis, C. (2002) Cognitive Walkthrough for the Web CHI 2002 vol. 4 No. 1 pp. 463–470</li> <li>Blackmon, M. H. Polson, Kitajima, M. <a href="https://www.researchgate.net/profile/Muneo_Kitajima/publication/221514993_Repairing_usability_problems_identified_by_the_Cognitive_Walkthrough_for_the_Web/links/00463537cb1c7e994e000000.pdf">(2003) Repairing Usability Problems Identified by the Cognitive Walkthrough for the Web CHI</a> 2003 pp497–504.</li> <li>Dix, A., Finlay, J., Abowd, G., D., & Beale, R. (2004). Human-computer interaction (3rd ed.). Harlow, England: Pearson Education Limited. p321.</li> <li>Gabrielli, S. Mirabella, V. Kimani, S. Catarci, T. (2005) Supporting Cognitive Walkthrough with Video Data: A Mobile Learning Evaluation Study <a href="/facts/MobileHCI/xfnmol5q">MobileHCI</a> ’05 pp77–82.</li> <li>Goillau, P., Woodward, V., Kelly, C. & Banks, G. (1998) Evaluation of virtual prototypes for air traffic control - the MACAW technique. In, M. Hanson (Ed.) Contemporary Ergonomics 1998.</li> <li>Good, N. S. & Krekelberg, A. (2003) Usability and Privacy: a study of KaZaA P2P file-sharing CHI 2003 Vol.5 no.1 pp137–144.</li> <li>Gray, W. & Salzman, M. (1998). Damaged merchandise? A review of experiments that compare usability evaluation methods, <i>Human-Computer Interaction vol.13 no.3</i>, 203–61.</li> <li>Gray, W.D. & Salzman, M.C. (1998) Repairing Damaged Merchandise: A rejoinder. Human-Computer Interaction vol.13 no.3 pp325–335.</li> <li>Hornbaek, K. & Frokjaer, E. (2005) Comparing Usability Problems and Redesign Proposal as Input to Practical Systems Development CHI 2005 391–400.</li> <li>Jeffries, R. Miller, J. R. Wharton, C. Uyeda, K. M. (1991) User Interface Evaluation in the Real World: A comparison of Four Techniques Conference on Human Factors in Computing Systems pp 119 – 124</li> <li>Lewis, C. Polson, P, Wharton, C. & Rieman, J. (1990) Testing a Walkthrough Methodology for Theory-Based Design of Walk-Up-and-Use Interfaces Chi ’90 Proceedings pp235–242.</li> <li>Mahatody, Thomas / Sagar, Mouldi / Kolski, Christophe (2010). State of the Art on the Cognitive Walkthrough Method, Its Variants and Evolutions, International Journal of Human-Computer Interaction, 2, 8 741–785.</li> <li>Rizzo, A., Marchigiani, E., & Andreadis, A. (1997). <a href="https://s3.amazonaws.com/academia.edu.documents/8651930/10.1.1.97.3355.pdf?response-content-disposition=inline%3B%20filename%3DThe_AVANTI_project_prototyping_and_evalu.pdf&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWOWYYGZ2Y53UL3A%2F20200212%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200212T093717Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=69b7f0d0d02ba3e92f25c7ee26ec60b83bb0f4697dd4f481fe2f154d43231c73">The AVANTI project: prototyping and evaluation with a cognitive walkthrough based on the Norman's model of action</a>. In Proceedings of the 2nd conference on Designing interactive systems: processes, practices, methods, and techniques (pp. 305-309).</li> <li>Rowley, David E., and Rhoades, David G (1992). The Cognitive Jogthrough: A Fast-Paced User Interface Evaluation Procedure. <i>Proceedings of CHI '92</i>, 389–395.</li> <li><a href="/facts/Andrew_Sears/edUhvo8k">Sears, A.</a> (1998) The Effect of Task Description Detail on Evaluator Performance with Cognitive Walkthroughs CHI 1998 pp259–260.</li> <li>Spencer, R. (2000) <a href="https://facweb.cdm.depaul.edu/cmiller/eval/p353-spencer.pdf">The Streamlined Cognitive Walkthrough Method, Working Around Social Constraints Encountered in a Software Development Company</a> CHI 2000 vol.2 issue 1 pp353–359.</li> <li>Wharton, C. Bradford, J. Jeffries, J. Franzke, M. Applying Cognitive Walkthroughs to more Complex User Interfaces: Experiences, Issues and Recommendations CHI ’92 pp381–388.</li></ul> <h2 id="external-links">External links</h2> <ul><li><a href="http://www.usabilitybok.org/cognitive-walkthrough">Cognitive Walkthrough</a></li></ul> <h2 id="references">References</h2> <ol> <li id="fn:1"><p>Spencer, Rick (2000). "The streamlined cognitive walkthrough method, working around social constraints encountered in a software development company". Proceedings of the SIGCHI conference on Human Factors in Computing Systems. The Hague, The Netherlands: ACM Press. pp. 353–359. doi:10.1145/332040.332456. ISBN 978-1-58113-216-8. S2CID 1157974. <a href="978-1-58113-216-8" target="_blank">978-1-58113-216-8</a> <a href="#fnref:1" class="footnote-back-ref">↩</a></p></li> <li id="fn:2"><p>Lewis, Clayton; Rieman, John (1994). "Section 4.1: Cognitive Walkthroughs". Task-Centered User Interface Design: A Practical Introduction. pp. 46–54. Retrieved April 10, 2019. <a href="http://hcibib.org/tcuid/chap-4.html#4-1" target="_blank">http://hcibib.org/tcuid/chap-4.html#4-1</a> <a href="#fnref:2" class="footnote-back-ref">↩</a></p></li> <li id="fn:3"><p>Wharton, Cathleen; Riemann, John; Lewis, Clayton; Poison, Peter (June 1994). "The cognitive walkthrough method: a practitioner's guide". In Nielsen, Jakob; Mack, Robert L. (eds.). Usability inspection methods. John Wiley & Sons. pp. 105–140. ISBN 978-0-471-01877-3. Retrieved 2020-02-11. {{cite book}}: |website= ignored (help) <a href="978-0-471-01877-3" target="_blank">978-0-471-01877-3</a> <a href="#fnref:3" class="footnote-back-ref">↩</a></p></li> <li id="fn:4"><p>Rowley, David E; Rhoades, David G (1992). "The cognitive jogthrough: A fast-paced user interface evaluation procedure". Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '92. pp. 389–395. doi:10.1145/142750.142869. ISBN 0897915135. S2CID 15888065. <a href="0897915135" target="_blank">0897915135</a> <a href="#fnref:4" class="footnote-back-ref">↩</a></p></li> </ol>