Identifying and Documenting Failure Points

Cognitive walkthrough evaluation reaches its full value only when the problems it uncovers are captured with enough precision and structure to drive real design change. Identifying failure points is not simply a matter of noting that "something felt wrong" — it requires a disciplined process of recognizing when and why a design stops supporting the user's mental process, classifying those breakdowns by type and severity, and recording them in a form that other team members can act on. This module covers every dimension of that process: what constitutes a failure, how to categorize and rate failures, how to write descriptions that are genuinely useful, how to use documentation templates efficiently, and how to distinguish severity levels in a way that reflects the experience of the target user rather than the evaluator's own expertise.

What Constitutes a Failure Point?

At its core, a cognitive walkthrough tests whether a design adequately supports the user's cognitive process at each step of a task. An evaluator works through each step by asking a structured set of questions — typically probing whether the user would know what to do next, whether they would notice the correct action, whether they would understand the feedback they receive, and so on. A failure point is any step at which the honest answer to one of those questions is "no" or even "probably not." The design has failed to provide what a typical novice user's mind needs at that moment.

This definition is deliberately broad because failures exist on a wide spectrum of severity. At one end, a failure might be a label that is slightly ambiguous — the user pauses, reads it twice, and proceeds correctly. That is still a failure, because the design required extra cognitive work it should not have demanded. At the other end, a failure might be a missing affordance so severe that the user has no reasonable path forward and abandons the task entirely. Both are failures; they simply sit at different points on the severity scale. The evaluator's obligation is to record both.

A subtlety worth internalizing is that a single interface element can simultaneously produce failures against multiple evaluative questions. Consider a button labeled "Process" placed in a toolbar alongside similar-looking controls. It might fail the question "Will the user notice this action is available?" because it is visually indistinct. It might also fail the question "Will the user know this action leads to their goal?" because the label is opaque. And it might fail the question "Will the user correctly interpret the system's response after clicking?" because the feedback is a status bar update far from where their attention is focused. All three failures stem from one element, but they must be recorded separately because they point toward different design remedies.

Evaluators sometimes hesitate to record failures that seem trivial. This is a mistake. Minor failures compound. A user who experiences a small moment of confusion at step 2, a slightly misleading label at step 4, and an ambiguous confirmation dialog at step 6 may abandon the task at step 7 — not because any single failure was catastrophic, but because the accumulated cognitive friction eroded their confidence and patience. Documenting minor failures preserves the ability to see those cumulative patterns during analysis.

Categorizing Failures by Type

Once a failure is identified, the next step is to classify it. The most productive classification scheme ties each failure to the specific evaluative question it violates. The standard cognitive walkthrough questions address roughly four cognitive stages: goal formation (does the user know what they are trying to accomplish?), action selection (does the user know which action to take?), action execution (can the user perform the action?), and feedback interpretation (does the user understand what happened?). Mapping every failure to one of these categories immediately reveals the character of the design's problems.

If a large proportion of failures cluster in the action-selection category, the interface is likely suffering from a structural labeling or information architecture problem — users understand the goal and can physically perform actions, but they cannot tell which action is the right one. If failures cluster in feedback interpretation, the problem is probably in the system's response layer: confirmations, error messages, and status updates are not communicating adequately. These structural patterns would be invisible if failures were simply listed in chronological order without categorization.

Beyond the evaluative-question dimension, failures can also be categorized by severity. A common three-tier scheme uses low, medium, and high ratings. High-severity failures prevent task completion or cause the user to take an action that is harmful or irreversible. Medium-severity failures allow task completion but introduce significant frustration, wasted time, or a meaningful reduction in user confidence. Low-severity failures are noticeable but cause only minor friction that most users will work through without lasting consequence. The table below illustrates how these categories interact with failure types in practice.

Categorization makes findings actionable in a second important way: it allows the design team to distinguish between isolated errors and systemic problems. A single medium-severity failure in step 3 of one task may be a localized issue with a specific widget. Five medium-severity failures all classified as "action selection — unclear label" across three different tasks signal a systemic problem with the interface's labeling conventions — a problem that a terminology audit or a consistent labeling pattern could resolve across the entire product at once.

Writing Effective Failure Descriptions

The quality of a failure description determines whether a design team can act on a finding or must go back to the evaluator for clarification. Vague descriptions like "the button is confusing" or "the feedback is inadequate" delay fixes because they leave the designer guessing about exactly what aspect of the element failed and why. Effective descriptions follow a predictable structure that answers four questions immediately: Where did the failure occur? What would the user think or do incorrectly? What evaluative question does this violate? What might fix it?

The location reference should be explicit: task number, step number, and the specific interface element. "Task 2, Step 4, the 'Finalize' dropdown menu" gives a designer an exact coordinate in the interface. The behavioral description should be written from the user's perspective, not the interface's — not "the dropdown does not clearly indicate its function" but "the user, seeking to confirm their order, would overlook the 'Finalize' dropdown entirely because its visual weight is identical to the 'Sort by' and 'Filter' dropdowns nearby; they are likely to scroll down looking for a prominent confirmation button instead." This user-perspective framing keeps the evaluation grounded in human cognition rather than drifting into a technical critique of the interface's implementation.

Linking the failure to a specific evaluative question creates traceability — the ability to follow a finding from the raw evaluation record back through the analysis to any design recommendation that emerges from it. This matters when a design team needs to justify a change to stakeholders, or when a later evaluation needs to verify that a previously identified failure has been resolved.

The optional redesign suggestion is worth including whenever one comes naturally to the evaluator, because it shortens the design team's path to a solution. It should be framed as a direction rather than a prescription: "Consider giving the 'Finalize' action a visually distinct treatment — a filled button in a contrasting color, positioned below the item list rather than in the toolbar, to signal that it is a terminal action rather than a sorting control." This frames the suggestion as a hypothesis for the design team to evaluate, not a mandate that bypasses their expertise.

The following example shows the difference between an inadequate and an effective failure description for the same problem.

INADEQUATE:
Task 2, Step 4 — The confirm button is not obvious enough.

EFFECTIVE:
Task 2, Step 4 — Interface element: "Finalize" dropdown in the toolbar.
Evaluative question violated: Q2 — Will the user know that this action leads to their goal?
Severity: High
User behavior predicted: The user's goal at this step is to confirm and submit their order.
They are scanning the screen for a clear confirmation affordance (e.g., a prominent "Confirm"
or "Submit" button). The "Finalize" label is unfamiliar terminology for this context and the
dropdown format suggests a configuration control rather than a terminal action. The user is
likely to scroll past the toolbar entirely, searching below the order summary for a more
conventional button. They may attempt to use the browser's back navigation in frustration.
Redesign direction: Replace the dropdown with a clearly labeled primary-action button
("Submit Order") placed directly below the order summary, using a filled style that
distinguishes it from secondary controls.

Using a Documentation Template

Cognitive walkthroughs generate a high volume of fine-grained observations in a relatively short time. Without a pre-defined structure to capture those observations, evaluators face a real risk of losing details in the moment — particularly when a single step surfaces multiple simultaneous failure points. A documentation template solves this problem by providing a fixed set of fields that the evaluator fills in rather than constructs from scratch, reducing the cognitive overhead of documentation and allowing full attention to remain on the evaluation itself.

An effective template includes fields for every piece of information that will be needed during analysis and reporting: task identifier, step number, the action being evaluated, the interface element involved, each evaluative question with a pass/fail indicator, the failure description (when applicable), severity rating, and an optional redesign note. Crucially, the template should have a row or entry for every step evaluated — not just steps where failures occur. Recording successful steps is important for several reasons. It confirms that the evaluator examined those steps, provides context for understanding which parts of the flow work well, and prevents the completed evaluation from appearing to document only problems, which can distort stakeholders' perception of the overall design quality.

When multiple evaluators conduct a walkthrough independently — a best practice for increasing the reliability of findings — standardized templates make it straightforward to merge their results. Because every evaluator uses the same fields in the same structure, a lead analyst can systematically compare entries for the same step across evaluators, identify where findings agree (increasing confidence in those failures) and where they diverge (warranting discussion about the discrepancy), and produce a consolidated finding set without having to reconcile incompatible formats.

The completed set of templates is itself a formal deliverable. It serves as an audit trail of the evaluation process — evidence of what was examined, when, and by whom. This matters in professional contexts where design decisions are scrutinized, in regulated industries where usability testing may be part of a compliance record, and in any situation where a future team needs to understand the basis for a design revision. A thorough template record makes the evaluation reproducible and its findings defensible.

A representative template structure for a single step might look like the following.

Distinguishing Severity Levels

Severity rating is one of the most consequential judgments an evaluator makes, because it directly determines the priority order in which the design team addresses failures. Rating severity too high across the board produces a finding set in which everything is marked urgent, making it impossible to triage effectively. Rating severity too low causes genuinely critical problems to be treated as polish items and shipped to users without correction. Getting it right requires a principled approach and a specific frame of reference.

The defining characteristic of a critical failure is that it prevents the target user from completing the task, or leads them to take an action that is harmful, irreversible, or deeply counterproductive. Examples include a required field that offers no visible input affordance, a workflow that deadlocks with no escape route, or a confirmation dialog whose language so closely resembles a cancellation confirmation that users routinely trigger the wrong outcome. These failures must be resolved before the interface is released, because shipping them means a definable proportion of users will be unable to accomplish core tasks or will make significant errors.

A moderate failure allows task completion but at a real cost. The user finds a path forward, but they do so after wasted effort, visible frustration, or a period of genuine uncertainty about whether they are on the right path. This erosion of confidence matters because it affects more than the immediate task — users who feel uncertain about an interface become reluctant to attempt more advanced features, more likely to contact support, and more likely to form a negative overall impression of the product. Moderate failures should be addressed in the current design cycle, not deferred.

A minor failure produces a momentary hesitation or a small increase in cognitive load that the user overcomes quickly and without lasting consequence. A slightly ambiguous icon that becomes clear when a tooltip appears, a progress indicator that takes a fraction of a second longer to appear than ideal, a secondary label that could be worded more precisely — these are genuine failures worth recording, but they are candidates for a later iteration rather than an immediate fix, particularly when development resources are constrained.

The most important calibration principle is that severity must be assessed from the perspective of the defined novice user, not the evaluator. Evaluators are, by definition, domain experts compared to the target user population. An evaluator might find a non-obvious feature annoying but manageable — they know enough about similar interfaces to infer what the interface intends. A novice user encountering the same feature may have no such inferential resources and may be completely blocked. Allowing the evaluator's own experience to anchor the severity rating systematically undercounts the severity of problems that affect novice users disproportionately. The corrective is to continually ask: "Given what a first-time user of this type of software knows and does not know, how would this failure affect them?" — and to let that answer, not the evaluator's personal experience, determine the rating.

A cognitive walkthrough is only as useful as the record it leaves behind. As you step through each action in a task and ask the guiding questions — will the user know what to do, will they notice the control, will they understand the feedback — the moments where the answer is "no" or "probably not" are your failure points. This topic is about recognizing those moments, naming them precisely, and writing them down in a form that another designer can act on.

A failure point is more than "this screen is confusing." It is a specific breakdown at a specific step, tied to a cause: the user cannot form the right goal, cannot find the control, cannot connect the control to their intention, or cannot tell from the feedback that they made progress. Categorizing failures by type keeps your findings diagnostic rather than vague, and a consistent documentation template (step number, expected action, what fails, why, evidence, proposed fix) makes your results comparable across evaluators and reusable later in the design cycle.

The last piece is severity. Not every failure deserves the same attention, and a list of twenty undifferentiated problems gives a development team no guidance. As you watch the videos below, pay attention to the criteria used to rate severity, the language used to describe a problem without blaming the user, and the distinction between findings produced by inspection methods like the walkthrough and findings produced by testing with real users.

NNgroup · 3:04 · 9,970 views · 4y ago · captions — Watch on YouTube

About this video

A short Nielsen Norman Group video on assigning severity ratings to design issues found in usability studies or heuristic evaluations. It explains the small set of criteria evaluators typically weigh — such as how many users are affected, how much the issue impedes task completion, and whether it is a one-time or persistent obstacle — and how those criteria combine into a priority ranking. The focus is on reporting findings so that teams know what to fix first.

Why this video

This is the most direct match for the "Distinguishing Severity Levels" subtopic and for turning a raw list of failure points into a prioritized set of recommendations; NN/g is the standard reference source for usability evaluation practice.

NNgroup · 5:54 · 66,865 views · 4y ago · captions — Watch on YouTube

About this video

A five-minute NN/g video covering common mistakes made when talking to users, including leading rather than probing, over-explaining, and filling silence in ways that distort what the participant tells you. It contrasts neutral probing questions with questions that plant an answer. The emphasis is on gathering evidence that reflects the user's actual experience rather than the researcher's assumptions.

Why this video

The same discipline of neutral, non-leading language applies when you write failure descriptions — describing the observable breakdown instead of asserting your preferred explanation keeps your documentation credible and useful.

Kerev Design · 6:18 · 47,165 views · 4y ago · captions — Watch on YouTube

About this video

This video distinguishes user testing from usability testing, two terms that are frequently confused, and explains what question each method is designed to answer. It situates both within the broader UX research process and discusses when a team would choose one over the other.

Why this video

Knowing what usability testing does and does not tell you clarifies the place of an inspection method like the cognitive walkthrough, and helps you label your findings honestly as predicted failure points rather than observed user behavior.

Together these videos reinforce the documentation and prioritization half of the module. The severity-rating video supports MO3 directly by giving you explicit criteria for ranking findings by user impact, and it supports MO2 by showing what a finished, reportable issue entry should contain. The interviewing video models the precise, non-leading language that makes failure descriptions written during a walkthrough defensible, again serving MO2. The user-testing-versus-usability-testing video frames where inspection methods such as the cognitive walkthrough sit among evaluation approaches, which supports MO1 by helping you recall why the method's guiding questions are structured as predictions about user reasoning rather than observations of users.

These videos were chosen from YouTube on October 3, 2026. Each has captions; the player turns them on. Videos on YouTube can be changed or removed — if one no longer plays, tell your instructor. If a player does not appear, use its Watch on YouTube link.