Heuristic Evaluation and Nielsen's 10 Principles — Topics & Learning Outcomes

Module Topics

Introduction to Heuristic Evaluation

• What Is Heuristic Evaluation? — Heuristic Evaluation is a discount usability engineering method that allows experts to identify usability problems in a user interface design without the need for live user testing.
• Origins and Context — Nielsen and Molich developed the heuristic evaluation method in 1990, later refining the set of heuristics to the ten canonical principles that are widely used today.
• When to Use Heuristic Evaluation — HE is appropriate at multiple points in the design lifecycle, from sketches and wireframes through high-fidelity prototypes and live products.
• Advantages and Limitations — HE offers speed, low cost, and the ability to evaluate interfaces without recruiting participants, but it also carries inherent limitations related to evaluator expertise and coverage.

Nielsen's Ten Usability Heuristics

• Heuristic 1 — Visibility of System Status — The system should always keep users informed about what is going on through appropriate feedback within reasonable time.
• Heuristic 2 — Match Between System and the Real World — The system should speak the users' language, using words, phrases, and concepts familiar to the user rather than system-oriented terminology.
• Heuristic 3 — User Control and Freedom — Users often choose system functions by mistake and need a clearly marked 'emergency exit' to leave the unwanted state without having to go through an extended dialogue.
• Heuristic 4 — Consistency and Standards — Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform and industry conventions consistently.
• Heuristic 5 — Error Prevention — Even better than good error messages is a careful design that prevents problems from occurring in the first place.
• Heuristic 6 — Recognition Over Recall — Minimize the user's memory load by making objects, actions, and options visible; the user should not have to remember information from one part of the dialogue to another.
• Heuristic 7 — Flexibility and Efficiency of Use — Accelerators — unseen by novice users — may often speed up interaction for expert users so that the system can cater to both inexperienced and experienced users.
• Heuristic 8 — Aesthetic and Minimalist Design — Dialogues should not contain irrelevant or rarely needed information, as every extra unit of information competes with and diminishes the relative visibility of relevant information.
• Heuristic 9 — Help Users Recognize, Diagnose, and Recover from Errors — Error messages should be expressed in plain language, precisely indicate the problem, and constructively suggest a solution.
• Heuristic 10 — Help and Documentation — Even though it is better if the system can be used without documentation, it may be necessary to provide help; such information should be easy to search and focused on the user's task.

The Heuristic Evaluation Process

• Preparing for the Evaluation — Before beginning an evaluation, evaluators must understand the system's intended users, primary use cases, and scope of the interface to be reviewed.
• Conducting the Individual Evaluation — Each evaluator independently inspects the interface, walking through it multiple times and noting any elements that violate one or more of the ten heuristics.
• Documenting Usability Problems — Each usability problem must be recorded with enough detail that someone unfamiliar with the evaluation can locate and understand the issue.
• Number of Evaluators — Research shows that the optimal number of evaluators for HE is between three and five, as this range captures the best return on investment for problem discovery.

Severity Rating

• Purpose of Severity Ratings — Severity ratings help design and development teams prioritize which usability problems to fix first given limited time and resources.
• Nielsen's Severity Rating Scale — Nielsen's standard severity scale runs from 0 to 4, with 0 indicating no usability problem and 4 indicating a usability catastrophe that must be fixed before launch.
• Factors Influencing Severity — Severity is determined by three key factors: the frequency with which the problem occurs, the impact it has on users, and the persistence of the problem across the interface.
• Assigning and Recording Ratings — Each evaluator assigns severity ratings independently before any aggregation discussion to prevent anchoring and groupthink from distorting individual assessments.

Aggregating Findings Across Evaluators

• Why Aggregation Is Necessary — Because different evaluators identify different subsets of problems, combining findings from multiple evaluators produces a more complete and reliable picture of the interface's usability issues.
• The Aggregation Meeting — Evaluators come together in a structured meeting to present their findings, deduplicate overlapping issues, and negotiate final severity ratings for each problem.
• Deduplication and Problem Merging — Deduplication involves identifying problems that multiple evaluators reported and deciding whether they represent the same underlying issue or distinct but related problems.
• Negotiating Severity Across Evaluators — When evaluators assign different severity ratings to the same problem, the group must reach a consensus rating through evidence-based discussion.

Writing Formal Heuristic Evaluation Reports

• Report Structure and Components — A well-structured HE report typically includes an executive summary, evaluation context, methodology description, consolidated findings table, and recommendations section.
• Writing Effective Problem Descriptions — Problem descriptions must be precise, neutral, and specific enough that a designer can locate and understand the violation without having been part of the evaluation.
• Formulating Recommendations — Recommendations translate each identified problem into actionable design guidance that the team can implement in the next iteration.
• Using Visual Evidence — Annotated screenshots and screen recordings significantly increase the clarity and credibility of HE reports by providing direct visual evidence of each violation.
• Report Quality Standards — Professional HE reports must be clear, consistent, and credible — free of vague language, unsupported claims, and internal contradictions.

Evaluating Peer Prototypes

• Module Assignment Overview — Each student independently evaluates three peer prototypes using Nielsen's ten heuristics, assigns severity ratings, and produces a formal HE report for each prototype evaluated.
• Maintaining Evaluator Independence — The independence requirement means students must complete their individual evaluations and draft reports before viewing or discussing findings with any other evaluator of the same prototype.
• Professional Conduct in Peer Evaluation — Evaluating a peer's prototype requires maintaining a professional, constructive tone that focuses on interface design decisions rather than on the designer's choices or abilities.
• Calibrating Evaluations Across Prototypes — Applying a consistent standard across three different prototypes requires active calibration — evaluators must ensure they are not applying stricter or looser criteria as they progress through evaluations.

Student Learning Outcomes

By the end of this module, students will be able to:

Course Outcomes (reference)