Designing User Proficiency Surveys

Introduction

Before a participant ever touches your prototype, you need to know something about them. A user proficiency survey is a short pre-test instrument that captures a participant's background, their familiarity with relevant technology, and their knowledge of the task domain. Without it, usability data is hard to interpret: a confusing task result might reflect a flawed interface, or it might simply reflect a participant who has never used software of that kind before.

In this topic you will decide which proficiency dimensions actually matter for your project (for example, frequency of use of similar tools, years of domain experience, device preferences, accessibility needs), then write questions that measure those dimensions without leading, biasing, or confusing the respondent. You will also think about structure and administration: question order, response scales, survey length, and when and how the survey is delivered relative to the test session.

The three videos below come from UX practitioners and focus on the craft of asking questions well. As you watch, pay close attention to the specific wording problems they flag — leading questions, double-barreled questions, vague scales, assumptions baked into the prompt — and keep a running list of fixes you can apply directly to your own pre-test survey draft.

Video 1: UX Research: How to Avoid Survey Mistakes

UX in the Head · 9:33 · 8,758 views · 3y ago · captions — Watch on YouTube

About this video

This video looks at why surveys are such a popular UX research method and then walks through the common mistakes that produce unreliable data. Based on its description, expect coverage of flawed question wording, poor survey design choices, and the kinds of errors that lead to results you cannot trust. It is framed around diagnosing and avoiding problems rather than simply praising surveys as a method.

Why this video

It speaks directly to the "writing effective survey questions" and "structuring and administering the survey" subtopics, and the mistake-focused framing gives you a practical checklist to review your own proficiency survey against before you send it to participants.

Video 2: 5 Tips for creating effective user research questions (+ free question bank) | Maze

Maze · 9:26 · 3,993 views · 3y ago · captions — Watch on YouTube

About this video

Maze, a research and testing platform company, offers five tips for creating effective user research questions and points to a free question bank you can draw from. The video's premise is that research quality is capped by question quality, so it concentrates on asking the right things in the right way. The accompanying question bank gives you concrete phrasing examples to adapt.

Why this video

The tip-based format and the downloadable question bank are immediately useful when you are drafting items for proficiency dimensions such as technology familiarity and domain experience, and the producer builds research tooling used in industry.

Video 3: UX Research: Asking The Right User Interview Questions

UX in the Head · 10:03 · 13,837 views · 3y ago · captions — Watch on YouTube

About this video

This video focuses on user interview questions: which ones to ask, which to avoid, and how phrasing shapes the answers you get. Although it is framed around live interviews rather than written surveys, the underlying principles of neutral, non-leading, answerable questions transfer directly. It also helps you see when a question is better asked face-to-face than on a form.

Why this video

It reinforces question-writing discipline from a second angle and helps you decide how to administer your proficiency screening — as a written pre-test survey, as a brief verbal intake, or as a combination of both.

How These Videos Support the Learning Outcomes

These videos support MO2 most directly: a proficiency survey is one of the data collection instruments in your user testing plan, and the responses you gather also inform your participant criteria and screening decisions. Learning to write neutral, precise questions means the background data you collect will actually explain patterns in your test results rather than muddy them. Indirectly, this work also supports MO1 and MO3, since knowing your participants' skill levels clarifies which prototype features must be functional and verifiable before testing begins.

About These Videos

These videos were chosen from YouTube on October 3, 2026. Each has captions; the player turns them on. Videos on YouTube can be changed or removed — if one no longer plays, tell your instructor. If a player does not appear, use its Watch on YouTube link.