Skip to main content
U.S. flag

An official website of the United States government

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Usability Test Quality Heuristics

The following indicators can help determine if a usability test will produce useful results.
This is not an exhaustive list, but it should be helpful in planning and assessing usability tests. Additions are welcome!

Study design

Indicator Good sign Bad sign
Study purpose Researcher clearly articulates the purpose of the usability test (for example, as a specific question to answer, an area of inquiry, etc.). Researcher does not specify a purpose for the test, or the purpose specified is very broad, like “testing the app” or “finding problems.”
Number of participants Tests include a sufficient number of participants necessary to see patterns, and there’s only one participant per session. Study includes only one participant, or includes more that two participants testing simultaneously (like a focus group).
Sampling bias Participants are a diverse set of people who actually use the application (or do the task the application is supposed to support). Participants are experts (vs. average users), friends, family, or close colleagues of the product team.

Moderator style

Indicator Good sign Bad sign
Question style Researcher asks follow up questions that are open-ended and non-leading. Questions are leading or subtly suggest potential solutions or hypotheses.
Task variability Researcher asks participants to complete similar tasks related to the study’s purpose. Researcher asks participants to complete different tasks, or tasks unrelated to the study’s purpose.
Framing Researcher asks participants to complete tasks with the product or service that align with their work-related goals. Researcher asks participants to complete tasks unrelated to their work-related goals. For example, asking a participant how they might send a fax when their job doesn’t call for that.
Priming Moderator asks participants to complete tasks without indicating how that might be done. For example, “How would you view the status of your application?” Researcher guides participants in completing tasks. For example, “Which of the links in the header would you click to login?”

Team participation

Indicator Good sign Bad sign
Number of teammates The team designates a single moderator for the test, and at least one member of the product team observes the usability test. A single person from the product team participates in and leads the test.
Observer role Observers do not moderate. They are generally quiet, and ask open-ended questions after the test has concluded. Observers interrupt the participant, or attempt to sell or explain the product. Observers debate the participant’s actions in their presence.

Sensemaking

Indicator Good sign Bad sign
Notetaking Tests are recorded or notes are taken for sharing with absent stakeholders. Tests are not recorded, or test results are not documented.
Synthesis Moderator debriefs with teammates immediately after each interview. Researcher looks for patterns across multiple participants and surfaces problems that affected several people. Moderator reports the most memorable problems without conducting affinity mapping or some other analysis technique.
Presentation of findings Researcher reports findings to team and stakeholders in an easy to follow, well prioritized way. Researcher presents the team a “basket of issues” or an unprioritized laundry list of potential changes.
Incorporation of findings Product team translates findings into future user stories or product refinements. Researcher reports do not affect the product backlog or ongoing development work.

18F User Experience Design Guide

An official website of the GSA’s Technology Transformation Services

Looking for U.S. government information and services?
Visit USA.gov