Chapter 9: Usability testing on 10 cents a day

WHY USERS LIKE MINDLESS CHOICESKEEPING TESTING SIMPLE -- SO YOU DO ENOUGH OF IT

Introduction

  • Usability is too often done:
    • too little
    • too late
    • for all the wrong reasons

Repeat after me: Focus groups are not usability tests.

  • Focus Group
    • groups of 5-8
    • react to what is shown
    • opinions
    • feelings
    • good for determing audience wants, needs, and likes -- in the abstract
    • should be done BEFORE designing the website
    • ARE NOT USABILITY TESTING
  • Usability Test
    • one person at a time
    • asked to:
      1. figure it out
      2. complete a typical task

Several true things about testing

  • If you want a great site, you've got to test
    • Your view of your website becomes biased very quickly. You know too much.
    • Testing is like having friends visit from out of town.
  • Testing one user is 100% better than testing none.
  • Testing one user early in the project is better than testing 50 near the end.
  • The importance of recruiting representative users is overrated.
  • The point of testing is not to prove or disprove something. It's to inform your judgement.
  • Testing is an iterative process (make, test, fix, test, fix, test, ... )
  • Nothing beats a live audience reaction.

Lost our lease, going-out-of-business-sale usability testing

How many users should you test?

  • 3 or 4 makes it more likely that you'll test again soon
  • 3 or 4 keeps your notes legible
  • 3 or 4 allows you to fix problems between tests so other errors can be found

Recruit loosely and grade on a curve

  • You don't need someone from your target audience
  • The prime person to test is anybody who knows internet basics
  • Reasons to grade on the curve:
    • We're all beginners under the skin.
    • It's usually not a good idea to design a site so that only your target audience can use it.
    • Experts are rarely insulted by something that is clear enough for beginners
  • Exceptions:
    • If your site is going to be used almost exclusively by one type of user and it's no harder to recruit from that group
    • If your audience is split between clearly defined groups with very divergent interests and needs
    • If your site requires specific domain knowledge
  • When recruiting:
    • Offer a reasonable incentive
    • Keep the invitation simple
    • Avoid discussing the site (or the organization behind the site) beforehand
    • Don't be embarrassed to ask friends or neighbors

Where do you test?

  • office or conference room

Who should do the testing?

  • Someone who is calm, empathetic a good listener, and inherently fair

Who should observe?

  • Anyone and everyone who has anything to do with it

What do you test, and when do you test it?

  • Test a comparable site first
  • Live site testing
    "Get it" testing
    Show them the site and see if they get its purpose, value, structure, etc.
    Key Task testing
    Ask the user to do something and watch how well they do it
  • It's better to give the user a task they have a hand in choosing