Tuesday, January 16, 2007
Using focus groups to define tasks for usability testing
You have a new project to do a usability test of a web site. After defining the site's audience groups, you face your biggest challenge — defining the tasks to use in the testing. You've interviewed the site's key stakeholders and gained a solid picture of what they think their site's visitors want to do. (You certainly know what they want the visitors to do.) But do you have a complete and accurate picture of the visitors' goals and objectives? You have only one way to ensure that you do — obtain input directly from people who might visit the site.
One common approach to eliciting such input involves focus groups. Jakob Nielsen has called the use of focus groups "voodoo usability" (link will open in a new window), but he is criticizing their use to gather design ideas or evaluate a site. I agree with Jakob that for those purposes, focus groups can do more harm than good. However, I am convinced that they can (if done well) provide valuable information regarding what site visitors want to do.
And I think I've done it well. In a recent project at UserWorks (new window), I introduced a focus group technique that I learned from a nonprofit organization. Here's how it works:
Thus, each focus group produced a list of six primary tasks for its audience segment, and identified potential design issues that could affect task performance. In the usability testing that followed the focus groups — which involved the same two audience segments — we designed our test procedures to use those six tasks with the participants from the respective audiences.
We at UserWorks found this technique both effective and cost effective. In particular, it makes voting somewhat private and thus avoids much of the social pressure that "public" voting can carry, and it tends to concentrate the votes by ensuring that only those items that are among someone's top choices receive any votes at all.
Best of all, it ensures that usability testing focuses on tasks that are important to the target audiences.
One common approach to eliciting such input involves focus groups. Jakob Nielsen has called the use of focus groups "voodoo usability" (link will open in a new window), but he is criticizing their use to gather design ideas or evaluate a site. I agree with Jakob that for those purposes, focus groups can do more harm than good. However, I am convinced that they can (if done well) provide valuable information regarding what site visitors want to do.
And I think I've done it well. In a recent project at UserWorks (new window), I introduced a focus group technique that I learned from a nonprofit organization. Here's how it works:
- Spend 15-20 minutes brainstorming all the things that the group might want to do on the site. Make this a pure brainstorming session, with no assessing or arguing about what people contribute.
- Spend another five minutes or so looking for items that may be combined into one.
- Hand out lined 4x6 index cards and ask participants to write down the five items that they would place at the top of their list. Have them write each item on a separate line, with blank lines between them.
Here's where it gets tricky. - When you say "Go," all participants simultaneously pass their cards one person to the left.
- After each passing, participants look at the list of five items that they're holding and place a mark (|) by their top
items from that list. - When all participants have marked their top three, pass again. (It can help to instruct them to make their mark diagonally if the card they're holding has a group of four.)
- When the cards have gone all the way around and participants have their own cards back, stop passing. (Participants do not get to make the three marks on their own cards.)
- Total the results and select the top five or six. These will become your tasks for usability testing.
- Then run the group through the top tasks, and ask targeted questions. This can give insights into design issues that you will need to watch out for during testing.
Thus, each focus group produced a list of six primary tasks for its audience segment, and identified potential design issues that could affect task performance. In the usability testing that followed the focus groups — which involved the same two audience segments — we designed our test procedures to use those six tasks with the participants from the respective audiences.
We at UserWorks found this technique both effective and cost effective. In particular, it makes voting somewhat private and thus avoids much of the social pressure that "public" voting can carry, and it tends to concentrate the votes by ensuring that only those items that are among someone's top choices receive any votes at all.
Best of all, it ensures that usability testing focuses on tasks that are important to the target audiences.