by Gary Teagarden and Jennifer Bruns
Conducting Usability Tests
This informative chapter covers usability testing for document and software design.
There are 10 steps in the process of usability testing:
1. Decide when to test
2. Select the test points
3. Choose the type of test
4. Set performance and learning objectives
5. Select testers and evaluators
6. Prepare the test materials
7. Set up the test environment
8. Record information accurately
9. Interpret the data
10. Incorporate the feedback
1. Decide When to Test
You can test at any time during the nine stages of the documentation development process. Usually you test after you have a draft finished so you can see the areas that need testing. But you can test during the three major phases: design, writing or development.
2. Select the Test Points
A test pint is an issue or feature of a document that might interfere with the efficient and effective application of a program to a user’s work activities. Test points fall into two areas: problems with content and problems with document design. Test points could be body text size, heading size, cropped screens vs. whole screens, cues for steps and page orientation.
3. Choose the Type of Test
There are three types of tests—dependent upon the test points selected in step two.
Can-they-do-it test
Can-they-understand-it test
Can-they-find-it-test
4. Set Performance and Learning Objectives
Because you want your tests to measure actual behavior, you must come up with numbers that correlate with the kind of performance you want from your users. These are often called operational objectives. There are two broad categories of performance objectives: Time-related and error-related.
5. Select Testers and Evaluators
The tester is the person who administers the test, arranges the meeting with users, sets up the test situation, records the test activities, and so on. The evaluator is the person taking the usability test.
6. Prepare the Test Materials
Depending on the complexity of your usability test, the written and other materials you supply for testers and evaluators can get complicated. See Tables 8.5 and 8.6 (P.248-249) in the text for a comprehensive list of potential test materials.
7. Set Up the Test Environment
The environment for your test may range from the user’s work environment (the field) to a controlled laboratory. Your best chance to learn about actual use in the context of the user’s work and information environment comes from field testing.
8. Record Information Accurately
Recommend using voice and video recorders, plus take copious notes so that you don’t miss anything.
9. Interpret the Data
Interpretation requires you to take into account all the elements that can go wrong with testing so that you get clear results.
10. Incorporate the Feedback
Following the interpretation your data then the next step is to incorporate the results into your final design.
The author talks about the test paradox: The earlier you test the weaker the results but the easier it is to make changes; the later you test the better the results but the harder it is to make changes.
Some large companies, such as IBM have full-blown usability labs replete with two-way mirrors, video cameras and eye-tracking devices. Ironically, often the best results from usability testing comes from field testing. In addition, field testing is less expensive. But field research poses issues of time, politics, budgets, ethics, and legality that require you to proceed carefully.
Subscribe to:
Post Comments (Atom)
12 comments:
This chapter was very useful to me since I've never ran a usability test before. I read it back when we began our usability project, and it was a great introduction to how to begin and execute the whole project. I find myself referencing the chapter while completing different parts of the project.
I found this to be a succinct summary of a useful chapter. As someone who hasn't conducted many usability tests, this chapter provides in depth information about how to conduct a usability test. Even the information about the kinds of tests seemed simple enough but it wasn't something I'd heard of before.
Reading this chapter helped a lot in both planning and conducting our group's usability tests. Personally, after having read it, II felt a lot more confident about the direction we would take. After conducting the tests, I found myself wishing that I had some of the two-way mirrors, video cameras and eye-tracking devices that IBM has, as there were so many different things to look out for during the testing; it was difficult to catch it all!
Usability testing is really eye opening. This is my second class that has exposed me to usability testing. I've found the field of usability testing to really be expanding in just the last couple of years. I probably receive some sort of advertising from usability consultants at least three times a month. With most of the working world glued to computer screens, the field seems to be wide open. But the funny thing is, most people don't know what usability testing is. Once it's explained to someone, it seems like one of those "duh" moments for most. As technical communicators, we need to continue to talk about the benefits of usability testing and make sure that it's part of our budgets no matter if you're working with a large corporation or developing Web pages for a small business. Sure, the eye tracking and high end testing labs are cool, but you can get solid results from simply observing someone navigate software or a Web site sitting at a kitchen table with a lap top computer.
I have read parts of this chapter several times while doing our own usability project. Since I knew absoultely nothing about the testing, this was very useful and insightful for me.
This chapter is good reference for testers before and during usability testing (obviously, since the author recommends testing throughout the entire process).
I thought that expanding on #5 may be worthwhile. I see evaluation as a part of anyone's role who's involved in the usability test, whether they are the test writers or test takers. The testing team needs to be aware of the usability problems, whether they be in the test or the software. Running the test with the test writers will serve great purpose as everyone will provide useful feedback.
Like the other posters, this chapter exposed me to an important field that I knew very little about. Even after reading the chapter, I think I still feel a little overwhelmed by the variety of documents that technical communicators could potentially be dealing with in their jobs. I suppose it's mostly an issue of scale, but some people focus on relatively succinct materials, while others work on lengthy tomes with hundreds or thousands of topics. I've only ever worked on small projects, so the amount of work and coordination involved in a larger project is something that I have difficulty wrapping my head around. It seems like usability testing a lengthy documentation product would require some impressive managerial skills. I suppose it takes a village to complete projects like that--it's unlikely to find very many people with all of the skills required.
Very nice summary Jennifer and Gary! I think it is helpful that we are in the process of doing this at the same time so we can actually apply this at the same time. I think that helps to reinforce the lesson.
I've done some usability tests before, but I think it's important to note that they can be conducted in many different ways, but that the main point of the study is to figure out what will work for your users.
As many others I felt this was valuable summary and chapter overall due to the usability project assignment. The part of the chapter on setting performance and learning objectives were particularly useful when we were designing the test forms. Interpreting the data section is most practical for me as I am primarily responsible for combining and interpreting the data of our usability testing.
This chapter was a helpful reminder of the ENG575-Usability class with Dr. Nord. Since we got to apply this theorethical knowledge on an in-class project in this term as well, this chapter reading was definitely time well spent.
Our group usability project is my first experience with usability testing. Conducting my tests was an interesting experience. Not only did I learn things about the website we were testing, I also learned things about how I would do the testing different if I had it to do over again. I was surprised to find that my test guinea pigs had a hard time understanding that they weren't being tested, but that the website was being tested. They felt bad when they weren't able to complete a task, almost like they had failed me. For future tests, I would spend more time making the evaluator feel comfortable and reemphasizing somehow that their skills were not what was being tested.
This article provided a nice summary of usability testing -- a step that absolutely must be completed before launching new software or technology.
This became apparent during the testing we did before "going live" with our electronic medical record. Users such as nurses, lab techs, therapists, and physicians followed test scripts that were written to mimic a patient's stay in the hospital. Many, MANY problems were revealed -- thankfully, in time for us to fix them.
Post a Comment