User Research

Inspection Methods

When the subject of Usability and user testing comes up, there are typically two elements that deter the process from evolving. The first is time and cost. It’s true that time is required to develop proper testing, build user profiles, test users in the field or in a lab, and gather the final results for analysis. The cost of all those parts can be considerable, if we simply look at empirical Usability testing, which is often and erroneously promoted as the only way to go. That’s not to say that empirical Usability testing isn’t valuable. It is indeed highly valuable. Taking time and cost into consideration though, there is an even more costly road that companies often take: redeveloping after market goals are not attained. Redeveloping in the aftermath is often damaging to a developing company, but it can be very damaging to a large company trying to develop new and innovative technology in the hopes of gaining new revenue platforms that will prevent stagnation. Simply put, performing quality research and analysis of user needs before development yields greater return on investment.

The following list is drawn from the book, Usability Inspection Methods, by Jakob Nielson and Robert L. Mack. They did not invent the methods, but are responsible for culling them all together in meaningful text. I always like to offer up the following list as a way to start working toward interacting with the user community, and allaying any fears about usability inspection methods. You’ll see that we have already covered some points in the list, but a deeper inspection with greater attention is warranted. You’ll also see that some of the points may not applicable. These can be easily recognized and judged. Finally, these methods are not set in stone. They can be combined and even built upon to create a hybrid inspection method(s) that suits business needs.

• Heuristic evaluation: This is the most informal method and involves having usability specialists judge whether each dialogue element conforms to established usability principles. These principles are normally referred to as the heuristics and give the method its name.
• Guideline reviews: These are inspections where an interface is checked for conformance with a comprehensive list of usability guidelines. However, since guideline documents contain on the order of 1000 guidelines, guideline reviews require a high degree of expertise and are fairly rare in practice. The method is somewhat of a cross between a heuristic evaluation, discussed previously, and a standards inspection, which will be discussed further on in this list.
• Pluralist walkthroughs: These are meetings where users, developers, and human factors people step through a scenario, discussing usability issues associated with dialogue elements involved in the scenario steps. This allows the information designer/tester and developer to work closely with defined goals that help to achieve the ultimate goal of timely completion and delivery to market, as well as the most usable product possible.
• Consistency inspections: These types of inspections have designers that represent multiple projects inspect an interface to see whether it does things in a way that is consistent with their own designs. Thus, consistency inspections are aimed at evaluating consistency across the family of products that has been evaluated by an inspection team.
• Standards inspections: These types of inspections have an expert on some interface standard inspect the interface for compliance. Thus, standards are aimed at increasing the degree to which a given interface is in the range of other systems on the market that follow the same standards.
• Cognitive walkthroughs: Cognitive walkthroughs use a more explicitly detailed procedure to simulate a user’s problem-solving process at each step in the human-computer dialogue, checking to see if the simulated user’s goals and memory for actions can be assumed to lead to the next correct action.
• Formal usability inspections: A formal usability inspection is intended to be very similar to the code inspection methods with which many developers are already familiar. In this method, the various participants have well-defined responsibilities: a moderator is appointed to manage both individual and focused inspections, and the full team inspection meeting; a design owner is responsible for design and redesigns; the inspectors have the job of finding problems; and a scribe records all defects and issues identified during the meetings. Inspections are performed through a six-step process: planning, a kickoff meeting, a preparation phase where inspectors review the interface individually, the main inspection review when the inspectors’ lists of usability problems are merged, and a follow-up phase where the effectiveness of the inspection process itself is assessed.
• Feature inspections: These focus on the functionality to be delivered – the usefulness of interface function, and not simply the usability of the interface as an implementation of the function. For example, whether the function, as designed, meets the needs of intended end-users. Feature inspections can involve not only evaluation of a function, but can also involve the design of that function.

In order to get the most out of any inspection method, User & Task Analysis must also be taken into consideration. Developing User Profiles helps to focus on user goals, macro and micro tasks, and ultimately develop a true requirements document that combines business requirements and user goals.