Quelle: (Newman, Lamming 95).
We tend to use Cognitive Walkthroughs, then, when we want to assess operation by users who are exploring the system's user interface and learning as they go. Particular measures we will be looking for concern users' success rates in completing tasks, and their ability to recover from errors. We should not expect to measure users' speed of task performance. We need a fairly complete description of the user interface in order to conduct Cognitive Walkthroughs, because we need to cover all possible routes that the user may take. However, we don't need to know the user's sequence of operation, because the analysis itself helps us to discover what the sequence is likely to be.
(0) The user starts with a rough plan of what he or she wants to
achieve - a task to be performed;
(1)The user explores the system, via the user interface, looking
for actions that might contribute to performing the task;
(2)The user selects the action whose description or appearance
most closely matches what he or she is trying to do;
(3)The user then interprets the system's response and assesses
whether progress has been made towards completing the task.
The analysis involves simulating steps 1, 2 and 3 at each stage of inter action, by asking questions of the form:
Q1: Will the correct action be made sufficiently evident to the
user?
Q2: Will the user connect the correct action's description with
what he or she is trying to do?
Q3: Will the user interpret the system's response to the chosen
action correctly, that is, will the user know if he or she has
made a right or a wrong choice?
The result of performing a Cognitive Walkthrough is usually to discover problems in these three areas, that is, where the questions receive a 'No' answer. Solutions to these problems are fed into the next iteration of design.
Figure 1. The design for a rapid-transit ticket machine, to be analysed by Cognitive Walkthrough.
We will work through an analysis of the machine's use by a first-time user. Let us suppose this user wishes to purchase a round-trip ticket to Dragon Plaza. We will add a complication: the traveller has only ten dollars in cash, but doesn't know this at the outset.
Our first step in the walkthrough is to answer the task-definition question:
Q0: What does the user want to achieve?
Answer: Purchase a round-trip ticket to Dragon Plaza.
Now we can enter into the walkthrough analysis itself. The initial display is as shown in Figure 1. We start by asking question Q1 about this display:
Q1: Will the correct action be made sufficiently evident to the
user?
Answer: There are two possible correct actions, press the
'Dragon Plaza' button or press 'round-trip'. The design doesn't
make this clear, for it instructs the user to choose the
destination before indicating the journey type. This doesn't
impede the user's progress, but it hides an available option.
Thus we have identified Design Flaw no. 1: Option to indicate
journey type first is not made sufficiently evident. One
possible way to rectify this flaw might be to provide a prompt
via a larger display (see Figure 2).
Figure 2. Making the range of methods more obvious to the user
The user is thus likely to be aware of only one correct action, pressing the 'Dragon Plaza' button. Following the walkthrough analysis sequence, we will now ask questions Q2 and Q3 about this action:
Q2: Will the user connect the correct action's description with
what he or she is trying to do?
Answer: Yes, the instructions for panel 1 and the button label
will enable the user to make the connection.
Q3: Will the user interpret the system's response to the chosen
action correctly, that is, will the user know if he or she has made a
right or a wrong choice?
Answer: The machine will respond by lighting up the button
pressed, as shown in Figure 3. This should appear to the user as
confirmation of a correct action.
Figure 3. Confirming the user's choice of destination with a lighted button.
The user must now indicate the journey type, using panel 2. We apply the same walkthrough steps as before:
Q1: Will the correct action be made sufficiently evident to the user?
Answer: Yes. The correct action is to press the 'round-trip'
button in panel 2, and the instructions labelling this panel
make it clear that this is the next step.
Q2: Will the user connect the correct action's description with
what he or she is trying to do?
Answer: Yes, the instructions and the labels on the buttons make
the connection very clear.
Q3: Will the user interpret the system's response to the chosen
action correctly, that is, will the user know if he or she has made a
right or a wrong choice?
Answer: Yes. The machine will respond by lighting up the
'round-trip' button and by displaying the journey type and fare, as shown in
Figure 4. This provides confirmation of this and the previous
action. If the user should press the 'one-way' button by
mistake, this too will be made clear.
Figure 4. The machine's response to selecting the journey type.
We'll now proceed to the next action, depositing money.
Q1: Will the correct action be made sufficiently evident to the
user?
Answer: Yes. Again, the user's main source of assistance is the
numbered sequence of instructions. According to this sequence
the next step is to deposit money.
Q2: Will the user connect the correct action's description with
what he or she is trying to do?
Answer: Yes, a request to deposit money is consistent with
purchasing a ticket.
Q3: Will the user interpret the system's response to the chosen
action correctly, that is, will the user know if he or she has made a
right or a wrong choice?
Answer: Unclear. We need to know what kind of response the
machine will provide to a correct action, that is, to depositing the
first portion of the fare. If the machine merely swallows the
money, the user will be little the wiser. According to Figure 4,
there is no means of indicating receipt of the money, and thus
no means for the user to keep track of the amount deposited.
This could be considered Design Flaw no. 2: No display of total
money received. A solution might be to extend the display as
shown in Figure 5.
Figure 5. Improving the design to show the amount paid so far.
The completion of the walkthrough of the normal ticket-purchase sequence is left as an exercise for the reader. Let us conclude this example by instead introducing the slight complication we've held in reserve - the passenger who has only ten dollars, and who discovers this only after depositing all of it.
Q1: Will the correct action be made sufficiently evident to the
user?
Answer: No, because there is no action that the user can take to
retrieve the money deposited. We have discovered Design Flaw no.
3: No means of retrieving money deposited. The design should
include a 'return money' button as shown in Figure 6.
Figure 6. Adding the capability to retrieve money deposited.
This initial analysis has been very useful, in pointing out three design flaws:
The analysis will not stop there, however. A design such as this will need to be subjected to a full range of walkthrough analyses, covering all of the likely tasks that users will want to perform, under all likely conditions.