Usability Testing : In Pursuit of the Untampered Truth

“One does not like to make definite assertions unless one has a little more definite knowledge.” said the remarkable detective Miss Jane Marple in Agatha Christie’s Nemesis. What you might ask does the sweet (ahem seemingly sweet) Miss Marple have to do with facilitating a usability test? In my opinion, quite a bit (now now don’t roll your eyes just yet). Miss Marple’s unusual detective techniques were seemingly friendly & unassuming, shrewdly leading people to pour information they thought was harmless into the gentle old lady’s hands. Oh, if only they knew.

In so many ways we are the same

Unlike Miss Marple a facilitator doesn’t usually find themselves confronted with a murder or a serious crime but their brilliant probing techniques can save a product from being criminally unusable

  1. The untampered truth – Both the facilitator and the detective are after the untampered truth, one about a serious crime another about a user’s true reactions to a product.
  2. Keen Observational Skills – Both need to be good listeners & observers in order to catch what is said and what is left unsaid.
  3. Question your questions – Both need to ask the right questions, one so that the suspect doesn’t become aware that they are under suspicion & another so that the user isn’t led towards an ideal answer as opposed to their true feelings about the product
  4. Bound by the hours on the clock (I don’t go to the drama it comes to me :/) – Both are on a timeline, admittedly one under more pressure than the other. The only thing that might happen if a usability test exceeds its time limit is the user killing the facilitator. Just kidding guys, hasn’t happened yet so I think we are safe.
  5. An open mind. – Both need to keep their opinions, thoughts in check till they have enough evidence to support their premises. One might have a hunch but still needs to test it to get to the truth & the same goes for the facilitator. If either of them let their preconceived notions color what they are observing it can lead the operation astray.
  6. Staging a test – This is more about how a given technique works differently for both. A usability test is conducted in a lab environment and this can affect the insights given as the user isn’t in their natural environment and knows they are being observed. This leaves the task of making the user comfortable, and a little closer to their natural way of using the product, to the facilitator. On the other hand, a detective staging a situation might just catch the killer!

The facilitators truth detection tool kit

Here are some tips that will help you in your noble pursuit of weeding out unusable design (or senseless & unlawful design, too dramatic? I say not enough!). Its best to catch bad features before they kill your product, right?

Become one with the observer in you
Listen & observe! Listening to the user is one of the most important points a facilitator needs to remember in a usability test. Along with listening also observe the user’s body language and other non-verbal cues the user may display. This can help formulate more relevant probing questions. For eg: A user may have a confused expression on their face while they go about looking for the search icon but once they find it they may forget all about it and not even mention it when asked about features of the interface they found difficult. Observing the user carefully helps pick up on so many such insights. A facilitator can change many gears during a usability test from asking the right probing questions to stepping back and letting the user take the driver’s seat while they use the product. Before the user knows, she is talking about the placement (or misplacement) of a button she wasn’t even asked about and is surprised by her own words on how much it annoys her. (Miss Marple would be proud!)

Let him complete dammit!
Allow the user to complete their points before asking them any questions. In fact, it helps to have a brief pause between the user’s answer and your following question. The user might have more to add and the above technique of observing the user’s nonverbal cues gives us hints as to whether the user has finished with their answer or they might have more. I always have had and still have problems with the so called “awkward” silences but I have seen them work and can attest to their benefits. From a user’s perspective, the facilitator is just having a friendly conversation about a product & is inquisitive about the user’s thoughts but there is so much more going on below the surface. One of the main maxims of usability testing is that valuable insights result from feedback the users give unprompted.

Memory – That thing we rely on? Yeah not so reliable
Another important thing to keep in mind is that when questioned about something in the past or the future it’s human nature to bend the truth to make it more appealing or socially acceptable. For example: when asked a user might say pastel colors on a webpage are calming and easy on the eye but in reality, they might have difficulty in finding what they are looking for in a lightly colored website. So, if during a test the user talks about an experience try and see if you can make them replicate it. Their recollections should always be taken with a pinch of salt as they may not be the truth.

Be open to a good chat
Ask open ended questions. As opposed to just YES or NO as answers, open ended questions provide more valuable insights from the user. They may share motivations or concerns we didn’t account for while building a better almost complete picture about why they feel the way they do about a topic. Ask questions that start with “how” or with words that begin with “w,” such as when, where, what, why etc. It’s very important to know the WHY behind a user’s actions. So, a facilitator has to dig out the truth by seeing how the user is reacting to the product in real time and asking questions that nudge them to give their unvarnished opinions. This is done by listening attentively, paying attention to the user’s body language, probing them at appropriate stages in the test and last but not least keeping the facilitator’s own judgements, biases or thoughts in check.

The goal is to unearth the truth

In any good detective novel, the reader is kept guessing till the end (in a real-life crime though the sooner the end the better coz entertainment is usually not a factor, not for anyone except the media I guess :/). Imagine a reader as the thirsty desert wandered looking for the oasis of a solution only to be led astray by what their own mind pays attention to and it usually is paying attention to its hallucinations. This definitely muddles the reader but in a real-life situation believing ones imagined theories and then following them to find a solution can lead a facilitator or detective down a long-twisted path to nowhere. The facilitator and the detective have trained themselves not only in the art of observing outer clues but also in being aware of their own way of dealing with the situation. What neither of them forget is that how they think and behave will influence how fast and accurately a solution is reached. They are a part of the solution influencing it and not passive observers. As another one of my favorite detectives and all round beautiful character Dirk Gently famously said “The term `holistic’ refers to my conviction that what we are concerned with here is the fundamental interconnectedness of all things. I do not concern myself with such petty things as fingerprint powder, telltale pieces of pocket fluff and inane footprints.”. One can look at the obvious factors like what the user says and how quickly they navigate through an interface but sometimes it’s the other things like their facial expression or exasperated sighs that could lead to the deepest insights. And the truth is important because otherwise someone could be wrongly incriminated or a design team could be wrongly guided and then the facilitator would be rightly incriminated (yeah not a stressful job at all). Do not be in a hurry to connect the dots because they might be mirages that lead you on to an illusory solution.

And before I let you go (if you are still here :D)

Facilitating is a combination of multiple skills that enable the facilitator to comprehend the user’s need and match their steps resulting in a sort of dynamic dance towards valuable insights. It’s dynamic because every user is as unique as they come and the facilitator has to be able to sync in with them. It’s fun and never like a routine, unless you choose to make it. I know of people who cook like it’s a routine (how can you find making what sustains you boring!!) but then I rarely cook so I am no one to question them :p. Facilitating requires one to be a different person every time they conduct a test and I don’t know many people who can act (let alone well) with new variables every other hour. A facilitator might have a script but that’s the only constant. It might seem stressful and intimidating but do enough of them with the spirit of learning and you might actually thoroughly enjoy them.

Lastly, usability testing is needed to gage a product’s viability for the user and not to reconfirm the designers or facilitators arguments. Usability testing is very much like this quote, again by the lovely Mr Gently, “I may not have gone where I intended to go, but I think I have ended up where I needed to be.”

References

  • https://www.nngroup.com/articles/first-rule-of-usability-dont-listen-to-users/
  • https://www.nngroup.com/articles/field-studies-done-right-fast-and-observational/
  • https://www.nngroup.com/articles/open-ended-questions/
  • https://www.smartinsights.com/user-experience/usability/10-mistakes-avoid-usability-testing/

You might also want to check out