Recently we observed a participant in usability testing fail most of the tasks we set him. Surprisingly, when we asked him if he found anything difficult he responded, “No, not at all. BrandX.com is the best!”.
This is not uncommon. In another recent usability test we ran, many participants attributed mistakes to some fault of their own instead of embracing the uncomfortable alternative - the brand they love had a lousy website.
Recent psychology and neuroscience research point to an uncomfortable fact -most cognitive activity (think 95%) is happening in our subconscious andinaccessible to our conscious mind. It appears that the rational “conscious” part of our brain is being notified after a decision has been made by our subconscious. Our conscious brain then has the unenviable job of explaining and rationalising decisions made by our subconscious.
For example, I see a donut and think “That looks good, but I probably shouldn’t eat it. Then again, I didn’t eat breakfast so maybe I should.” I think I’m in control. In reality, my subconscious has already determined to eat the donut. Without me knowing, my conscious mind has rationalised my subconscious decision.
Similarly, test participants will tell us things that they genuinely believe is true - but their behaviour would suggest otherwise. A little frightening really isn’t it?
This makes me nervous about poorly constructed focus groups and surveys. If the research is true (there’s a lot of it to suggest it is) - it forces a rethink of qualitative research.
What do you think? How do you get around not only facilitator bias, but bias inherent with participants themselves?