When to go guerilla with recruiting

Amanda Stockwell is President of Stockwell Strategy, a UX research practice focused on lean research methods and integrating user knowledge with business goals to create holistic product strategies for organizations of all sorts. View her #LinkedInLearning courses on UX Research Basics, Card Sorting, and Interviewing.

Everyone has heard of guerilla research, the magical solution to getting feedback “in the wild” quickly and cheaply. Many in the Lean and Agile circles have latched on to guerilla research as the only way to do UX research. While I’m a huge fan of pretty much anything that helps us pragmatically collect and integrate research, guerilla research has its time and place.

Wait, what exactly is “guerilla research”?

Let’s start by defining exactly what we mean when we say guerilla research. The dictionary.com definition of guerrilla is:

1. a member of a small independent group taking part in irregular fighting, typically against larger regular forces
2. referring to actions or activities performed in an impromptu way, often without authorization

I certainly hope you’re not irregularly fighting against larger regular forces or acting without authorization, though as someone who has worked in non-research friendly environments, I feel your pain. The second definition better fits what we mean by guerilla research.

In fact, guerilla research is just a way to say that you’ve taken a regular UX research method and altered it in some way to reduce time and cost. This typically means that you also reduce the rigor of the method. The most common method to “guerilla-ize” is usability testing; instead of recruiting participants to come into a lab setting, you can stake out a spot at a coffee shop and recruit random patrons to participate in short sessions or you can conduct the study remotely. You may also reduce the total number of people you talk to so that you can shorten the length of time it takes to get results. Guerilla research has been praised as a way to integrate research into shorter and shorter timelines and limited budgets.

Things to keep in mind about guerilla testing

Right so, guerilla testing sounds pretty awesome. Why doesn’t everyone just guerilla-ize everything if you can get good feedback in a shorter time frame?

Well, remember how I said that in order to get those time and resources savings, you’ll have also have to reduce the rigor of your research? That can be OK, but can also lead to some risks, especially when it comes to the participants involved in your research.

Remember that research participants are supposed to be actual or representative users of your service or product. Of course, whatever you build should be easy enough for most people to use, but you can miss out on some quality feedback if someone isn’t really a target user. Perhaps they won’t really care about your product or won’t have the same underlying knowledge that your real users would have.

For instance, I recently helped conduct usability testing of some tax software and we were trying to determine the usability of finding specific forms. Although we’d recruited tax professionals and thought they’d have the knowledge needed, none of our participants could find the form. We realized part-way through the sessions that our participants didn’t typically use that version of the form in their daily work. We couldn’t confidently tell whether they really had trouble locating the item or if they got stuck because they were unfamiliar with the form and process associated with it. We’d recruited and done some screening, but we still ended up with inconclusive results.

Even if you do have a product that doesn’t require specific background knowledge, you still may run into biased results. Let’s say you set up shop at a cafe to test first impressions of a new filtering process for shoes on an ecommerce site. You may talk to five people, but those five people are all patrons of a specific shop, in a specific neighborhood, in a certain time period.

Think about it; the visitors to a fancy cafe in a business area at lunchtime are going to be very different than the visitors to a cheap spot off a highway in the middle of the night. This means that you have to carefully choose your guerilla location. No matter where you choose, you’re more likely to over-represent a particular point of view when everyone comes from the same context.

One other note about approaching people in public to take part in your studies; be aware that people are likely going to be surprised and a little unsure of what to expect. You don’t have as much time to build rapport or give context, and you might be in the middle of a public place. All of this means that people may be more hesitant than usual to be frank and forthright, especially when discussing sensitive topics. If your topic of conversation is health, money, sex, or relationship-related, random intercepting may not be the recruiting tool for you.

There are also plenty of online recruitment panels that you can use; some online tools have panels built in and some panels are separate. Independent recruitment panels often allow some screening and allow you to get quick turnaround, but you may also encounter some issues.  Only a certain subset of people gets and takes the opportunity to be a part of a panel, and there have been lots of shared instances of the professional user research participant, who will “frequently supplement their income by participating in user research… and say and do whatever it takes to get into a study.”

Using an online research tool’s integrated recruitment panels, like Optimal Workshop’s group of participants, allows you to get quality guarantees about your participants. If you can tell that they lied about their qualifications or really aren’t the right fit, you can flag that person and get a replacement participant.

So when do I really need to carefully recruit?

Guerilla research really is awesome, but there are some times when investing time and money into carefully recruiting participants will be worth it.

Here’s a handy chart to summarize when it’s really worth the investment in recruiting and when to go guerilla:

Guerilla recruiting	More rigorous recruiting
When you’re working on a product that doesn’t require specific domain knowledge	When background/domain knowledge is really important to your study goals
When you’re gathering first impressions of something anyone may see, such as a landing page	When you’ll be discussing sensitive topics such as health, money, sex, or relationships
When budget or time is very cramped and the alternative is no research at all	If you have the luxury of time and resources on your side

I would love to hear about your experiences with recruiting participants, both good and bad! Share your funny story about recruiting mishaps, and I may include it in my session about getting the best participants for your research at UX New Zealand. Leave a comment below or Tweet me on @MandaLaceyS. See you there!

Want to hear more? Come to UX New Zealand!

If you’d like to hear more about getting the right participants for your research, plus a bunch of other cool UX-related talks, head along to UX New Zealand 2017 hosted by Optimal Workshop. The conference runs from 11-13 October including a day of fantastic workshops, and you can get your tickets here.