Recruiting sucks. If you can afford it, hire a recruiting company. If not, here are some tips to make it hurt less. Big lesson: make sure participant fidelity is proportionate to prototype fidelity.
Finding representative users for your site visits and studies is hell. Ensuring a steady stream of them can be a nightmare.
Using the pros
The easiest but most expensive solution is to hire a recruiting company. People like Market Decisions will recruit any type of participant for any type of study in (almost) any part of the world. If the person doesn’t show or doesn’t meet the criteria, they’ll find a replacement. They handle everything from finding and qualifying individuals though to calling them the day before to remind them to show up.
Some of these recruiting companies are more used to finding people for focus groups or other marketing style studies. The magic words to use are “one on one in-depth interview.” This tells them that you want to recruit a certain number of people to arrive at your location one person at a time for about a one to two hour session. Once you make this clear, the rest of the interaction will proceed much more smoothly.
Obviously you pay for this service. In fact, you can end up paying more to recruit each individual than you end up giving the person in gratuity fees. Sometimes this is worthwhile if the cost is still less than the opportunity cost of doing it yourself. Often however you just don’t have the cash to hire a recruiter.
DIY recruiting
If you are time-rich and cash-poor, you’ll end up doing all the recruiting tasks yourself. Be prepared for a headache.
First you have to work out the attributes that you are looking for in a study participant. Then, you have to find a large number of people who are interested in helping you out, and carefully match them against the attributes that you defined.
Then you have to make sure they can attend at the time and place that you are running the study, you have to send them directions, and you’ll have to give them a reminder call to ensure they actually show up.
You can expect to recruit anything from one in ten to one in one hundred of your initial people (depending upon how well you prequalified them). Of this group, about 30% will just fail to turn up despite all the promises that they made to you. Of the remaining participants, about 1 in 5 will not match the attributes you recruited for as well as you’d like.
It sucks. However, once you get into the right mindset it’s not exactly a hard task. The benefits of running the studies also far outweigh the initial pain in recruiting participants.
Qualifying your participants
Whether you use an agency or do it yourself, you need to know what type of people you’re looking for. If you are using personas as a development aid, base your recruiting efforts around finding people who match your persona characteristics.
Don’t make life harder than it has to be. Remember that even though you made your persona male, there could be many women who share the same traits. You might need to relax age ranges a bit, and instead focus on other personal qualities. If you need medical professionals, do they have to be fully qualified or would students who have taken a certain set of basic classes be acceptable?
Write down the attributes that you want your participants to have. Now rearrange the list so that the most important attributes are at the top. You’ll want to qualify people on these most important attributes first in order to save both you and the respondent time if they aren’t a good match.
Now turn the attributes into questions which don’t give away the answer that you are looking for. You will need to have a set of questions that you ask each respondent in order to qualify them for your study. Asking respondents to choose from a couple of options or asking an open-ended question which has a well-defined answer will stop people from trying to guess what it is that you want them to say.
Where to find people
There are lots of potential recruits out there – it’s just a question of getting in touch with them.
Cheaper
- Other offices in your building (if your product is general-purpose)
- Advert on your site/mailing list/social media (you’ll get “selection bias” – you’re only recruiting from people who already know about you)
- Post to social media/message board/forum/club site
- Ask friends and family to suggest (you’ll run out of suggestions fast – don’t ever run studies with your friends or family because it can be messy)
- Classified advert (e.g. Craigslist)
- Ask your sales people very nicely if you can visit some clients (good luck with that…)
- Go to where these people hang out, grab them
More money involved
- Advert on a suitable site
- Advert using Google Adwords
- One of the many online usability testing sites (but then you lose the richness of a face to face session)
- Use temps from a temporary staffing agency. There are agencies for many lines of work. You can either get individuals for half day assignments, or if you have a more involved task set you can design a study that uses a temp for a couple of days at a time (they’ll need breaks!). This often works out cheaper than other methods because the “gratuity” is just the agency fee.
Typically you’ll want to gather people’s contact information through these channels, and then follow up with a phone call where you ask them the qualifying questions. In that phone call, make it clear that you aren’t selling anything and that participants receive a gratuity for taking part.
How much to pay?
The going rate for well qualified business professionals can be $250 or up for a one to one and a half hour study session. Students and less hard-to-reach roles can be had for $50 to $75. Typical office workers will normally consider giving their time for $150 per session.
Shocked at how much it’ll cost you? Remember, those are the figures in dollars. There are other ways of rewarding participants and the dollar cost can be quite a bit lower.
The first thing to consider is making the study as fun as possible. People seem to have more fun if they are asked to create something or if they are being shown something exclusive. People who are fans of your product may value a tour of your offices or a chance to meet with some of their “heroes” more than cash.
A car review site hosted some group sessions at the studio where they take photos of new model cars. After the session they had pizza and beer and let the session attendees see the process they use for capturing images of each new vehicle. Some of the review writers were there to chat with these enthusiasts as well. It didn’t hurt that they had a straight-from-the-factory next model year sports car sitting in the studio at that time. It would be hard to put a dollar value on the experience that the attendees had that evening.
Busy executives are probably too expensive for you to afford, but if you can get past their PA and convince them that you aren’t selling something, they’ll often take part just for the fun of it. This type of person is typically used to expressing opinions and giving feedback.
Another consideration is payment “in kind.” For shorter studies this may mean giving people a t-shirt or other piece of marketing schwag with your logo on it. For a longer study, can you give people a year’s subscription to your service, free support, a certain number of free downloads, or something else that would be equally valuable to you in terms of customer retention as it would be to the participant?
Microsoft rewards usability study participants with software. The value to participants is much higher than the manufacturing costs for the organization.
One administrative note: In the USA, if you give any individual more than a certain value of cash or product in each year then it becomes taxable income and needs the appropriate paperwork. The easiest way to deal with this is to avoid the situation by not recruiting the same individual for more than one or at most two studies. This is a good general practice anyway as repeat study participants have a tendency to “go native” and start to take on the traits of the development team rather than the attributes that you initially recruited them for.
The personal approach
If your study is likely to take less than five minutes and is mobile, consider going to where your target audience hangs out. You can get many responses in a short space of time. You’ll need a very brief recruiting script to sell them on helping out, but people can be very co-operative. It helps if your study is actually fun or exciting.
- Everyone has to eat. Where do your target audience eat lunch?
- Commuters waiting for a train won’t have much else to do
- People standing in long lines may be good candidates (Apple stores on launch day?) but you may need permission if you are recruiting on private property.
You can entice people to participate with a reward such as a free coffee (make sure it’s from the place they are eating at!) or smaller denomination gift cards.
You can also use this approach to recruit people for later studies at your location. Be sure to gather their information and ask brief pre-qualification questions. If you just leave them with your contact details they will be much less likely to get in touch.
Participant fidelity is proportionate to prototype fidelity
If you are testing your first-draft conceptual paper prototypes, you’re probably more concerned about whether someone can figure out the general interaction principles rather than whether you’ve got every piece of information in exactly the right place.
That means you can relax your recruiting criteria accordingly. For early low-fi work, you can often call on co-workers who aren’t involved in your project, or other individuals who meet the expected technical expertise level of your proposed user base but without necessarily having all the domain knowledge that real users would bring to the table.
The closer that you get to real code, the closer your recruits will need to match your criteria. Once you start testing real content, it’s essential to have real users.
I once tested an application for stock traders. We had real traders looking at paper prototypes. We scrolled a piece of paper across the “screen” with made-up prices for the different stocks. We nearly caused heart attacks when our numbers were off from what the traders expected. Obviously their domain knowledge – essential for proper understanding of the real interface – was too advanced for the level of our prototype.
How to handle poor matches
Believe it or not, there are “professional” participants out there. If you recruit from Craigslist or if you are recruiting in an area where many other companies are doing the same thing, you will come across people who attend focus groups and usability tests for a living.
There’s nothing wrong with this, but these professional testers are NOT typical users. They’ve learned how to anticipate the answers to recruiting questions which will qualify them for a study, regardless of whether or not that’s the truth. Even the recruiting agencies are taken in sometimes. When the individuals start creating multiple aliases in order to avoid blacklisting, there’s something very weird going on… If you encounter one of these individuals, or if your participant is just plain not what they said they were, you’re probably best off just cancelling the session. You may find that even if the person isn’t a perfect match for the study you had in mind, you can still gather useful information from them about their habits and behaviors. Maybe they use a competitor product and can give you a guided tour, or they might be an expert user who uses your existing products on a daily basis.
In any situation where the participant is a bad match, you should still give them the gratuity. If they showed up they deserve it. Even if you deeply suspect that the person is a serial test-taking participant who scammed their way into your study, they still get the gratuity. Just be sure to keep a note of their details so you never recruit them again.
What next?
Now you’ve found your people, you need to run a study. Are you doing a field visit, a paper prototype test or a usability study on some alpha or beta code?