One More Reason that Replication Efforts in Science are Failing
A dialogue has started on how to deal with the replication crisis in science — where scientists often find it difficult to uncover results that are similar to existing peer-reviewed articles (many of which are highly cited, touted as facts in textbooks, and the source of great acclaim in TED talks and best-selling tradebooks). A quick review can be found here. Let me introduce another issue worthy of consideration to improve the quality of scientific research. We need to get locker room discussions about what scientists do to recruit and retain research participants into the spotlight.
Why do so many scientists study college students? The answer: they are a low cost, captive audience that are often mandated by course instructors to complete research studies. This is not the case for feral adults. Trying to recruit adults from the community is a daunting endeavor. Should you post flyers on supermarket bulletin boards? Craigslist ads? Facebook groups? Notes on car windows? A Hare Krishna approach, using free dead flowers to activate the reciprocity principle of persuasion?
My Well-Being Lab is recruiting participants for two large-scale community studies. We scoured the scientific literature to uncover the recruitment strategies by other scientists and found far too many sparse method sections. If we are going to take replication seriously, more information is needed in empirical papers. Not just the study procedures but how people are recruited and from where.
I know of clinical trial researchers who increase compliance by giving away free breakfast — and it worked! I know another researcher that explicitly recruits physically attractive research assistants to be on the front lines of recruiting participants. Sex sells and in this context, sex persuaded people to complete laboratory experiments. These acts are nowhere to be found in published articles. What else goes unreported in recruiting and maintaining participant involvement that is central to meaningful outcomes?
Here is a paper that deviates from the wall of silence to give the juice on which recruitment strategies work and which do not:
The more details, the better. The field benefits from the collective wisdom of researchers who have figured out best practices.
If you live in the Northern Virginia or DC area, you can earn up to $85 for participating in our study — click here for details. And if you are suffering from social anxiety symptoms, you can earn up to $75 for participating in our other study — click here for details.
Dr. Todd B. Kashdan is a public speaker, psychologist, professor of psychology and senior scientist at the Center for the Advancement of Well-Being at George Mason University. His latest book is The upside of your dark side: Why being your whole self — not just your “good” self — drives success and fulfillment. For more, visit toddkashdan.com
