Selection of participants

Libby et al. (2002) suggest that the choice of participants should match the goal(s) of an experiment, and that researchers should not use more sophisticated participants than is necessary to achieve those goals. Sophisticated participants are appropriate when the goal of an experiment is to investigate the effects of a regulation in a setting that requires a strong understanding of institutional features (i.e., studies that peer into the minds of experts). For example, experienced auditors are likely necessary for research that investigates how auditors' behavior changes in response to regulations that affect the style and content of the audit report. Alternatively, less sophisticated participants are appropriate when the goal of an experiment is to investigate fundamental psycho-logical biases or economic behaviors (Libby et al., 2002). Student subject pools may be a useful source of less sophisticated participants (see, e.g., Elliott, Hodge, Kennedy, & Pronk, 2007, for a discussion of when MBA students are likely to be appropriate). Researchers are also increasingly using Amazon's Mechanical Turk (AMT) platform to recruit participants (see, e.g., Farrell, Grenier, & Leiby, 2014; Krische, 2014; Rennekamp, 2012). Regardless of their source, the use of less sophisticated participants allows researchers to address many more unanswered questions, because the availability of less sophisticated participants is much greater. A number of studies have used less sophisticated MBA student participants as proxies for managers to investigate regulatory effects (Brink et al., 2014; Jackson, 2008; Seybert, 2010; Wang & Tan, 2013). However, very few studies have used less sophisticated participants to examine the effects of regulation on earnings management and accounting choice on auditors or directors (see Persellin, 2013, for an exception). While we believe the studies we review in this paper have generally made appropriate use of sophisticated auditor and director participants, we do not know how many interesting research questions have been abandoned because researchers assumed that they could not be answered using less sophisticated participants. Some of this may be due to entirely appropriate concerns that those involved in the review process will be dismissive of auditor and audit committee studies that do not use actual auditors and directors as participants. Thus we not only encourage researchers to use less sophisticated participants (again, where appropriate), but we also encourage journal editors and reviewers to be open-minded to their use, and to actively consider that less sophisticated participants might be appropriate for answering certain questions related to fundamental biases and behaviors. At the same time, if the goal is to peer into the minds of experts, more costly experienced participants are necessary. Here we encourage researchers to be entrepreneurial in their attempts to obtain appropriate participants. We also encourage editors to recognize that the costs of additional experimentation are much higher in such studies.

Questions:

The above readings explain the methodological considerations associated with selecting participants. How do you feel about the use of less sophisticated vs. more sophisticated participants? Do you think most participants find the experience of taking part in a study interesting?