Educating Managers from an Evidence-Based Perspective Author(s): Denise M. Rousseau and Sharon Mccarthy Source: Academy of Management Learning & Education, Vol. 6, No. 1 (Mar., 2007), pp. 84101 Published by: Academy of Management Stable URL: http://www.jstor.org/stable/40214519 Accessed: 05-05-2016 02:32 UTC REFERENCES Linked references are available on JSTOR for this article: http://www.jstor.org/stable/40214519?seq=1&cid=pdf-reference#references_tab_contents You may need to log in to JSTOR to access the linked references. Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at http://about.jstor.org/terms JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact support@jstor.org. Academy of Management is collaborating with JSTOR to digitize, preserve and extend access to Academy of Management Learning & Education This content downloaded from 130.102.42.98 on Thu, 05 May 2016 02:32:44 UTC All use subject to http://about.jstor.org/terms e Academy of Management Learning & Education, 2007, Vol. 6, No. 1, 84-101. Educating Managers From an Evidence-Based Perspective DENISE M. ROUSSEAU SHARON MCCARTHY Carnegie Mellon University Evidence-based management (EBM) means managerial decisions and organizational practices intormed by the best available scientific evidence. In this essay we describe the core features of educational processes promoting EBM. These include mastering behavioral principles where the science is clear and developing procedural knowledge based on practice, feedback, and reflection. We also identify key factors in organizational research, education, and management practice that inhibit EBM's use and ways these can be overcome. - "a wasteland of vocationalism that needed to be transformed into science-based professionalism " H. A. Simon (1991: 139) on business education in the 1950s. - "a variety of pressures in the organizational field of business education are rapidly steering us toward deprofessionalization. " C. Q. Trank and S. L. Rynes (2003: 189) on 21st century business education. - "a fundamental property of everyday life is that people believe ahead of the evidence." K. E. Weick (2006; 1724) Evidence-based management (EBM) means managerial decisions and organizational practices informed by the best available scientific evidence. Much like its counterparts in medicine (e.g., Sackett Straus, Richardson, Rosenberg, & Haynes, 2000) and education (e.g., Thomas & Pring, 2004), the judgments EBM entails also consider the circumstances and ethical concerns managerial deci- An H. J. Heinz II Professorship supported the preparation of this article. Bob Dammon, Judi DePalma, Keith Hunter, and Jonathan Goodman contributed to our understanding of the teaching processes used by different disciplines. Paul Goodman and Severin Hornung provided helpful comments on previous drafts. Joshua Manning created the graphics. We also thank our management students for sharing in the experiences upon which this article draws. sions involve. In contrast to medicine and educa- tion, however, EBM today is only hypothetical. Contemporary managers and management educa- tors make limited use of the vast behavioral sci- ence evidence base relevant to effective organizational practice (Walshe & Rundall, 2001; Rousseau, 2005, 2006a; Pfeffer & Sutton, 2006). Refocusing management education on evidence promises improved managerial decision making and better organizational outcomes. It can reduce the use of ineffective management practices while making effective approaches more widespread. Using evidence makes it possible for well-informed managers to develop substantive expertise throughout their careers as opposed to the faddish and unsystematic beliefs today's managers espouse (Abrahamson, 1991; Staw & Epstein, 2000) - a contributing factor in the early retirement of other- wise capable people whose expertise is deemed outdated. It can bring together scholars, educators, and management practitioners to the betterment of both scientific knowledge and individual and collective learning - but only if we radically revamp our approach to management education. Peter Drucker, seminal management thinker, was perhaps the first to assert that most business issues - from morale problems to strategic implementation - are generic "repetitions of familiar problems cloaked in the guise of uniqueness" (from Lowenstein, 2006; Drucker, 1966). Business problems often reflect the workings of replicable processes in the way workers and work groups act and how firms engage in market activities. If a problem is generic, managers can learn its under- Copyright of the Academy of Management, all rights reserved. Contents may not be copied, emailed, posted to a listserv, or otherwise transmitted without the copyright holder's express written permission. Users may print, download or email articles for individual use only. This content downloaded from 130.102.42.98 on Thu, 05 May 2016 02:32:44 UTC All use subject to http://about.jstor.org/terms 2007 Rousseau and lying dynamics, and related evidence-based principles, to guide their choice of effective action. In finance, for example, the understanding of principles such as the time-value of money forms the basis of managerial competencies in financial analysis. Such principles exist to inform the behavioral side of organizational decisions, too. Robust principles of cause-and-effect such as "challenging goals, when accepted, lead to higher performance" are the product of decades of social science research (e.g., Locke & Latham, 1984). Nonetheless, it is unlikely that firms make as pervasive use of goal setting as its well-established effectiveness warrants, let alone that they pay suf- ficient attention to employee accepfance. In contrast to the work of our colleagues in finance, pre- cious few behavioral principles make it into widespread management practice. Instead there is persistent use of ineffective practices such as downsizing to improve efficiency (cf. Cascio, Young, & Morris, 1997) and performance appraisal to improve performance without employee feedback or rater training (Woehr & Huffcutt, 1994). The gap between behavioral science evidence and what even professionally trained managers do is enormous. Of the many reasons for this gap, one stands out: We typically don't teach management students, from undergraduates to executives, to understand or use evidence (Rousseau, 2006a). Herein we explore the implications of teaching management from an evidence-based perspective. Combining experiences in management education with those in healthcare and other professions, we describe key features of teaching EBM and examine how current practices in research, teaching, and organizations hamper use of best available evidence in management education. Finally, we suggest specific collaborations among research- McCarthy 85 around high-caliber evidence have led to better service to patients and clients, organizations and communities. But to use evidence, practitioners from physicians to managers must first learn what it says and then how to apply it. To learn what teaching evidence-based practice entails we begin by identifying its features in the fields of medicine and education and contrast these with contemporary management education (Table 1). Principle 1: Focus on Principles Where the Science is Clear The first principle of evidence-based teaching is to educate people on principles founded upon a convergent body of research. Students in schools of medicine and education learn about the causal processes underlying human physiology, child development, and cognitive psychology (e.g., Sackett et al., 2000; Sherman, 2002). The ultimate goal of evidence-based teaching is the learners' acquisition, memory, and retrieval of principles of causeeffect relationships, enabling their professional practice to reliably yield desired results. Even where principles are couched in conditionals, they provide learners with a more mindful and systematic way of diagnosing and solving managerial problems (cf. Longer, 1991). Principles must be mastered, and deeply, reflectively understood, so practitioners can solve the array of problems they will face over the course of their careers. Principles based on evidence are the basis for procedures that translate these principles into action. Evidence-based training thus involves learning both principles (knowing what) and procedures (knowing how). For example, training to become a school teacher involves the principle that students ers, educators, and practitioners to foster evidence- learn more when they actively process information based management. (e.g., Ruhl, Hughes, & Schloss, 1987). This principle translates into the practices associated with student-centered learning, where students are active TEACHING FROM AN EVIDENCE-BASED PERSPECTIVE: LEARNING FROM OTHER FIELDS In the last 25 years, research evidence has become central to effective practice in fields as diverse as medicine (Lemieux-Charles & Champagne, 2004; Sackett et al., 2000), education (Thomas & Pring, 2004), policing (e.g., Petrosino, Boruch, Farrington, Sherman, &Weisburd, 2003) and psychology (DeAngelis, 2005). Cumulative knowledge in the physical and social sciences offers convergent evidence of pervasive cause-and-effect relationships, which in turn have been translated into everything from powerful drug therapies to effective early childhood educational interventions. Practices built agents in their own education (in contrast to teach- er-centered learning depending primarily on lectures). Student-teachers experiment with techniques to promote active learning - such as short lectures punctuated by student reflection on poten- tial applications, simulations and/or break-out group discussions. Over their careers, teachers might create innumerable applications of the ac- tive learning principle to foster student learning in challenging situations that teachers could not even anticipate during their formal training. In management education, we can see how evidence-based principles of cause-and-effect might be applied to teaching the well-established prin- This content downloaded from 130.102.42.98 on Thu, 05 May 2016 02:32:44 UTC All use subject to http://about.jstor.org/terms 86 TABLE 1 Features of Evidence-Based Teaching Across Different Disciplines Features Management Medicine Education Principles of cause/effect Research is fragmented, Syntheses available, teachers Syntheses available; identified through and highly specialized; educated in clinical teachers educated in research many teachers lack processes. teaching practices. behavioral science training. Make decisions explicit Numerous decisions are Trained to recognize decision implicit, made without demands. Many decisions managerial awareness. are discrete. Diagnosis Formal training varies. Explicit formal training basis Formal training varies. often limited to case of educational process, analysis. Repeated practice Education often limited Formal practice opportunities, Formal practice opportunities to classroom e.g., internships and opportunities, e.g., instruction. Some residencies. internships, programs, especially for executives offer opportunities for practice, feedback, reflection, and renewed practice. Contextualizing findings Requires experience to Problem-solving sessions led Student teaching provides to particular situations adjust applications to by experienced experience with circumstances. practitioners (e.g., rounds); different settings and Learners often have internships, residencies students, little experience. Field provide exposure to projects useful. context. Decision supports Few. Numerous. Patient care Practice guidelines in protocols, elaborate expert several teaching systems to gauge patient domains, prognosis, handbooks of drug interactions, on-line information, etc. ciples of goal setting (Locke & Latham, 1984). Among these principles are that 1. Specific goals are more effective motivators of performance than general goals. 2. Challenging goals are more effective motivators of high performance than less challenging goals. 3. Goal acceptance is critical to goal achievement when goals are not set by the employee. 4. Prevention or control-oriented goals (with a ceiling or natural limit, such as 100% safety or zero defects) create vigilance and negative emotion in employees, whereas promotion or growth-oriented goals (with no limits such as increasing staff competency) promote eagerness and positive emotion. Teaching present or future managers to apply evidence-based goal-setting principles would involve presentation of the above four principles, followed by exercises that develop each into actions, techniques, and practices the individual can reliably perform, such as translating work plans into appropriate goals in terms of specificity, content, and process. Teaching principles would be augmented by exploring conditions that promote or inhibit their use via both reflection and practice (cf. Longer, 1991; Morin & Latham, 2000). In educat- ing both teachers and managers, a few essential principles well taught via ample practice and reflection on their practical uses and limitations offer learners knowledge more readily transferable to their day-to-day behavior than do survey courses that touch superficially on many topics. Unfortunately, formal schooling in organizational behavior and related management fields tends to focus more on a general review of theories, includ- ing such long-discredited old saws as Herzberg and Maslow. Seldom does management education distill the high-caliber evidence that does exist into principles on which learners or managers can base their professional practice. The fragmentation of management research (Rousseau, 1997) contributes to the underuse of evidence by making access to relevant research difficult, certainly for current managers but also for those management educators whose heavy course loads make it vir- This content downloaded from 130.102.42.98 on Thu, 05 May 2016 02:32:44 UTC All use subject to http://about.jstor.org/terms 2007 tually impossible to keep up-to-date. Fundamental to any shift toward evidence-based management education, nonetheless, is a focus on core principles underlying the behavior in and of organizations that the best available research supports. Principle 2: Develop Decision Awareness in Professional Practice Decision awareness is the next principle of evidence-based teaching, where learners come to recognize when they are making a decision. Through development of critical thinking and training to reflect on how they would conduct themselves on the job, learners come to realize that any decision may be an opportunity to apply evidence, regard- comes experienced by individual hospital patients, while visiting each patient in the company of medical students and residents whose task it is to discuss treatment alternatives and implications. Although we do not mean to paint an overly rosy picture of evidence-based teaching in other fields, our essential point is this: What managers learn during formal education would be more effectively transferred to their workplace were they more adept at attending to and evaluating the decisions available to them. Along these lines, mental rehearsal by managers participating in training programs, where they visualize themselves applying their newly acquired knowledge, has been found to aid that new learning's transfer and use on the job (Morin & Latham, 2000). less of whether an action is taken. Awareness that an action (or nonaction) is itself a decision, that is, a controllable factor leading to an outcome, is central to professional practice based on the best evidence. This recognition might be straightforward for a surgeon in an operating room who is aware that using a particular replacement valve (metal versus tissue) in a cardiac bypass operation determines whether the patient will require anticoagulant medication over a lifetime (metal yes, tissue no). In the case of practicing managers, the many decisions they make in a given day are often less apparent. Managers may not realize that whether they give feedback to an employee at a given point in time can have substantial consequences for that person's subsequent performance (cf. Walshe & Rundall, 2001). Day-to-day actions managers take involve a host of discretionary behaviors, some habitual, others more deliberate. Decision aware- ness leads learners to reflect upon their interactions with others and their role in shaping the environment in which they work - a condition that requires sustained effort for managers given the unquestioned patterns of thought and habitual practices that characterize organizational life (Argyris, 2004). Compounding difficulties in developing decision awareness is the unstructured, nondiscrete, ambient nature of many organizational problems (e.g., the on-going conflict between design engineers and manufacturing personnel in a company where their disputes are taken for granted). It's not surprising that managers with Principle 3: Diagnose Underlying Factors Related to Decisions Diagnosis is the third feature of evidence-based teaching. Recognizing that a decision is to be made promotes analysis of the underlying conditions motivating that decision. Diagnosis begins with learning to ask appropriate questions, for example, what causes this person's low performance or that group's conflict? Formulating the problem as "what causes X" leads to further diagnostic questions regarding who, when, where, why, and how X occurs. Developing the capacity for critical thinking is perhaps the most crucial capability required in EBM. Research on medical diagnoses suggests that expert diagnosticians use general categories to represent particular cases, and these categories in turn aid the formation of hypotheses while assessing a patient (Wortman, 1972). Experienced managers make decisions in a similar fashion (Schaub & Strohschneider, 1989). In problem simulations, experienced executives have been found to spend considerably more time than do students assessing the situation. Experienced executives devote more time gaining an overall picture and don't act until that has been achieved. They set goals and work on understanding the situation. In contrast, inexperienced students spend much less time in these activities. They do less fact gathering and diagnosis, instead responding to isolated facts in a more ad hoc fashion only limited exposure to EBM would find it difficult to recognize opportunities for problem solving. (Schaub & Strohschneider, 1989). share their insights with each other while sup- tice, reflection, and renewed action are character- ported by a mentor (Argyris, 2004; Mintzberg, 2005). Such is intent of grand rounds, an essential part of istic of medical school training - and may be more medical education, where physician teachers re- plines than others. While the fields of finance and accounting provide opportunity for diagnosis and Decision awareness is enhanced when learners view the course of disease, treatment, and out- Deliberate efforts we have described above to improve student diagnostic capabilities via prac- characteristic of certain business-related disci- This content downloaded from 130.102.42.98 on Thu, 05 May 2016 02:32:44 UTC All use subject to http://about.jstor.org/terms 88 practice, management students get relatively few practice opportunities as part of their behavioral science training. Moreover, no management program provides the degree of practice, reflection, and feedback common in physician and teacher training. When educating current and future managers, we can build diagnosis and reflection into our use of management cases, but only if we change the traditional way in which cases are taught. The current use of management cases as a teaching tool tends to deemphasize instruction in evidencebased principles. Instead, how and why the protagonists in the case achieved certain outcomes is for the most part the stuff of anecdotes and Monday morning quarterbacking, rather than knowledge and experience informed by cumulative evidence (see the teaching notes that accompany any of the standard cases published by management schools). This approach risks creating confident amateurs who believe they can become experts by emulating a case's ostensibly successful manager. Nonetheless, there is no reason why cases cannot be used to teach the use of evidence in management decisions; better approaches exist to teaching cases when evidence-based education is the goal. Consider how pedagogy used in teaching dentists and physicians to make evidence-based clinical decisions (e.g., Werb & Matea, 2004; Sackett et al., 2000) might be adapted for training man- agers in case analysis and decision making: 1. Structure and pose a [managerial] question, 2. Search for best available evidence, 3. Critically appraise the information found, and 4. Apply relevant information to case issues, questions, and solution implementation. Learners might be provided with (or, later with training, asked to identify) critical managerial questions regarding the case and then required to access and apply evidence appropriately. Powerful learning could also follow if more cases were written to exemplify how managers themselves use evidence effectively. Certain existing cases do a very good job of showing how systematic fact gathering can be used to provide diagnostic information and feedback (e.g., Michael Beer's 2002 Merck Latin America A, B cases). Such cases con- nect behavioral science principles to analysis of case facts by calling attention to specific ways of structuring diagnosis and problem solving rather than centering upon one manager's personal experience. Principle 4: Contextualize Knowledge Related to Evidence Use The fourth feature of evidence-based teaching is that learners become able to adapt their knowledge to settings and their circumstances. Evidence has little value until it can be acted upon effectively, and this requires taking into account the circumstances of its use (Atkins, Siegel, & Shutsky, 2005). With its focus upon general principles, science tends to be viewed as context-free (a condi- tion in organizational research subject to challenge, e.g., Rousseau & Fried, 2000, although universality is not a requirement for scientific generalizations, see Mitchell, 2000). Nonetheless, evi- dence-based practice entails not only the query "can it work" but also "will it work here?" and "is it worth it in this case?" (Canadian Health Research Foundation, 2005; Dobrow, Goel, Upshur, 2004). Rather than blindly applying rules, experts condition their judgments upon facts and context, sup- plementing their scientific knowledge with local experience, priorities, and needs. Novices have limited ability to generate inferences and relations not explicitly stated in the problem they seek to solve (Chi, Glazer, & Rees, 1982). Inferring additional information from the literal cues a case or problem statement provides is often necessary in order to take the situation's complexity into account. A novice might readily recommend a socalled best practice as a solution to an organizational problem, let's say, suggesting that interunit conflict in a social service agency be resolved via team building. On the other hand, a person with more organizational savvy is likely to realize that the relative power of the people promoting teamwork, workplace politics, and varying degrees of competence among workplace members can spell the difference between ready adoption of interunit teams and an aborted implementation. As learners analyze and solve problems over time, the feedback they obtain, when based upon valid knowledge (e.g., from mentors or colleagues, or from their own informed analysis), deepens their understanding of how best to apply cause-and-effect principles. (In the case of the interunit conflict above, the expert might recommend a multistep process beginning with a relatively easy-to-implement increase in communication as a first step toward more constructive coordination and problem solving.) Such procedural knowledge is often tacit, derived over time through experience (often via rehearsal and practice). It can require practitioners to deepen their capacities to pay attention to their workplace actions, what Weick and Roberts (1993) have referred to as mindfulness, a state of cau- This content downloaded from 130.102.42.98 on Thu, 05 May 2016 02:32:44 UTC All use subject to http://about.jstor.org/terms 2007 Rousseau and tious, systematic information processing suitable for nonroutine and/or high-risk situations). Though approaches to developing it vary across practice domains and circumstances, procedural knowledge is fundamental to the modes of learning contemporary evidence-based fields use. Evidence-based teaching entails repeated practice opportunities to build know-how around a field's core principles. Practice helps learners ac- quire tacit knowledge otherwise difficult to convey explicitly (e.g., the tone of voice to employ when asking questions of a patient, talking to students' parents, or giving feedback to an employee). It also develops skills in situations where effectively applying procedures requires special competencies (e.g., knot tying for surgeons, laying out effective visual presentations for classroom use, or diagnosing a complicated multilevel managerial problem where individual, group, and organizational factors interact to impair performance). It takes time and experience to combine evidence-based knowledge with personal skills and contextual information. Practice, feedback, and self-reflection are the heart of the effective application of evidencebased principles (Dorner & Scholkopf, 1991; Ericsson, Krampe, & Tesch-Romer, 1993) - although con- temporary management education provides limited practice and less reflection than would be needed to develop a modicum of mastery in applying evidence-based principles. Merely receiving a grade on a test or case analysis is insufficient to constitute either practice or reflection. Principle 5: Develop Evidence-Based Decision Supports Decision supports promote evidence-based practice by making it easier for practitioners to identify critical factors (e.g., checklists, patient-care proto- cols), diagnose the appropriateness of a particular solution (e.g., diagnostic questionnaires to assess relevant worker or customer perceptions, clinical tests to indicate whether antibiotics should be given), and follow effective procedures (e.g., flowcharts). The settings in which practitioners work may provide an array of established patient-care regimens, routines, or checklists, as in the case of healthcare. Alternatively, evidence-based practi- tioners who work on their own or in less structured settings can learn to develop their own decision supports. An experienced 40-something manager we know adapted Porter and Lawler's (1968) moti- vation model, which he had indeed come across in an undergraduate course in organizational behavior, to guide his interactions with subordinates. He uses this framework as a mnemonic in training his McCarthy 89 managers how to motivate and support their sub- ordinates. Providing learners with examples of how decision supports can be developed (e.g., designing their own frameworks or practice routines, etc.) can make it easier to transfer evidence-based practices into daily decisions. Note, however, that decision supports are merely tools, unlikely in themselves to bring about effective management practice without an understanding of the underlying principles whose use they are designed to support. Moreover, no decision support is written in stone. Rather, they require updating over time and should act to guide but not limit inquiry or practice considerations. Principle 6: Prepare To Access New Evidence Developing effective evidence-based practice does not stop on graduation day. The next step is preparing people to update their knowledge, by accessing new practice-related evidence over time. The expectations students leave with upon graduation are critical to their further development as evidence-based managers. Successful learners should expect to update their knowledge over the course of their careers. In particular, they need to know that the evidence base is growing and how to tap into it once they leave. In medicine, busy clinicians find time for selective patient-driven read- ing by searching for best available evidence relevant to their current practice. One hospital we know has several book clubs meeting monthly where nurses and physicians in specialty areas such as psychiatry or pediatrics discuss the latest publications in their area. These examples suggest a critical issue in management education how we prepare learners to access new evidence after they leave our programs and classrooms. Conclusion The broad and deep evidence base available to aid in managerial decisions offers a reliable basis for managers to act upon everyday. Evidencebased teaching in established fields highlights the design features critical in teaching EBM. Nonetheless, the ways in which behavioral science research can best contribute to management education and practice would need to be informed by the distinctive features of our research and our teach- ing settings, and the specific needs of end users. Doing so is essential to overcoming the barriers we face in teaching EBM. This content downloaded from 130.102.42.98 on Thu, 05 May 2016 02:32:44 UTC All use subject to http://about.jstor.org/terms 90 preciate that even physical laws are contingent on WHY WHAT WE TEACH IS NOT certain conditions (see Mitchell, 2000). Some social EVIDENCE-BASED Evidence-based teaching is not the dominant mode in management education. Reasons for this are found in our research norms and practices, teaching environments, and values of end users those managers, staff, consultants, and related practitioners who impact organizational decisions, actions, and practices (Table 2). Science-Based Challenges Managers make daily decisions for which relevant high-caliber evidence exists (often in meta-analyses and integrated research reviews) regarding effective practice. From performance appraisal to strategic decision making, research demonstrates robust and generalizable cause-and-effect relationships relevant to effective organizational practices. The challenge the scientific paradigm poses for teaching evidence-based management is tied to difficulties in (1) evaluating evidence quality, (2) accessing this information, and (3) tracing clear cause- effect connections to critical outcomes. Educated people, including many faculty members, may not have clear ideas of the rules of evidence in social science (Westen & Bradley, 2005). Scholars in the philosophy of science attribute this to overuse of the physics model and failure to ap- scientists are reluctant to take a stand regarding the implications of evidence because they know that the tradition is to advance science by discon- tinuing existing theories, with less emphasis given to convergence. Nonetheless, there are innumerable durable scientific generalizations, from the early laws of effect and practice (e.g., Thorndike, 1913; Ericsson, 2006) to the job performance- job satisfaction relationship (e.g., Petty, McGee, & Cavender, 1984; Judge, Thoresen, Bono, & Patton, 2001), which have pervasive implications for such recurring organizational problems as improving performance and motivating high performers. Still, up-to-date research evidence can be difficult to obtain - especially for managers and busy educators. Largely organized by specialty topics, specialists' knowledge - or at least their vocabulary - can be required to track down evidence. It often resides in diverse locations, in bits and pieces, without systematic integration. Norms of science frequently reward novelty rather than cumulativeness. The general absence of research summaries where lay people can read and understand the evidence's practical implications hampers the development of EBM (Note: this omission in management research contrasts with the on-line TABLE 2 The Challenges in Promoting Evidence-Based Management Features Management Medicine Education Decision constraints Authority and politics can limit Physician exercises high control Educator exercises moderate control managerial discretion to over practice. EB medicine is over own practice. EB practice is practice EBM. normative. increasingly normative. Updating/On-going Often limited due to source Numerous sources including on- Numerous sources including on-line access to research specialization and lack of line collaboratives, journals, etc. collaboratives, journals, etc. availability to practicing managers. Evidence-based Often absent, requires Reinforced in multiple ways Varies as a function of culture organizational leadership through practitioner rounds, in- organizational leadership, peer focus on evidence and fact service education, research support, etc. gathering, communities of involvement. practice for knowledge sharing. Participation in Limited today. Long established and on-going Participation in education, research Formerly common, with part of professional practice in evaluation, and learning research widespread corporate medicine. becoming more widespread. research participation in 1960-1970s. Quality movement in 1980s promoted evidence-based decision making. Research participation declined in 1990s with rising competitive and time pressures. This content downloaded from 130.102.42.98 on Thu, 05 May 2016 02:32:44 UTC All use subject to http://about.jstor.org/terms 2007 summaries available in medicine and education developed by various collaboratives.) When surveying educators themselves regarding how they identify relevant evidence, the most frequently identified source we find is the textbook. But even for textbook writers, it is difficult to adequately synthesize the literature where experts in a partic- ular area have not assembled systematic reviews. Little wonder many management textbooks rely heavily upon popular trade books as sources. The existing mass of management research needs to be distilled into readily accessible, updatable repositories. Pfeffer and Sutton (2006) have argued that there is too much evidence, making it difficult to pick one's way intelligently through the research base to find knowledge that might best inform management practice. In that regard, Inter- net users in particular have a tough time winnowing quality information from the chaff (e.g., Achenbach, 1999) as searches on matters related to managerial decisions may well turn up at least as many opinion pieces by consultants or would-be gurus as actual scientific studies or their summaries. Our experience indicates that management students often give more attention to colorfully written, if unsubstantiated, opinion, than to the less vivid, and far-more abstruse evidence that researchers assemble. Nonetheless, we believe the problem is less about too much evidence and more about the lack of synthesis. Management scholars are rewarded often for new ideas, as opposed to assembling cumulative evidence regarding ideas other scholars might already have entertained. Even the Academy of Management Review pursues the editorial policy of rejecting papers unless they develop new theory. Providing systematic evidence what we can know with confidence is a recipe for rejection. Failure to target summaries to nonspecialists and in particular to practicing managers further compounds the problem. Meta-analyses and systematic literature reviews are important not only to advance science but also to support evidence-based textbooks and management practice. More scholarly effort - and journal space (paper or electronic) - should be dedicated to such compilations - if we care about closing the research-practice gap. Norms in organizational research promote novelty and fragmentation, targeting work to particular specialties rather than accumulation and broader use. In their seminal article introducing evidence-based management, Walshe and Rundall (2001) counter prevailing scientific practices by arguing for management- oriented abstracts, which would make research findings accessible to the nonspecialist. To its credit, the Academy of Management Review has for years modeled an informative approach to ev- idence-based management by including practice implications in its articles. Similarly, the Academy of Management Executive, now Perspectives, pub- lishes research briefs for laypeople and targets summaries of management evidence for use in the classroom. Perhaps most significant in promoting evidence-based practice is the acceleration of access to summaries of strong evidence in medicine, education, and policing made available on-line by cooperatives of researchers and practitioners. Websites give the general public and practitioners access to the evidence on effective practices in medicine (the Cochrane Collaboration) and other areas such as education and policing (the Camp- bell Collaboration). It is not uncommon for both physicians and their patients to log on to the Cochrane Collaboration seeking information regarding a course of treatment. The management field currently lacks such a repository. It doesn't take much imagination, however, to envision an Academy of Management Collaboration or some variation of it in our future - a source of evidence-based information to guide managers, staff, consultants, and investors to more effective management practice. Creating communities committed to translating evidence into its implications for practice is critical to building consensus on what constitutes best evidence and its effective application. Thoughful use of design processes is required to create a truly effective community out of management education's disparate constituencies (cf. Boland & Collopy, 2004; Romme, 2003; Rousseau, 2006b). Management research to date imposes another constraint on EBM by its limited attention to how organizational practice and managerial decisions might be made more effective. Argyris (2004) uses behavioral decision-making research as a case in point. Research on nonrationality in human decision making identifies numerous biases but seldom pays attention to how biases in judgment and perception can be overcome. As the movement toward evidence-based practice extends into management, we look toward consciousness raising on the part of scholars regarding the value of more comprehensively connecting the dots, exploring the links between research topics and constructive action organizational actors might take. As in the healthcare arena, closer connections between management research and practice may entail both "evidence-based practice and practice-based evidence" - where practitioners create useful new data and researchers inform practice through more directly applicable studies (Peile, 2004). This content downloaded from 130.102.42.98 on Thu, 05 May 2016 02:32:44 UTC All use subject to http://about.jstor.org/terms 92 Teaching-Based Problems Substantial barriers to EBM exist in management teaching too, including (1) student attitudes and tastes, (2) faculty preferences and expertise, and (3) our poorly informed models of managerial learning. Management students have their own expectations and preferences. They often are turned off by certain research areas. The advantages bureaucracy, formalization, and structure provide firms (e.g., in terms of coordination and consistent policies; Pearce, 2001) aren't a big hit with students unless framed to play up the importance and power of managers. In our experience, one way to lose a class really quickly is to teach "substitutes for leadership," that is, how organizations can be structured via rewards and socialization to do without strong leaders (Kerr & Jermier, 1978). A class of would-be leaders has little interest in find- ing out how systems and practices might render them less central to their organization. Students have strong normative beliefs about what they as future managers need to know. The popularity among students of the case method and Harvard Business Review articles touting an expert's viewpoint suggest that face validity and readability trump substantive validity and scientific evidence in winning student endorsement. Students would rather hear about current company practices and the experiences of corporate leaders. The corporate conduct modeled in television and film (shades of Donald Trump's The Apprentice) exploits and reinforces this taste for leader-centric material. In sum, student expectations drive course content and our teaching styles in directions that fail to serve their own longer term interests. Individual faculty members thus aren't always upto-date. Some may be unaware of the cause- effect principles current management research supports. Researchers are often specialists whose expertise may not cover the gamut of topics a core management class commonly covers. Complicating matters further, adjunct faculty without current, or even any, advanced education in management research sometimes teach behavioral management courses by relying instead on their practical experience. Educators' working models of managerial learning pose fundamental problems to the extent they are ill-informed regarding how learners might actually become experts in management. Expertise is the possession of a large body of knowledge and procedural skill (Chi, Glaser, & Rees, 1982). Students in professional schools tend to begin as novices with limited knowledge of the evidence base they must master. As they move along the pathway from novice toward expert, the knowledge they possess becomes increasingly organized or chunked in ways that facilitate its accessibility and appropriate use (Chi, Glaser, & Rees, 1982; Ericsson & Lehman, 1996). EBM as described here provides a pathway for learners to progress toward greater expertise by developing more systematic and valid mental models of professional practice. Management education is hampered by a dearth of good examples regarding how real managers use research evidence or systematically gather data. Pfeffer and Sutton (2006) illustrate the forms EBM might take, suggesting that such exemplars do exist among today's managers and can be incorporated into teaching EBM. However, few management schools have built their curricula around an evidence-based model of how learners become competent managers. The essential point is this: A class of would-be leaders has little EBM would benefit from the grounding of manage- interest in finding out how systems and practices might render them less central to their organization. ment education in high-quality research regarding how managers can best develop expertise, including the roles played by foundational knowledge, experience, reflection and practice. Studies of expertise demonstrate that as a general rule, the deficiencies people show in solving problems are not due to poor problem-solving Faculty members have their preferences too. Professors tend to teach the way they themselves were taught, further compounding the problem of expos- ing students to current evidence. Faculty who learned Herzberg's 2-factor theory in graduate school may continue to teach the satisfierdissatisfier model in their introductory organizational behavior courses, even though it was discredited over 40 years ago (House & Wigdor, 1967). Although elementary and secondary school teachers are required to take continuing education courses to keep current, college professors are not. strategies per se but to failure to master the requi- site knowledge needed for solving problems (Chi, Glaser, & Reese, 1982). In the case of management, both faculty and student educational backgrounds tend to be quite diverse: some have prior behavioral science training, some managerial expertise, some a bit of one or neither. In consequence, students (and some faculty) may possess only a small portion of the knowledge needed to effectively ap- ply research on organizational behavior, strategic This content downloaded from 130.102.42.98 on Thu, 05 May 2016 02:32:44 UTC All use subject to http://about.jstor.org/terms 2007 Rousseau and decision making, entrepreneurship, or whatever domain knowledge a management curriculum covers. Basing management education on evidencebased principles might cause us to rethink the basic prerequisites, including undergraduate course work, required for mastering the core of management education. Lack of a common educational background characterizes management generally. That an MBA is neither a guarantee of managerial competence, nor even a requirement for it is no surprise. Indeed as many have argued (Pfeffer & Fong, 2002; Mintz- berg, 2004; Ghoshal, 2005) management education does not prepare future managers well - at least not in its present state. The current content of man- agement education is highly idiosyncratic, reflecting instructor preference and textbook coverage. As we have noted this content is not closely tied to high-quality research, and if it were, current teaching methods and student qualifications are still insufficient to promote its mastery. Lack of the req- uisite behavioral science knowledge poses a real challenge for proponents of evidence-based management education. Managers and their would-be teachers need more systematic behavioral science knowledge to understand and effectively apply our evidence base. In the same way that chemistry and biology are prerequisites for medical school, basic behavioral science knowledge and critical thinking skills may be essential to EBM. The teaching constraints on EBM involve not only what we teach but also how. Management education often provides little opportunity to prac- tice. By practice we mean that one applies what's been learned, reflects upon it and the results produced, revises one's understanding, and practices again to get better at it. Such an approach to learn- ing involves a curriculum, not merely a single course, or the efforts of one professor. It is a way of (professional) life for any person seeking to develop expertise, including a professional manager. Having a knowledgeable educator guide the learner's practice and reflection accelerates the development of expertise, as the institutionalized practices of student teachers and medical resi- dents suggest. Given the limited practice management education affords in applying effective behavioral principles, transforming learners into evidence-based managers may well hinge upon the quality of their postgraduate developmental McCarthy 93 reflection and learning (Beenen, 2006). Although nurses and physicians don't treat patients independently before completing their education, many business schools require students to have work experience before they arrive. This work history provides a context for classroom learning but not necessarily exposure to evidence-based prac- tice. In contrast, for student teachers and medical residents, evidence-based practice is accelerated by developmental experiences supervised by an expert after graduation. In the case of future evidence-based managers, there is tremendous need for follow-up beyond formal schooling to continue learning the evidence the effective application of which can make them expert managers. The ultimate postgraduate education may lie in creating evidence-based organizational cultures. In this regard, we resonate with Henry Mintzberg's perspective: "Shared reflection among managers . . . need not be only an exercise in a classroom." Managers, Mintzberg observes, can meet together to discuss what worked and what did not work - as an on- going part of their job. Such an experience spills over to promote other learning in the organization where these managers in turn meet with their staff to also promote their learning (Mintzberg, 2005: 245). EBM: Resistance From Users Ultimately, the success of EBM lies in the hands of managers, in particular those of organizational leadership, who must embrace the concept to make it work. Motivating individual managers to adopt an EBM perspective involves overcoming a variety of hurdles, some more challenging than others. What resistance can we anticipate from users? As Hammersley (2004) points out: "who would argue that practice should not be based on evidence?" (p. 134). The alternatives seem like nonsense, or as Fowler (1995) muses, like "the tossing of a coin." No one has ever done controlled ran- domized trials to test whether parachutes affect survival after jumping out of a plane (Smith & Pell, 2003). But it is hard to argue against using evidence. Still, it's easy to keep to the status quo in business, where "evidence" is optional in making even the most critical decisions. Although much evidence exists, managers have been slow to incorporate it into their daily practice. In fact, current their development of professional practice skills in management writings show signs of moving in the opposite direction, with a push toward "intuitive" decision making (cf . Gladwell, 2005). tend to focus either on project work or on screening for full-time employment, with limited attention to there is a cultural perspective viewing management as self-taught and experience-based via experiences. Students of management seldom have internships where mentors work to guide a systematic fashion. Instead, their internships Resistance to EBM comes in several forms. First, This content downloaded from 130.102.42.98 on Thu, 05 May 2016 02:32:44 UTC All use subject to http://about.jstor.org/terms 94 hands-on decision making. Firms have only recently begun to accept the notion that business education and in particular an MBA have value. Business history and lore are filled with companies and careers built by college dropouts (Bill Gates, Steve Jobs, and Charles Schwab to name a few). The idea that we can inform business deci- sions with academic research simply doesn't fit this image. Graduates intending to practice EBM will have to make their way in this environment and find ways to influence the appreciation their organizations and colleagues have for EBM. Medicine can enlighten us on some ways to overcome these cultural constraints. These include the nonetheless, what is the justification for failing to use the clear evidence we do have?) Managers can find it difficult to evaluate in hindsight whether their use of evidence led to a better decision. Feedback loops in many managerial decisions are not neat and tidy. An organization's external environment often complicates managerial outcomes, making it tough to figure out why one decision resulted in a positive outcome and another didn't. The timing and skill with which a decision is implemented also can influence its outcome and obscure the value of evi- dence (Hammersley, 2004: 136). EBM is likely to be adopted more quickly by multiple reinforcing factors of a younger genera- some organizations than by others. Already, health sicians share research findings and applications, and on-line collaborations disseminating research relevant to medical practice (Sackett et al.f 2000; (Walshe & Randall, 2001; Rousseau, 2005). Public tion (since the 1980s in medicine) educated in evidence, communities of practice through which phy- Walshe & Rundall, 2001; Cochrane Collaborative, 2005). Second, managers may resist EBM because it creates accountability. If norms in managerial practice actually supported basing decisions on validated principles, then managers would be accountable for ad hoc or "intuitive" decisions that didn't work out - much in the same way that physicians who treat patients using non-evidencebased therapies are vulnerable to lawsuits. Accordingly, developing EBM as a practice standard could in some future world require managers to justify their actions based upon relevant research. At times, the evidence can go against the manager's immediate self-interest, making it more convenient to ignore the course of action the best available evidence supports. A case in point is the dysfunctional consequences associated with disproportionately high CEO pay relative to that of the rank-and-file workers (Cowherd & Levine, 1992). Holding oneself accountable to managing in a way that is based on the best scientific evidence can constrain the entitlements and perks many executives currently enjoy. Third, EBM requires the capacity and willingness to search for and evaluate evidence. Much has been made of speed as a business resource, and using research can take time. For busy managers, finding the balance between speed and decision quality necessitates tough choices, and quality won't always win, even in situations where satisficing is costly. In fact, since well before the age of the "one minute manager" (Blanchard & Johnson, 1983) quick action has had a popular following. (The fact that there may indeed be no relevant or consistent evidence remains a challenge; care managers are being challenged to use the same evidential approach that clinicians employ institutions too may become early adopters because administrators spending public dollars face the need more often than other managers to legitimate their decisions. In particular, educational institutions from early childhood to secondary schools experience pressure to base their practices on evidence. In the United States, the "No Child Left Behind" legislation impacting public schools literature mentions "scientifically based research" 110 times (Slavin, 2002, in Pring, 2004). Experiences of evidence-based pioneers in other fields offer insight into ways of implementing evidence-based management effectively. TEACHING EVIDENCE-BASED MANAGEMENT Educating managers to promote evidence-based decisions and practices necessitates a closer alliance among management educators, researchers, and practitioners (Figure 1). By its focus on teaching where the science is clear and adopting a developmental model of managerial expertise, the content of management education shifts radically from the highly idiosyncratic content faculty cover today toward better integration of science and teaching. Focus on Evidence-Based Topics We recommend judicious selection of course topics to focus learner attention on well-supported findings. Less is more when fewer topics are chosen carefully to reflect the best evidence. It is better to ignore the tentative or merely novel in order to give more systematic attention to learners' mastery of higher quality knowledge. The best evidence is made available via meta-analyses (quantitative summaries of effect sizes across large numbers of This content downloaded from 130.102.42.98 on Thu, 05 May 2016 02:32:44 UTC All use subject to http://about.jstor.org/terms 2007 FIGURE 1 Prototype of Evidence-based Management Community research studies), systematic reviews of research (including qualitative as well as quantitative studies, books as well as articles), and controlled inter- vention studies. Evidence-based principles may be most effectively taught in relation to the general problems they can be used to solve (e.g., coordination, motivating performance, retention). Linking research evidence to relevant problem domains allows learners to store and retrieve the concepts they acquire using problem categories that make practice and on-going use easier. It also gives more attention to the recurrent challenges our stu- dents will face in organizations, promoting a more imaginative and proactive stance to predicaments many people believe to be out of their control. It is difficult for people to recognize that their actions may contribute to a problem or might even solve it- until they believe in their personal ability to take constructive action (Weick, 2006). The same body of research can be relevant to multiple organizational problems. Thus, teaching with a focus on evidence can develop both faculty and student appreciation for the broad relevance basic re- search has to recurrent organizational decisions. Research on conflict and negotiation can inform coordination, problem solving, team building, etc. (e.g., Thompson, 2007). Likewise, there is an array of organization-related processes to which evi- dence on sensemaking applies (from career development to inter-organizational relations, e.g., Sut- This content downloaded from 130.102.42.98 on Thu, 05 May 2016 02:32:44 UTC All use subject to http://about.jstor.org/terms 96 Academy of Management Learning & Education March cliffe, Brown & Putnam, 2006). Though the reach of evidence can be broad, inevitably, if we take pains to link it to teaching, we will also spot gaps where critical research remains to be done. Evidence-based principles provide learners with a powerful framework to draw upon throughout their professional lifetimes. Teaching learners the underlying causes of behavior in (or of) organizations, and ways to put this knowledge to use, involves developing their declarative knowledge (what the principles are and what they mean) as well as critical procedural knowledge (how to implement or take action based upon each principle). Developing declarative and procedural knowledge entails a cycle of information dissemination, practice, reflection on outcomes associated with differ- ent situations and settings, and trying again. Developing declarative and procedural knowledge entails a cycle of information dissemination, practice, reflection on outcomes associated with different situations and settings, and trying again. Careful attention to essential principles permits learners to understand cause- effect processes that generalize across situations and over time (e.g., Thompson, Gentner, & Lowenstein, 2003). Opportunities for practice via case studies, exercises, and student-run experiments are essential to the development of procedural knowledge. In the case of goal setting, for instance, this approach permits learners to develop a sophisticated understanding of the variety and nature of goals (specific/general; promotion/prevention). Learners can appreciate when certain goals are better applied than others. Through case analysis and exercises, learners can come to understand that promotion goals, while preferable to encourage innovation and creativity on the part of employees in a product design team, may be inadequate to ensure high reliability on an aircraft carrier (Weick & Roberts, 1993; Meyer, Becker, & Vandenberghe, 2004). Helping learners discern how and when to combine promotion and prevention goals contextualizes their knowledge in order to make it useful over situations and time. Practitioners will also face the challenge of having to find evidence relevant to decisions they were not formally trained to make. Having on-going ties to researchers and other evidence-based practitioners provides the learner the pointer knowledge needed to find relevant evidence and put it to ef- fective use. A question that remains is what to do with an important managerial concern where the evidence is nof clear. Helping learners understand how best to solve problems in areas where the science is not clear is a suitable topic for advanced learners (thoughtful practitioners, experienced managers) who have mastered the core evidence base. Such circumstances require learning how to experiment to generate local knowledge without falling prey to the biases to which typical managerial experience is subject (Gilovich, 1991; Dawes, 2001). We can learn from how other evidence-based domains have responded to practice needs when science falls short. In these circumstances, evidence-based practitioners need to generate and interpret their own locally developed evidence (differentiating the little "e" of local evidence from the "big E" of scientific evidence, Rousseau, 2006a). Such is the case in nursing with regard to the novice-to-expert pathway. Developing nursing expertise involves training nurses to augment their science-based practice by gathering local evidence where scientific evidence is lacking. Drug side-effects are a case in point, since all possible unintended consequences of newly developed drugs are unknowable. In such cases, expert nurses gather their own local evidence (small "e") via observation. Haag-Heitman (1999) offers a case of a nurse who noticed confusion in an elderly patient. Because it was similar to that of another patient on the same anxiety drug, the nurse withheld all medication but the anxiety drug. When similar patient confusion occurred, the nurse called the physician to indicate that the anxiety medication was causing a problem. Once he agreed to stop the drug, the patient slept soundly, and ultimately the nurse and physician made a record of the anxiety drug's adverse effect for the pharmacy (Haag-Heitman, 1999: 223). Such local experiments are part and parcel of learning systems such as the Toyota Production System, which offers systematic ways for workers and managers to acquire high-quality knowledge from their own experience where systematic evidence is otherwise lacking (Spear & Bo wen, 1999). Creating More Effective Ways for ManagerLearners To Practice Given the tremendous variation in organizational settings and the limited time allotted to practice opportunities, management education will benefit from new, creative forms of practice. Simulation technologies, for example, have been created to help members of organizations appreciate the complexity of how their own organizations operate (Carroll, Sterman, & Marcus, 1998). Our university This content downloaded from 130.102.42.98 on Thu, 05 May 2016 02:32:44 UTC All use subject to http://about.jstor.org/terms 2007 uses a simulation known as the Management Game to develop managerial decision-making capabilities. Such simulations may provide a more realistic training experience and practice opportunities over time, as parameters are updated to keep the learner challenged. Simulations are increasingly used in training physicians (e.g., Groopman, 2005). These can take the form of simulated human bodies on which to practice procedures. Such experiences give valuable learning opportunities both to seasoned doctors, filling the gaps in their experience (often making them aware of unfounded or dysfunctional practices they may have unconsciously acquired on-the-job), and to novices, who can practice alternative ways of doing things without risk to a living patient. Practice opportunities are directly tied to the timing of management education. Mintzberg (2004) ar- gues persuasively for waiting to educate managers until they are in the midst of managing, when they can integrate their education with their job demands. In this context certain questions arise: "Why should students be educated solely in the Manage Learner Expectations Management students as described above enter into professional programs with some strong expectations regarding the curriculum and course work. Management schools attempt to meet these expectations by cases analysis and lots of discussion involving current corporate practices. No wonder students can often come to believe that experts are only in the business world. Management re- search takes a back seat - or is left to be the ar- cane pursuit of scholars only to be read by one another. Recently, in reaction to a course on managing performance based on evidence, one student remarked, "when I heard the class was about evi- dence, I was afraid it was a PhD course." Indeed, management students and scholars typically interpret the word "theory" quite differently. To a scholar, theory reflects a general statement of causal conditions underlying a phenomenon and is subject to test; a useful theory is one that evidence supports. To a student, theory means an abstraction not directly tied to the real world. Managing expectations early and often is criti- vacuum of a business school?" "Can the value of cal to the cultural shift that evidence-based man- EBM techniques be appreciated by individuals with no managerial experiences?" Business schools currently work with a wide mix of experienced and inexperienced learners. More experienced learners might resist this new EBM perspective (reflecting their prior organizational and professional socialization), and thus, the framing of EBM becomes critical. Educational programs agement represents. We have brought local executives who practice evidence-based management into our classes. We have assigned learners the task of identifying what the evidence has to say relative to common organizational practices (e.g., whether to use ratings or rankings in determining which employees to reward). Understanding what constitutes quality evidence is particularly important as a means of inoculating students from faddish thinking. In our own teaching we educate management students in the forms that quality evidence can take including meta-analyses, sys- must continue to build opportunities for students to gain carefully mentored