Business memorelating to Audit. All questions and materials needed are attached. Please, ensure you are familiar with auditing before accepting this project. Thank you.

THE ACCOUNTING REVIEW Vol. 90, No. 3 2015 pp. 881-915 American Accounting Association DOI: 10.2308/accr-50911 Are Fraud Specialists Relatively More Effective than Auditors at Modifying Audit Programs in the Presence of Fraud Risk? J. Efrim Boritz University of Waterloo Natalia Kochetova-Kozloski Saint Mary's University Linda Robinson University of Waterloo ABSTRACT: Previous studies indicate that auditors are able to identify fraud risk factors, but may not be able to translate this knowledge into an audit plan that effectively takes these factors into account to increase the likelihood of detecting fraud. Fraud specialists may be able to compensate for such limitations. This study investigates the relative merits of involving fraud specialists in assisting auditors by developing an audit plan that would effectively address fraud risk in a revenue cycle. Results show that fraud specialists did not differ from auditors in the number of procedures selected from a standard audit program; nor were these procedures cumulatively more effective than those selected by auditors. Fraud specialists generated a greater number of nonstandard additional audit procedures, and those procedures were marginally more effective, but less efficient, than those of auditors, except for certain groups of procedures. Finally, although the fraud specialists proposed significantly more additional (non-standard) procedures than auditors, their proposed budget increase for this category of procedures was significantly smaller than the budget increase proposed by auditors. Adjustments to the overall time budget did not differ between fraud specialists and auditors. We thank the anonymous reviewers, Chris Agoglia, Joe Brazel, Paul Caster, Mary Curtis, Lynford Graham, Karim Jamal, Khim Kelly, Thomas Kozloski, Bill Messier, Ed O'Donnell, Scott Vandervelde, as well as participants of the 2008 Investigative Forensic Accountants' Conference, 2008 International Symposium on Audit Research (ISAR), 2011 European Audit Network (EARNet) Symposium, 2008 CAAA Annual Conference, 2008 University of Toronto Mississauga Forensic Conference, 2009 AAA Auditing Section Midyear Meeting, and research workshop at Concordia University for their comments. Special thanks to Nicholas Hodson, Valerie Steele, and to the professional accounting firms that provided participants and audit experts for their assistance in conducting this study. We acknowledge the capable research assistance provided by Tim Bauer and Adam Presslee. We gratefully acknowledge the funding provided by the Canadian Academic Accounting Association/Deloitte Research Grant Program. Editor's Note: Accepted by Kathryn Kadous. Submitted: November 2010 Accepted: August 2014 Published Online: September 2014 881 Boritz, Kochetova-Kozloski, and Robinson 882 Keywords: fraud; specialist; audit planning; audit procedures. Data Availability: Data are available from the authors upon request. I. INTRODUCTION S ince the passage of Statement on Auditing Standards (SAS) 99 (Public Company Accounting Oversight Board [PCAOB] 2008) and the Sarbanes-Oxley Act of 2002 (U.S. House of Representatives 2002), policy-makers and regulators have promulgated additional guidance aimed at improving auditors' performance in assessing and responding to fraud risks of audit clients (PCAOB 2008). Auditors are expected to address fraud risk through the design of their audit methods and programs and by involving specialists (PCAOB 2008; International Auditing and Assurance Standards Board [IAASB] 2010a, 2010b).We study whether fraud specialists are relatively more effective than auditors in designing an audit program that will address elevated fraud risk. Our goal is to examine whether the expertise of fraud specialists can directly contribute to planning the nature and extent of audit procedures, and whether the mix of procedures that such specialists recommend is likely to be more effective and efficient than the procedures proposed by auditors in a setting where ex ante fraud risk is rated at above a low level. Auditing standards (PCAOB 2008, SAS No. 99; IAASB 2010a, ISA 240; IAASB 2010b, ISA 330) have recommended and prior research (e.g., Asare and Wright 2004) has shown that when there is a perceived heightened risk of fraud, auditors are likely to seek the assistance of risk management partners or fraud specialists, such as forensic accountants, in developing an audit plan. However, since consultations with specialists are costly, auditors may avoid referring issues to them unless they believe that a substantially elevated risk of fraud is present in an engagement, especially when a requirement for such consultation is not a strict one (Gold, Knechel, and Wallage 2012). In such a setting, auditors may convince themselves that the fraud risk in an engagement is lower than it really is. Alternatively, auditors may erroneously assume that consulting with fraud specialists would safeguard them against audit program design weaknesses when, in fact, some fraud specialists may have limited competence in audit program design (American Institute of Certified Public Accountants [AICPA] 2006; DeZoort and Stanley 2006; Meservy, Romney, and Zimbelman 2006). By directly examining fraud specialists' recommendations for an audit plan under conditions of an elevated fraud risk, we seek to clarify whether there are benefits in requesting fraud specialists to participate in audit program design.1 We test four hypotheses using an experimental approach. First, we examine whether, under conditions of elevated2 fraud risk, fraud specialists select more procedures from a standard audit program for a revenue cycle than auditors, and whether the procedures selected by fraud specialists from a standard program are more effective. Second, we examine whether fraud specialists develop more additional (non-standard) procedures than regular financial statement auditors. Third, we study differences in the effectiveness and efficiency of additional procedures proposed by fraud specialists and financial statement auditors by comparing those procedures against a benchmark based upon the judgments of an expert panel of auditors. Fourth, we examine whether fraud specialists modify the audit time budget for a revenue cycle differently from financial statement auditors. 1 2 In this study, the judgments of fraud specialists are compared to the judgments of audit partners and managers, i.e., our fraud specialists have more extensive audit experience than audit seniors. Audit seniors have been shown to have limited fraud risk assessment and response capabilities (see Hammersley, Johnstone, and Kadous 2011). We use the terms ''elevated'' or ''heightened'' to denote the fact that the risk was higher than low (5.1 on a scale from 1 to 10), but not definitive in the sense that fraud was presumed to exist. In other words, the participants' task involved planning an effective and efficient audit program, not a fraud investigation. The Accounting Review May 2015 Audit Program Modification: Fraud Specialists versus Auditors 883 The experimental case was presented to the fraud specialists and to the financial statement auditors under the scenario that the audit team has assessed the client as having a ''higher than low'' risk of fraud, and that the participants' assistance is sought to assess the adequacy of the audit program and to change it as necessary. Similar to Hoffman and Zimbelman (2009), we do not manipulate fraud risk as a factor in our experiment because our goal is to investigate the proposed changes to the audit program given a heightened risk of fraud. The participants were provided with a copy of a standard audit program adapted from Asare and Wright (2004) and asked to identify which procedures in the standard audit program were needed. They were also asked to supplement the standard program with any additional procedures that were required, in their opinion, to address the elevated risk of fraud in the revenue cycle. Finally, they were asked about the amount of time that should be budgeted for standard and additional procedures, as compared to the previous year. Our results reveal the following three insights. First, in a situation with elevated fraud risk, fraud specialists did not select more procedures from a standard audit program than financial statement auditors; nor were the selected procedures more effective than those selected by auditors. Second, the fraud specialists proposed more additional procedures than did auditors, and the specialist-proposed additional procedures were marginally more effective, but significantly less efficient, than the additional procedures proposed by auditors. Third, fraud specialists increased time budgets to reflect the additional effort that they proposed via extensive non-standard procedures. However, although they proposed significantly more additional procedures than did auditors, their proposed time budget increases for those procedures were significantly lower than adjustments proposed by the auditors. The next section reviews relevant literature and develops our hypotheses. Section III describes our methodology, and Section IV presents the results of the analysis of the experimental data. Section V discusses conclusions, limitations, and avenues for future research. II. LITERATURE REVIEW AND HYPOTHESES Fraud Risk Assessment and Audit Planning Audit research has addressed fraud risk assessment and associated planning decisions. Nieschwietz, Schultz, and Zimbelman (2000) and Hammersley (2011) provide detailed reviews of the literature on auditor judgment in fraud-related tasks. While earlier studies examined whether auditors incorporate information about fraud risk factors in their fraud risk assessments (e.g., Pincus 1989; Loebbecke, Eining, and Willingham 1989; Hackenbrack 1993; Heiman-Hoffman, Morgan, and Patton 1996), more recent studies have focused on whether information about fraud risk factors and associated risk assessments is reflected in auditors' decisions about the nature, timing, and extent of audit tests. These studies have shown that auditors respond to a high risk of fraud at a client primarily by changing the extent of the procedures and increasing the overall audit investment (e.g., Zimbelman 1997; Houston, Peters, and Pratt 1999; Glover, Prawitt, Schultz, and Zimbelman 2003; Mock and Turner 2005) and, to some degree, by modifying the nature of the proposed audit tests (e.g., Wright and Bedard 2000; Graham and Bedard 2003; Hammersley et al. 2011). This research has established that auditors respond to audit risk factors with an increased assessed risk of fraud, and that they are able to recognize the need for modifying the audit program in response to such risk (Cormier and Lapointe-Antunes 2006; Hammersley 2011), but are unable to design procedures that are effective and efficient at identifying fraud (Hammersley et al. 2011). PCAOB inspectors have also observed that auditors ''failed to expand audit procedures when addressing identified fraud risk factors'' (PCAOB 2007, 3) and failed to use the fraud risk assessment process ''with a view toward identifying ways to modify the audit plan in order to address the [fraud] risk'' (PCAOB 2007, 4). The Accounting Review May 2015 884 Boritz, Kochetova-Kozloski, and Robinson As the risk of fraud and associated losses to business organizations have been projected to rise (e.g., KPMG 2009; DeMarco 2010), researchers have investigated specific techniques and approaches that may increase auditors' effectiveness in fraud risk assessment and the quality of the corresponding adjustments to the audit program. Most notably, brainstorming techniques have shown good results in stimulating auditors' ability to develop quality ideas about potential fraud in a client, to incorporate them in fraud risk assessments (Carpenter 2007; Brazel, Carpenter, and Jenkins 2010), and to identify specific modifications to an audit program at a client where fraud is suspected (Brazel et al. 2010; Hoffman and Zimbelman 2009). The use of strategic reasoning (Wilks and Zimbelman 2004; Hoffman and Zimbelman 2009), priming auditors to reconsider the important fraud risks documented during planning before they start evidence evaluation, and use of specific fraud risk memos that include, along with a summary, detailed fraud risks discussed during the brainstorming session (Hammersley, Bamber, and Carpenter 2010) have shown potential as methods of making auditors more effective in addressing fraud risk during the audit planning and evidence evaluation process. Finally, research has documented that auditors recognize situations where their expertise alone is not sufficient in addressing the risk of fraud and that they are willing to consult with a technical expert, such as a risk management partner (Asare and Wright 2004). Since financial statement auditors are unable to design effective audit plans to address fraud risk, a potential response to heightened risk of fraud is to involve fraud specialists. The propensity to consult with fraud specialists tends to increase when firm-level guidance makes consultations mandatory and binding, particularly under conditions of high fraud risk and tight deadline pressure (Gold et al. 2012). However, to our knowledge, there is no published research that provides systematic empirical evidence (either archival or experimental) that fraud specialists design audit plans differently from auditors. Asare and Wright (2004) consider their participants' referrals to risk management partners as a positive outcome on the assumption that they are better than regular auditors at designing audit plans that effectively address fraud risks. In contrast, Jamal (2008, 103) suggests that forensic accountants may not be able to respond to fraud risk better, in the context of an assurance engagement, than financial statement auditors because their expertise may reside in ''doing detailed ex post data collection and evidence documentation work required to make a case for court.'' In other words, forensic specialists may be able to investigate a suspected fraud, but their expertise may not be suitable for designing audit tests to respond to fraud risk effectively and efficiently in the context of planning an assurance engagement. Fraud Specialists' Expertise The usefulness of consultations with fraud specialists, such as forensic accountants, is based on the premise that expertise in fraud investigation is directly transferable to the audit setting. In other words, it is assumed that the knowledge, skills, and attitudes that fraud specialists bring to the task of audit planning under conditions of elevated fraud risk are more relevant to the specific task than the knowledge, skill, and attitudes possessed by financial statement auditors who are not trained in financial forensics or fraud investigation. Research on audit expertise (Libby 1995, 178-188) and the effects of knowledge on judgment quality (Bonner 2008, 66-70) suggests that if fraud specialists' knowledge and skills are a better match for the specific task of audit program design under conditions of ''higher than low'' fraud risk than those possessed by auditors, then specialists will outperform auditors on that task. However, academic research has not yet addressed this conjecture from either the theoretical (i.e., the analysis of expertise required for the task) or the empirical points of view. There are studies examining opportunities for forensic auditors (Beasley 2003; DeZoort and Stanley 2006), curriculum and certification requirements for forensic accountants and auditors (Rezaee 2002; Meservy et al. 2006), and forensic experts' classification of fraud risk factors (Apostolou, Hassell, and Weber 2000; Hansen and Klamm 2004). None of The Accounting Review May 2015 Audit Program Modification: Fraud Specialists versus Auditors 885 these studies focus on whether fraud specialists' expertise is suitable for the audit-planning task and, as a result, whether their judgments can be effective in an audit-planning context. Hammersley (2011) and Hammersley et al. (2011) note that special characteristics of the financial statement fraud setting and auditors' limited expertise in making fraud-related judgments make it a challenging setting for audit program planning; however, these studies focus on judgments by regular financial statement auditors rather than fraud specialists. It would be reasonable to expect that, if consulted, fraud specialists with prior auditing training are likely to have sufficient knowledge that would enable them to recommend procedures that would effectively address the risk of fraud in the context of an audit engagement (Association of Certified Fraud Examiners [ACFE] 2010; AICPA 2006, 2010; Canadian Institute of Chartered Accountants [CICA] 2010; IFA Alliance for Excellence in Investigative and Forensic Accounting [IFA Alliance] 2010). In contrast, financial statement auditors are able, generally speaking, to design effective audit programs, but lack the knowledge and skill to design fraud discovery procedures (Houston et al. 1999) or modify the existing audit program well enough to address the risk of fraud (Hammersley et al. 2011). This lack of requisite expertise in fraud investigation would limit their performance in planning an audit under conditions of elevated fraud risk, as compared to fraud specialists with prior audit training. In addition to knowledge and skills, there is an attitudinal component of the expertise of fraud specialists that influences the quality of fraud risk judgmentsprofessional skepticism. When auditors are disciplined for their failure to detect the financial statement fraud of a client, 60 percent of the time, it is found that they did not exercise sufficient professional skepticism (Beasley, Carcello, and Hermanson 2001). Nelson (2009) relates professional skepticism to both experience and specialization, and notes that specialists and auditors with domain-specific experience are more likely to identify risks of domain-specific errors and modify their audit-planning decisions accordingly. We infer that Nelson's (2009) conclusion applies to fraud specialists and that the increased skepticism on the part of a fraud specialist, combined with his or her knowledge and skill derived from training and experience, ceteris paribus, will lead to better recognition of fraud risks (Hammersley et al. 2011), followed by the selection of more effective and efficient procedures from a standard audit program, as well as identification of non-standard procedures that will address the risk of fraud. Use of Specialists Although reliance on specialists may improve the effectiveness of an audit, this potential improvement comes at a cost. As noted by Hasseldine, Holland, and van der Rijt (2011), the very skepticism that can lead to positive planning outcomes can make specialist services unattractive to engagement partners, who may fear that excessive skepticism may needlessly inflate audit costs. Therefore, unless the improvements are significant, engagement partners may not avail themselves of the services of specialists due to cost-benefit considerations. For example, a report by the Canadian Public Accountability Board (CPAB 2009, 16) notes that ''some partners appeared to be reluctant to use specialists in difficult areas.'' Johnstone and Bedard (2003) find that the intent to use specialist personnel (i.e., personnel with relatively high industry expertise, functional expertise, and/or high-risk client experience) moderates the negative relationship between audit risk (i.e., risk of misstatement due to fraud or error) and the likelihood of client acceptance. They call for research on the effects of specialization on audit quality that systematically investigates specialists' application of their knowledge across audit tasks, focusing on where gains from the use of specialists are most likely to occur, how those gains can be leveraged by firms, and what factors might inhibit those gains (Johnstone and Bedard 2003, 1022). There are no published studies involving fraud specialists in an audit-planning context. However, several published studies in auditing have looked into use of computer information The Accounting Review May 2015 886 Boritz, Kochetova-Kozloski, and Robinson technology (IT) specialists in audit engagements. Brazel and Agoglia (2007) demonstrated that auditors with high accounting information systems expertise were able to effectively incorporate, and even compensate for, deficiencies in recommendations of computer assurance specialists in control risk assessments and planning of control tests within a complex accounting systems environment. However, in their study, the recommendations of computer assurance specialists were manipulated and provided to the participant auditors, rather than elicited directly from the specialists. The use of IT specialists is often less frequent than expected (Janvrin, Bierstaker, and Lowe 2008, 2009). Similarly, Hasseldine et al. (2011) observe that some potential users of tax specialists are reluctant to avail themselves of their services out of a concern that specialists' propensity to focus (excessively) on risks can lead to implementation of costly risk-mitigating measures that might not be necessary. Such behavior can create concerns for auditors seeking to stay within budgeted audit fee levels, especially under the current conditions of downward fee pressure (McCann 2010; Reason 2010). Finally, the effectiveness of specialists' services cannot simply be assumed. For example, in its inspection report, the CPAB (2009, 16) noted that ''the work of some [valuation] specialists lacked rigor.'' Fraud specialists are not routinely employed to help design or adjust audit programs in situations of elevated risk. Brazel et al. (2010) find that a fraud specialist attends a primary brainstorming/planning session in only 31 percent (Std. Dev. 46.34) of audit engagements in their sample (Brazel et al. 2010, 1284, Table 3). Fraud specialists are not necessarily trained for, and do not typically work on, financial statement audit-planning tasks where fraud has not yet been identified or fraud risk is assessed at a low level. Instead, they focus on evaluating situations where a fraud has already been identified; the job of the fraud specialists is to determine the magnitude of the problem. These are different tasks with different goals. As such, unless the use of a specialist can be shown to clearly contribute to the more effective and efficient delivery of professional services, financial statement auditors are likely to underutilize such specialists. This underutilization can be particularly problematic in a fraud risk context since, unlike the technical risks addressed by IT and tax specialists, fraud risks relate to strategic actions taken by client personnel to perpetrate and conceal wrongdoing. Audit responses to such actions may require insights from both auditors and specialists in order to guide the design of effective and efficient audit plans, especially when fraud risks are elevated, but not to the level that clearly calls for a fraud investigation. Hypotheses Based on the above discussion, we predict that the fraud-related knowledge, skills, and skepticism of fraud specialists will influence their program-planning judgments and will lead to greater effort in addressing fraud risk, such as selection of more (and more effective) procedures from a standard audit program, as well as identification of more (and more effective) non-standard procedures, as compared to financial statement auditors. We examine modifications proposed to the standard audit program separately from the proposed non-standard additional procedures because the previous research has distinguished these planning decisions as two distinct sets of judgments (e.g., Asare and Wright 2004). Indeed, the expertise paradigm would suggest that selecting from a list of standard procedures is a cognitively different task than developing an unaided list of additional, non-standard procedures. Fraud specialists' expertise is likely to apply more directly to the latter, and less so to the former. We expect that fraud specialists will do the following: (1) increase overall audit effort by proposing greater numbers of both standard and additional (non-standard) audit procedures; (2) select more effective audit procedures from a standard program when the risk of fraud is higher than low; (3) propose more effective additional, non-standard procedures than those proposed by financial statement auditors; and, as a corollary to (1), (2), and (3), (4) increase the budgeted hours allocated to all (standard and additional) procedures in the audit program. Stated formally: The Accounting Review May 2015 Audit Program Modification: Fraud Specialists versus Auditors 887 H1a: When risk of fraud is other than low, fraud specialists will select more procedures from a standard audit program than will auditors. H1b: The standard procedures selected by fraud specialists will be more effective, as judged by audit experts, than those selected by auditors. H2a: When risk of fraud is other than low, fraud specialists will propose more additional procedures than will auditors. H2b: The additional procedures proposed by fraud specialists will be more effective, as judged by audit experts, than the procedures proposed by auditors. When fraud specialists are consulted for program-planning advice, they may feel less constrained by the budget than the audit team. As a result, fraud specialists may identify procedures that, while effective, may not be viewed as efficient by the engagement partner or the audit team members (Jamal 2008). This could be due to the fact that a typical audit engagement is much more budget-driven than a typical forensic engagement, as described to us by partners from Big 4 forensic and assurance practices in Canada and in the U.S., and from a specialty forensic firm in Canada. In other words, fraud specialists may propose procedures that would address fraud risk successfully, but, in the eyes of the engagement partner, may be too costly (i.e., inefficient). Therefore, our third hypothesis states: H3: When risk of fraud is other than low, fraud specialists will propose additional procedures that are less efficient, as judged by audit experts, than the additional procedures proposed by auditors. While individual procedures may be effective and/or efficient, a combination of several such procedures, when added to the audit program, may result in a significant increase in the budgeted hours for the engagement. If H1a and H2a are supported, then fraud specialists should increase the budgeted hours for the engagement to a greater extent than auditors, as we anticipate that they are likely to suggest conducting a greater number of standard and extended procedures than auditors. Therefore, our fourth hypothesis states: H4: When risk of fraud is other than low, fraud specialists will increase the budgeted hours allocated to the procedures in a standard audit program and additional procedures to a greater extent than will auditors. III. METHODOLOGY Experimental Design and Task The experimental design includes one between-participants factor, Participant Type (fraud specialists versus financial statement auditors). Fraud specialists and auditors participated in an audit-planning task by selecting procedures from a standard audit program for the revenue cycle, followed by proposing additional, extended procedures and revising the associated time budget.3 3 We also attempted to manipulate another factor (i.e., mode of involvement in audit planning at two levels: participative versus consultative) between subjects in the fraud specialist subsample. Thus, we administered two versions of the case: one where participants made their own risk assessments (participative mode), and another where the risk assessments were provided to them (consultative mode). However, our manipulation was either (1) not sufficiently strong to produce meaningful results, or (2) resulted in no significant differences in fraud specialists' selections of procedures compared with those of auditors. Since there were no differences between those conditions with respect to the main dependent variables in the study, these conditions were collapsed into one. The Accounting Review May 2015 888 Boritz, Kochetova-Kozloski, and Robinson Participants completed an audit case that was adapted from Asare and Wright (2004) and was based on an actual company that had issued fraudulent financial statements. The company in the case is a manufacturing firm and the audit of the revenue cycle requires no industry-specific training. The information in the case indicated fraud risk as being higher than low, but not extremely high; that is, there was no direct indication in the case that fraud had been discovered. First, participants were presented with comparative financial statements and information about the company. Next, they were asked to help in designing an audit program for the revenue cycle by (1) selecting procedures from a standard audit program, (2) proposing additional procedures, and (3) modifying as necessary the budgeted hours for procedure categories. In the final part of the experimental instrument, participants were asked to provide demographic information. Packages containing sets of case materials in individual envelopes were distributed by mail to contacts at the firms who agreed to participate in the study. The contacts distributed the cases to fraud specialists (or auditors) in their firm. Once the cases were completed, they were returned by the participants directly to the researchers in a self-addressed, postage-paid envelope. Standard tests for early versus late respondents revealed no differences in key variables of the study. Participants Thirty-two fraud specialists and 16 auditors completed the case. Half of the fraud specialists were from Big 4 accounting firms and half were from medium-size firms. Sixty percent of the auditors were from Big 4 firms and 40 percent were from medium-size firms. The participants in both groups were partners, directors, or managers with a significant amount of practical experience. On average, the fraud specialists were 41 years old, had 12 years of specialized fraud-related experience, and six years of auditing experience. The auditors were, on average, 36 years old, had 13.25 years of auditing experience, but no specialized fraud experience. Thus, on average, auditors had significantly greater auditing experience (13.25 versus 6.02 years, p 0.002) than fraud specialists; however, fraud specialists had significantly greater specialized fraud experience (12.06 years versus 0.00, p , 0.001). Nevertheless, fraud specialists' average audit experience was substantial (6.02 years). Twenty-one of the fraud-specialist participants had an IFA (Investigative Forensic Accountant) specialist designation; some also had a CPA and/or CFE (Certified Fraud Examiner) designation. All IFA-designated fraud specialists were also Chartered Accountants (CAs). On average, the fraud specialists spent about 52 percent of their time on fraud-related activities and 37 percent on litigation-related activities. The auditors spent, on average, 89 percent of their time on audit-related activities. Thus, we conclude that both the auditor and fraud-specialist samples were qualified and experienced in their respective domains and were appropriate participants for the experimental task. Dependent Variables Measures of the dependent variable, the quality of the audit program for the revenue cycle, include the following: (1) the number of procedures selected from a standard audit program for the revenue cycle, in total and by type of procedure (i.e., tests of the aged trial balance, confirmations, adequacy of bad debt provisions, cut-off, and analytical procedures); (2) the effectiveness of procedures selected from the standard program; (3) the number of additional procedures proposed to supplement the standard audit program; (4) the effectiveness of additional participant-proposed procedures; (5) the efficiency of additional procedures; and (6) the proposed revisions to the time budget for the final audit program for the entire revenue cycle and by type of procedure. We invited two groups of audit experts, each consisting of ten audit partners and senior managers, to serve as our expert panels. We asked the first panel to evaluate the effectiveness and The Accounting Review May 2015 Audit Program Modification: Fraud Specialists versus Auditors 889 efficiency of the procedures in the standard program, and the second panel to evaluate the effectiveness and efficiency of the additional procedures recommended by auditors and specialists. We invited audit experts, rather than forensic or fraud experts, to rate effectiveness and efficiency of procedures because our discussions with audit partners from the Big 4 assurance practices in the U.S. and Canada indicated that the ultimate decision about the use of fraud specialist-recommended procedures on an audit engagement lies with the engagement partner. Hence, we believe that it is appropriate to use expert panels of experienced auditors (as opposed to partners in forensic practice or forensic specialty firms) in judging the quality (effectiveness, efficiency) of an audit program. The audit experts on both panels were partners and senior managers from two Big 4 (six experts on the first, and four experts on the second panel) and the two next-largest international firms (Grant Thornton and BDO Dunwoody) (four and six, respectively). Four (six) of them were male and six (four) were female. Their average age was 39 (42), and they had, on average, 17 (18) years of audit experience. We conclude that both of our panels of experts had the expertise required to evaluate the effectiveness and efficiency of audit procedures, and they were very comparable to each other in terms of experience, rank in the firm, gender, and age (all p-values . 0.10). The audit expert panels were not involved in any of the foregoing phases of the study. They were given the same case information as the auditors and fraud specialists. They were not told that a fraud had actually occurred in the case company. The first panel was given a standard audit program for the revenue cycle and was asked to assess the effectiveness and efficiency of each procedure in this program. This was done using a scale from 0 to 10, where 0 meant ''totally ineffective (inefficient)'' and 10 meant ''extremely effective (efficient).'' The second expert panel was given a standard audit program (but not asked to evaluate it), as well as a list of the 65 additional (non-standard, extended) procedures that participants had proposed during the main experiment. These procedures were presented to the experts as additional, extended procedures that members of the audit team had generated as a part of their planning process. The audit experts were asked to review the additional procedures and assess the effectiveness (efficiency) of each procedure on a scale from 0 to 10, where 0 meant ''totally ineffective (inefficient)'' and 10 meant ''extremely effective (efficient).'' We did not specifically define the terms ''effectiveness'' and ''efficiency'' for the panels, but expected them to apply the common understanding of these terms as they are used in auditing; that is, ''effectiveness'' of an audit procedure is its ability to achieve an audit objective, regardless of the cost, and ''efficiency'' of an audit procedure is the achievement of an audit objective without undue or excessive cost. Effectiveness takes precedence over efficiency, since there would be little value in conducting a procedure that was efficient, but ineffective. However, at a satisfactory level of effectiveness, various procedures may possess different degrees of efficiency. We used experts' average ratings of effectiveness and efficiency of each standard procedure (first panel) and each additional procedure (second panel) as ''weights'' to develop overall effectiveness and overall efficiency scores for standard procedure selections and for additional procedure listings, respectively. We also analyzed the overall quality of an audit procedure, defined as the combination of its effectiveness and efficiency. In our analyses, we control for audit experience to separate the effect of audit experience from fraud experience. As Table 1 shows, fraud specialists in our sample had an average of about six years of audit experience, whereas the auditors had an average of 13 years of audit experience. The audit experience of the auditors may compensate, to a degree, for their lack of specialized fraud experience and may make it difficult to tease out the contribution of the fraud specialists to the audit program plan. Therefore, as we test each hypothesis, we provide additional analyses with audit experience as a covariate. The Accounting Review May 2015 Boritz, Kochetova-Kozloski, and Robinson 890 TABLE 1 Participant Demographic Data Mean/Count (Standard Deviation/Percentage of the Sample) Fraud Specialists (n 31)a Demographic Variable Age Gender Years of Specialized Fraud Experience Years of Auditing Experienceb Current Position 41.29 (8.25) 21 (67.7%) 12.06 (7.50) years male years 35.81 7 0.00 (6.56) years (43.8%) male (0.00) years 6.20 (7.00) 12 (38.7%) 16 (51.6%) years partners senior managers/ directors managers 13.25 5 6 (6.93) years (31.3%) partners (37.5%) senior managers 3 CA designation CPA designation CFE designationc IFA designationc Number of times contacted for [sought] consultation by audit group [of fraud specialists] during the last five yearsd Auditors (n 16) 26 5 3 28 10 21 21 10 4.69 (9.7%) (83.9%) (16.1%) (9.7 %) (90.3%) (32.3%) (67.7%) (67.7%) (32.3%) (9.49) yes no yes no yes no yes no 4 (25%) 1 (6.3%) 16 (100%) 0 (0%) 3 (18.8%) 13 (81.3%) 0 (0%) 16 (100%) 0 (0%) 16 (100%) 0.12 (0.00) managers principal yes no yes no yes no yes no a 31 out of 32 fraud specialists provided demographic information. Range of audit experience was from 0 minimum to 36 maximum years for fraud specialists, and from 4 minimum to 25 maximum for auditors; range of specialized fraud experience was from 4 minimum to 30 maximum years for fraud specialists. c CFE Certified Fraud Examiner (ACFE 2010); IFA Investigative Forensic Accountant (IFA Alliance 2010). d Range was from 0 minimum (11 participants) to 45 (one participant) times for fraud specialists, and from 0 minimum (15 participants) to 2 maximum (one participant) for auditors. b IV. ANALYSIS AND RESULTS Tests of H1 H1a predicts that fraud specialists will select a greater number of procedures from the standard audit program for the revenue cycle than financial statement auditors. We test this hypothesis by comparing the number of procedures selected from a standard audit program for the revenue cycle by fraud specialists and auditors. When we compare the number of standard procedures selected in total, we find no statistically significant difference between the number of procedures selected by fraud specialists and auditors (Table 2, Panel A: means 14.28 versus 13.81, respectively, p 0.422, one-tailed). When we examine differences in procedure selection by groups of procedures, we find no significant differences (Table 2, Panel A). Thus, we conclude that H1a is not supported. H1b predicts that the standard procedures selected by fraud specialists will be more effective than those selected by auditors. To test H1b, we use the expert panel's ratings of standard procedure The Accounting Review May 2015 Audit Program Modification: Fraud Specialists versus Auditors 891 TABLE 2 Standard Audit Program Selections Descriptive Statistics Panel A: Number of Standard Procedures Selected Fraud Specialists Auditors (n 32) (n 16) Mean Mean (Std. Dev.) (Std. Dev.) p-value Variable Number of Standard Procedures Selected for: tests of aged trial balance (out of 3) 2.06 (1.16) 4.28 (2.36) 3.50 (1.55) 1.47 (0.80) 3.97 (2.44) 1.81 (1.28) 4.56 (2.34) 2.25 (1.39) 1.63 (0.81) 3.56 (2.07) 14.28 (7.90) 13.81 (7.30) confirmations (out of 6) tests of adequacy of bad debt provision (out of 4) cut off tests (out of 2) analytical procedures (out of 8) Total number of procedures selected from standard program (out of 23) 0.250 0.349 0.294 0.265 0.286 0.422 Panel B: Proposed Revisions to the Time Budget for the Final Audit Program Variable Revisions in % relative to original budget for: tests of aged trial balance confirmations tests of adequacy of bad debt provision cut-off tests analytical procedures other tests Total revisions to the time budget for the final audit program, in % relative to original budget for all tests Fraud Specialists (n 32) Mean (Std. Dev.) Auditors (n 16) Mean (Std. Dev.) 1.00 (3.46) 4.63 (5.52) 4.88 (4.61) 6.53 (6.11) 4.28 (5.37) 6.88 (10.61) \u00025.00 (18.71) 3.13 (5.44) 5.00 (7.53) 1.56 (3.01) 3.13 (13.89) 23.12 (32.24) 18.50 (15.68) 20.44 (19.44) p-value 0.042 0.189 0.472 0.002 0.340 0.006 0.356 All p-values are based on a t-test and are one-tailed given the directional nature of the hypotheses. The p-values in bold are equal to or below 0.05. The Accounting Review May 2015 892 Boritz, Kochetova-Kozloski, and Robinson effectiveness and efficiency to compare auditors' and fraud specialists' selections from a standard program. Appendix A provides the proportions of participants who selected individual procedures from the standard audit program for the revenue cycle, and our first expert panel's ratings of the effectiveness and efficiency of those procedures. We create cumulative effectiveness scores for participants' selections from the standard program by group of procedures, and for the entire standard program, using average ratings provided by the first panel of audit experts as effectiveness ''weights'' for each standard procedure (see the second column of Appendix A). For instance, a cumulative effectiveness score for the ''tests of aged trial balance'' group of standard procedures is calculated as 5.6 3 (0 or 1, depending on whether a participant selected procedure ''a'' from this group) 6.9 3 (0 or 1, depending on whether a participant selected procedure ''b'' from this group) 6.2 3 (0 or 1, depending on whether a participant selected procedure ''c'' from this group). If the participant selected procedures ''a'' and ''b,'' but not ''c,'' then his or her effectiveness score for this category would be 5.6 3 1 6.9 3 1 6.2 3 0 12.5; if he or she selected all three, then his or her score would be 5.6 3 1 6.9 3 1 6.2 3 1 18.7. We perform this calculation for each participant and for each group of standard procedures. Descriptive statistics for these scores, as well as for effectiveness scores for all selections from the standard program, are presented in Table 3. We perform the same type of calculation for efficiency scores for the same groups of procedures and for overall selections from the standard program (see Table 3 and Appendix A). Fraud specialists show an average effectiveness score of 104.06 (Std. Dev. 57.18) as compared to auditors with a mean of 99.06 (Std. Dev. 51.71), but this difference is not statistically significant using a t-test (p . 0.10) (Table 3, Panel A). Efficiency scores for the standard program show a similar pattern, with means of 100.31 (Std. Dev. 55.37) and 94.22 (Std. Dev. 49.22) for fraud specialists and auditors, respectively (p . 0.10). When we examine effectiveness scores by category of procedures, we do not find significant differences, based on a series of t-tests, between fraud specialists and auditors in effectiveness of standard procedures in individual categories (Table 3, Panel B: all p-values . 0.10). Similarly, the efficiency scores of standard procedures, when analyzed by category, are not significantly different between fraud specialists and auditors (Table 3, Panel C: all p-values . 0.10). The individual t-tests reported in Table 3 do not take into account the potential inter-dependencies between the effectiveness and efficiency of procedures selected in the standard program.4 Thus, we perform MANCOVAs with standard program effectiveness and efficiency scores as dependent variables and Participant Type (fraud specialists versus auditors) as the independent variable, and the number of standard procedures selected from the standard program and/or audit experience as a covariate. Participant Type is significant: F(2,44) 5.505, p 0.007 when we control for the number of procedures selected from the standard program; F(2,42) 6.414, p 0.004 when we control for audit experience; and F(2,41) 6.552, p 0.003 when we control for both (not tabulated). We perform further analyses whereby we include audit experience and the total number of procedures selected from the standard program as covariates in a series of ANCOVAs with overall effectiveness of audit procedures selected by participants from a standard program as a dependent variable. These analyses do not include efficiency as a covariate because, strictly speaking, in order for the procedure to be effective, it does not need to be efficient. When we control for the total number of standard procedures selected and audit experience, we find that at the standard program level, fraud specialists did not select more effective procedures than auditors (F(1,42) 0.347, p 0.559, not tabulated); marginal means for standard program effectiveness score are 100.686 for auditors and 99.954 for fraud specialists (not tabulated). This finding indicates lack of support for H1b. 4 Indeed, overall effectiveness and efficiency scores of standard procedures are highly correlated both in our participant sample (0.984, p , 0.001, two-tailed) and in the expert panel (0.786, p , 0.001, two-tailed). The Accounting Review May 2015 Audit Program Modification: Fraud Specialists versus Auditors 893 TABLE 3 Effectiveness and Efficiency of Audit Procedures Selected by Participants from a Standard Program Panel A: Overall Scores Variable Fraud Specialists (n 32) Mean (Std. Dev.) Auditors (n 16) Mean (Std. Dev.) 104.06 (57.18) 100.31 (55.37) 99.06 (51.71) 94.22 (49.22) Overall Effectiveness Score of Procedures Selected from the Standard Program Overall Efficiency Score of Procedures Selected from the Standard Program p-value 0.385 0.356 Panel B: Effectiveness Scores by Group of Procedures in the Standard Program Variable 1. Tests of aged trial balance 2. Confirmations 3. Tests of adequacy of bad debt allowance 4. Cut-off tests 5. Analytical procedures Fraud Specialists (n 32) Mean (Std. Dev.) Auditors (n 16) Mean (Std. Dev.) 12.98 (7.22) 33.03 (18.05) 18.52 (11.26) 11.24 (6.14) 28.29 (17.03) 11.42 (7.91) 35.19 (18.01) 16.88 (10.21) 12.43 (6.17) 23.14 (12.91) p-value 0.250 0.349 0.313 0.266 0.146 Panel C: Efficiency Scores by Group of Procedures in the Standard Program Variable 1. Tests of aged trial balance 2. Confirmations 3. Tests of adequacy of bad debt allowance 4. Cut-off tests Fraud Specialists (n 32) Mean (Std. Dev.) Auditors (n 16) Mean (Std. Dev.) 12.95 (7.25) 29.45 (16.15) 16.30 (9.68) 10.22 (5.57) 11.54 (7.90) 31.41 (16.10) 15.19 (8.89) 11.29 (5.60) p-value 0.270 0.347 0.352 0.267 (continued on next page) The Accounting Review May 2015 Boritz, Kochetova-Kozloski, and Robinson 894 TABLE 3 (continued) Variable 5. Analytical procedures Fraud Specialists (n 32) Mean (Std. Dev.) Auditors (n 16) Mean (Std. Dev.) 31.39 (19.21) 24.79 (14.05) p-value 0.115 All p-values are based on a t-test and are one-tailed given the directional nature of H1b. Tests of H2 H2a and H2b predict that when risk of fraud is other than low, fraud specialists will propose more additional procedures that are also more effective at detecting fraud than the additional procedures proposed by auditors. We test these hypotheses in two ways. First, we compare the additional procedures suggested by auditors against those suggested by fraud specialists. Appendix B summarizes 65 additional procedures that one or more of our participants proposed to add to the standard audit program. This list includes procedures that were suggested by Asare and Wright (2004, Exhibit 3, based on the description of a fraud in a 1998 Securities and Exchange Commission [SEC] Accounting and Auditing Enforcement Release). The additional proposed procedures fall into the following seven areas: (1) review of contracts, (2) revenue recognition, (3) confirmations, (4) inventory/warehouse, (5) allowance for bad debts, (6) cut-off tests, and (7) analytical procedures. Appendix B, Panel A indicates that fraud specialists, as a group, proposed a greater variety of additional procedures than did auditors, selecting 60 out of 65 procedures on the list, whereas auditors, as a group, proposed only 22 out of 65 procedures on the list. Across all categories and types of procedures, a greater proportion of fraud specialists proposed additional procedures v2(10, n 48) 13.885, p 0.027, one-tailed). Panel B of Appendix B shows that 29 fraud specialists (91 percent of the sample) proposed between one and 18 additional procedures, whereas 15 (94 percent of the sample) auditors proposed between one and five additional procedures. The average number of additional non-redundant procedures proposed by fraud specialists was 4.47 (Std. Dev. 3.95), whereas for auditors, it was 2.75 (Std. Dev. 1.44), and this difference was statistically significant (p 0.05, one-tailed). Thus, we find support for H2a. Fraud specialists proposed a greater number of additional procedures, on average, than did auditors. Similarly to the calculation of effectiveness and efficiency scores for standard procedures described previously, we create cumulative scores for the effectiveness of additional procedures by category for each participant (based on the categories in Appendix B, Panel A) using the average ratings provided by the second panel of audit experts as ''weights'' for each procedure (see the third column of Appendix B, Panel A). We perform this calculation for each participant and for each of the seven additional procedure categories. Finally, we calculate overall effectiveness scores for all additional procedures proposed by each participant by summing category-level scores. The scores for efficiency of additional procedures are calculated in a similar way using corresponding average ratings from the fourth column in Appendix B, Panel A. Descriptive statistics for these scores are reported in Table 4. An examination of mean overall effectiveness scores using a t-test indicates a marginally statistically significant difference between fraud specialists and auditors (Table 4, Panel A: p 0.074, one-tailed). This provides a preliminary indication that the overall effectiveness of additional procedures listed by fraud specialists is greater than that for auditors (means are 26.84 for fraud specialists versus 18.39 for auditors). The Accounting Review May 2015 Audit Program Modification: Fraud Specialists versus Auditors 895 TABLE 4 Effectiveness and Efficiency of Participant-Proposed Additional Procedures Panel A: Overall Scores Variable Fraud Specialists (n 32) Mean (Std. Dev.) Auditors (n 16) Mean (Std. Dev.) 26.84 (21.62) 26.96 (24.68) 18.39 (10.32) 18.28 (10.12) Overall Effectiveness Score of Additional Procedures Overall Efficiency Score of Additional Procedures p-value 0.074 0.093 Panel B: Effectiveness Scores by Procedure Category Variable 1. Procedures to understand the marketing program and review contracts 2. Procedures to test revenue recognition. 3. Confirmations 4. Inventory-related procedures 5. Procedures to test allowance for bad debts 6. Cut-off tests 7. Analytical procedures Fraud Specialists (n 32) Mean (Std. Dev.) Auditors (n 16) Mean (Std. Dev.) 11.82 (9.48) 3.11 (4.52) 1.05 (2.26) 2.64 (3.85) 3.52 (3.46) 0.89 (3.27) 3.77 (4.51) 7.06 (5.64) 4.94 (4.90) 0.00 (0.00) 1.29 (2.37) 3.30 (4.45) 0.00 (0.00) 1.78 (3.36) Fraud Specialists (n 32) Mean (Std. Dev.) Auditors (n 16) Mean (Std. Dev.) 12.20 (12.96) 3.13 (4.63) 0.93 (1.99) 7.02 (5.64) 5.04 (5.00) 0.00 (0.00) p-value 0.036 0.103 0.035 0.102 0.425 0.142 0.063 Panel C: Efficiency Scores by Procedure Category Variable 1. Procedures to understand the marketing program and review contracts 2. Procedures to test revenue recognition 3. Confirmations p-value 0.068 0.098 0.035 (continued on next page) The Accounting Review May 2015 Boritz, Kochetova-Kozloski, and Robinson 896 TABLE 4 (continued) Variable 4. Inventory-related procedures 5. Procedures to test allowance for bad debts 6. Cut-off tests 7. Analytical procedures Fraud Specialists (n 32) Mean (Std. Dev.) Auditors (n 16) Mean (Std. Dev.) 2.68 (3.90) 3.38 (3.31) 0.80 (3.00) 3.85 (4.56) 1.39 (2.53) 3.13 (4.43) 0.00 (0.00) 1.71 (3.29) p-value 0.117 0.414 0.146 0.051 All p-values are based on a t-test and are one-tailed given the directional nature of H2b and H3. Bold values are equal to or below 0.05; values in italic are equal to or below 0.10. Similarly to standard procedures, the effectiveness and efficiency of additional procedures are positively correlated.5 Therefore, we perform a MANCOVA with additional procedure effectiveness and efficiency scores as dependent variables and Participant Type (fraud specialists versus auditors) as the independent variable, and audit experience and number of procedures proposed as covariates. Participant Type is marginally significant (F(2,41) 2.023, p 0.072, onetailed, not tabulated), while both covariates are significant (for audit experience F(2,41) 3.304, p 0.047; for number of additional procedures proposed F(2,41) 525.567, p , 0.001, not tabulated). These results suggest that fraud specialists' expertise makes a contribution to planning additional effective and efficient audit procedures beyond that which comes from their audit experience. To test H2b further, we perform a series of ANCOVAs with effectiveness of additional procedures as a dependent variable, Participant Type as a fixed factor, and audit experience and number of additional procedures proposed as covariates. Using the same logic as in tests of H1b, we do not include efficiency as a covariate in these tests. We find no difference between fraud specialists and auditors (F(1,42) 0.002, p 0.964, two-tailed, not tabulated), although the number of additional procedures proposed is significant (F(1,42) 838.361, p , 0.001, two-tailed, not tabulated). The marginal means for effectiveness of additional procedures score for fraud specialists and for auditors are 24.613 and 24.545, respectively. We interpret these results to indicate that additional procedures proposed by fraud specialists are not necessarily more effective by their nature than those proposed by auditors; they appear to be marginally more effective due to their greater number. They also appear more effective when analyzed in combination with their efficiency ratings. These findings indicate limited support for H2b. Tests of H3 H3 states that fraud specialists' proposed additional procedures will be less efficient than those proposed by auditors. We use cumulative efficiency scores of additional procedures, calculated in the manner similar to effectiveness scores described in the previous section, as a dependent variable to test this hypothesis. In Table 4, Panel A, the overall efficiency of additional procedures proposed by fraud 5 Correlations between effectiveness and efficiency ratings of additional procedures were 0.980 (p , 0.001, two-tailed) for the participant sample and 0.945 (p , 0.001, two-tailed) for the expert panel. The Accounting Review May 2015 Audit Program Modification: Fraud Specialists versus Auditors 897 TABLE 5 Tests of H3 Analysis of Efficiency of Additional Procedures Panel A: ANCOVA for Efficiency of Additional Procedures (Efficiency of Additional Procedures) as a Dependent Variable, by Participant Type (Fraud Specialists versus Auditors) Source df F p-value (two-tailed) Participant Type Audit Experience Effectiveness of Additional Procedures Number of Additional Procedures Error 1 1 1 1 43 4.045 5.058 23.373 11.103 0.051 0.030 0.000 0.002 p-values in bold are equal to or below 0.05. Variable Definitions: Audit Experience number of years of audit experience, a covariate; Number of Additional Procedures number of non-redundant, meaningful additional procedures proposed by a participant as a supplement to the standard program for the audit of revenue cycle, a covariate; Effectiveness of Additional Procedures overall effectiveness score of additional procedures, a covariate; and Participant Type fraud specialists or auditors, a fixed factor. Panel B: Marginal Means for Efficiency of Additional Procedures, by Participant Type (Fraud Specialists versus Auditors) Participant Type Marginal Mean Standard Error Fraud Specialists Auditors Grand Mean 23.712 26.392 24.052 0.719 1.029 0.586 Covariates appearing in the model in Panel A are evaluated at the following values: Audit Experience 8.65; Effectiveness of Additional Procedures 24.59; and Number of Additional Procedures 3.98. specialists is 26.96 (Std. Dev. 24.68) versus 18.28 (Std. Dev. 10.12) for those proposed by auditors. A t-test of differences in mean efficiency scores of the additional procedures shows a marginally significant difference between fraud specialists and auditors (Table 4, Panel A: p 0.093, one-tailed). This indicates a preliminary lack of support for H3 in that the overall efficiency of additional procedures is marginally greater for fraud specialists than for auditors, whereas H3 predicted that the overall efficiency of additional procedures would be lower for fraud specialists than for auditors. To test H3 further, we performed a series of ANCOVAs with efficiency of additional procedures as a dependent variable, Participant Type as a fixed factor, and audit experience, number of additional procedures proposed, and effectiveness of the proposed procedures as covariates. We include effectiveness of additional procedures as a covariate based on the logic that an efficient procedure must be effective; otherwise, it is useless. Table 5 reports the results.6 We find that efficiency of additional procedures proposed by fraud specialists is lower than the 6 MANCOVA results are discussed in the ''Tests of H2'' section. The Accounting Review May 2015 898 Boritz, Kochetova-Kozloski, and Robinson efficiency of auditors' additional procedures (Table 5, Panels A and B: F(1,43) 4.045, p 0.026, one-tailed, marginal means are 23.712 for fraud specialists and 26.392 for auditors). Thus, H3 is supported. Tests of H4 H4 states that fraud specialists would revise the time budget for a standard audit program of the revenue cycle to a greater degree than auditors. Therefore, we compare revisions to the previous year's budgeted hours for the various categories of procedures by fraud specialists and auditors. Table 2, Panel B indicates that there is no significant difference between fraud specialists and auditors in terms of the proposed revision to the overall time budget for the audit program (p 0.356, one-tailed, with mean revisions of 18.50 percent relative to the original budget for fraud specialists and 20.44 percent for auditors). However, fraud specialists revised the time budget differently from auditors for the following categories of procedures: tests of the aged trial balance (mean increase of 1 percent by specialists versus mean decrease of 5 percent by auditors, p 0.042, one-tailed); cut-off tests (mean increase of 6.53 percent versus mean increase of 1.56 percent, respectively, p 0.002, one-tailed); and for other tests (mean increase of 6.88 percent versus 23.12 percent, respectively, p 0.006, one-tailed). In comparison with Asare and Wright's (2004) auditors, who made no meaningful adjustments to the time budget, both auditors and fraud specialists in this study adjusted the total time budget significantly upward (p , 0.001 for mean increase to budgeted hours above zero). To recognize variations in the direction of adjustments (increase, decrease) to the time budget for individual groups of procedures, we use non-parametric tests. Wald-Wolfowitz runs, Mann-Whitney U Tests, and Kolmogorov-Smirnov tests indicate fraud specialists proposed statistically significantly different changes to the time budget than did auditors in the following areas of the standard program: test of aged trial balance, cut-off tests, and other tests (all p , 0.01, two-tailed), with a greater number of increases (based on ranks) to the budgets for the former two groups of procedures, and a lower number of increases to the budget for the latter. Fraud specialists increased the time budgets for all types of procedures (p , 0.001), but they only did so to a significantly greater extent than auditors for tests of the aged trial balance and for cut-off tests. For other procedures, while proposing an increase in the time budget, fraud specialists actually suggested a smaller increase than auditors (Table 2, Panel B: 6.88 percent versus 23.12 percent, respectively, p 0.006, one-tailed). Thus, H4 is not supported. Combined with the results of tests of H2a, which indicated that the fraud specialists added significantly more additional procedures than did auditors, this finding may also indicate that fraud specialists' time budgets may not accurately reflect the amount of time it takes to perform the greater number of procedures they recommended. In summary, we observe that fraud specialists appear to increase the number of additional procedures without proportionate adjustments to the total time budgeted for those procedures. That is, they proposed a greater number of other additional procedures that are also more effective than those proposed by auditors (see discussion of H2a and H2b analyses), but they did not propose proportionate increases in the time budget for these tests (H4). Additional Analyses: Overall Program Quality To investigate our findings further, we analyze the overall quality of the audit programs proposed by fraud specialists and auditors. First, to supplement our analyses of H1b and H2b, we compare the overall effectiveness of all procedures, standard and extended, across fraud specialist and auditor groups. An ANCOVA using the cumulative effectiveness scores for standard and additional procedures as a dependent variable showed that Participant Type was not significant The Accounting Review May 2015 Audit Program Modification: Fraud Specialists versus Auditors 899 (F(1,41) 0.553, p 0.461, marginal means are 125.542 for fraud specialists and 124.043 for auditors, not tabulated), while the following control variables were statistically significant: number of procedures selected from the standard program (F(1,41) 4205.578, p , 0.001) and number of additional procedures proposed by the participant (F(1,41) 404.369, p , 0.001) (not tabulated). We interpret these results as indicating that while fraud specialists' proposed additional procedures are marginally more effective than those proposed by auditors (H2b), due to the lack of similar superior effectiveness in selecting standard procedures (H1b), the cumulative effectiveness of their audit programs is not higher than that of auditors. A limitation of this analysis is that it solely measures the cumulative effectiveness of the individual procedures, but does not assess synergies that may exist among procedures. Second, we perform a similar analysis to supplement our tests of H3. We compare the overall efficiency of all procedures, standard and additional, between the two participant groups. ANCOVA results (not tabulated) indicate that Participant Type is not significant (F(1,41) 1.292, p 0.131, marginal means are 120.412 for fraud specialists and 121.967 for auditors), while the following covariates are significant: number of additional procedures proposed by the participant (F(1,41) 39.908, p , 0.001), overall program effectiveness (F(1,41) 8271.151, p , 0.001), and years of audit experience (F(1, 41) 3.657, p 0.063). Thus, it appears that when we consider overall program efficiency, fraud specialists' performance is not significantly different from auditors' (see, also, tests of H3). Role of Audit Experience Both the fraud specialists and the auditors who participated in this study had a significant amount of audit experience. Table 1 indicates that in our sample, fraud specialists had an average of about six years of audit experience and 12 years of specialized fraud experience, whereas auditors had an average of 13 years of audit experience, but no specialized fraud experience. The greater audit experience of the auditors may compensate, to a degree, for their lack of specialized fraud experience and may make it difficult to tease out the contribution of the fraud specialists to the audit program plan. Therefore, as we tested H1b, H2b, and H3, we perform