JPMA-01726; No of Pages 12 Available online at www.sciencedirect.com ScienceDirect International Journal of Project Management xx (2015) xxx - xxx www.elsevier.com/locate/ijproman Does Agile work? A quantitative analysis of agile project success Pedro Serrador a,b , Jeffrey K. Pinto c a Serrador Project Management, Box 38032, 1250S. Service Rd., Mississauga, ON L5E 3G3, Canada b Humber College, 205 Humber College Blvd, Toronto, ON M9W 5L7, Canada c Black School of Business, Penn State Erie, Erie, PA 16563, United States Received 10 September 2014; received in revised form 16 December 2014; accepted 5 January 2015 Abstract The Agile project management methodology has been widely used in recent years as a means to counter the dangers of traditional, front-end planning methods that often lead to downstream development pathologies. Although numerous authors have pointed to the advantages of Agile, with its emphasis on individuals and interactions over processes, customer collaboration over contracts and formal negotiations, and responsiveness over rigid planning, there are, to date, very few large-scale, empirical studies to support the contention that Agile methods can improve the likelihood of project success. Developed originally for software development, it is still predominantly an IT phenomenon. But due to its success it has now spread to non-IT projects. Using a data sample of 1002 projects across multiple industries and countries, we tested the effect of Agile use in organizations on two dimensions of project success: efciency and overall stakeholder satisfaction against organizational goals. We further examined the moderating effects of variables such as perceived quality of the vision/goals of the project, project complexity, and project team experience. Our ndings suggest that Agile methods do have a positive impact on both dimensions of project success. Further, the quality of the vision/goals is a marginally signicant moderator of this effect. Implications of these ndings and directions for future research are discussed. 2015 Elsevier Ltd. APM and IPMA. All rights reserved. Keywords: Success; Agile; Methodology; Efciency 1. Introduction Projects continue to proliferate in society today, in both the public and private sectors of the economy. Investments in projects number in the trillions of dollars annually. Just as ubiquitous as these projects, unfortunately, are their significant failure rates. The CHAOS reports have identified the current state of project success rates across organizations, noting that in spite of much higher visibility and importance placed on project performance, failure rates have remained high and relatively stable across over a decade of research (The Standish Group, 2011). Further, specific examples of project failures shed light on E-mail addresses: pedro@serrador.net (P. Serrador), Jkp4@psu.edu (J.K. Pinto). the impact they have on organizations. Consider, for example, the following: Joe Harley, then-CIO at the Department of Work and Pensions for the UK government, stated that only 30% of technology-based projects and programs are a success at a time when taxes are funding an annual budget of 14 billion (about $22 billion USD) on public sector IT, equivalent to building 7000 new primary schools or 75 hospitals a year (Ritter, 2007). \"Motorola's multibillion-dollar Iridium project ... could be considered a success on the basis it was 'on time' and 'on budget' from an engineering point of view, but was a catastrophic commercial failure because it did not adjust to what was being learned about the changing business environment.\" (Collyer et al., 2010, p. 358). The project http://dx.doi.org/10.1016/j.ijproman.2015.01.006 0263-7863/00 / 2015 Elsevier Ltd. APM and IPMA. All rights reserved. Please cite this article as: P. Serrador, J.K. Pinto, 2015. Does Agile work? A quantitative analysis of agile project success, Int. J. Proj. Manag. http://dx.doi.org/ 10.1016/j.ijproman.2015.01.006 2 P. Serrador, J.K. Pinto / International Journal of Project Management xx (2015) xxx-xxx team and management at Motorola failed to see that during the course of the project, quickly expanding cell phone networks would undercut Iridium's satellite phone business model. It is with this setting in mind that researchers and practitioners began seeking alternative methods for project implementation, recognizing that traditional models for planning and execution may not be optimal or tuned for the specific challenges that projects face. Indeed, it is due to these challenges that \"light weight\" project management techniques such as Agile have been gaining popularity since first developed (Dyb and Dingsyr, 2008). Part of the ethos of Agile methods is that less initial planning is better and an evolutionary process is more efficient (Dyb and Dingsyr, 2008). Agile methodologies contrast with traditional project management approaches (such as waterfall) by emphasizing continuous design, flexible scope, freezing design features as late as possible, embracing uncertainty and customer interaction, and a modified project team organization. Further, Agile is described as iterative and incremental, seeking to avoid the standard approaches that emphasize early design and specification freeze, a fixed project scope, and low customer interaction. These more traditional project development approaches pursued a goal of logical sequencing that required deliverables to be set in advance and project development evaluated based on performance at a series of capabilities gated reviews. Unfortunately, evidence continues to accumulate suggesting that a rigid development process can result in significant downstream pathologies, including excessive rework, lack of flexibility, customer dissatisfaction, and the potential for a project to be fully developed, only to discover that technological advances have eclipsed the need for it. So, for example, to revisit the post-mortem analysis of Motorola's Iridium project, it became clear that in dynamic environments, projects need to cope with changes in technology during the course of their development both for technology and other projects. If assumptions fail, unsuccessful projects can often result. \"While useful as a guide, excessive detail in the early stages of a project may be problematic and misleading in a dynamic environment\" (Collyer et al., 2010, p. 109). Though Agile methods are continuing to gain in popularity and are spreading beyond their original birthplace among software development projects (Dyb and Dingsyr, 2008), little research has been done as to whether Agile projects truly are more successful. To date, the majority of research examining the methodology's usefulness has been anecdotal, based on small-sample case studies, or research limited by sample size, industry or geography. Further research in this area will help inform both practitioners and researchers to the value of agile methods. The purpose of this paper is to investigate, through a large-scale quantitative study, the evidence that Agile methods work better than traditional approaches for achieving project success. As we have noted, Agile has become a widely used and generally-accepted approach for planning and executing projects in IT settings. There is a wealth of anecdotal and case-study information pointing to the utility of the Agile process; however, to date, what has been lacking are more comprehensive quantitative studies of projects in Agile settings, testing the efficacy of the Agile philosophy as it directly relates to project success. This paper reports on the results of a recently completed study of Agile projects and their success rate. We sought to investigate the efficacy of Agile on different dimensions of project success, across multiple industries, in order to identify the degree to which Agile can be directly linked to project success, its viability across multiple project environments, and the potential for intervening (moderator) variables to affect this relationship. 2. Literature review 2.1. Agile/iterative methods As early as 1958, Koontz noted that \"no effective manager makes a plan and then proceeds to put it into effect regardless of what events occur\" (Koontz, 1958, p. 54). Deviations commonly found in the management of projects typify this perspective. After Hllgren and Maaninen-Olsson. (2005), we define \"deviation\" as \"a situation, regardless of consequence - positive or negative, large or small - that deviates from any plan in the project.\" (p. 18). They further note the inevitability of deviation in project plans, suggesting the solution lies not in more sophisticated initial plans but in methodologies that can facilitate actions to resolve deviations. In the IT project environment, this need for improving the planning process has increasingly led companies away from the traditional, front-end planning process to one that revolves around multiple iterations through the development cycle. Iterative methodologies, such as rolling wave, have been in use for years and can be thought of as predecessors to Agile methods. As part of their rationale for the use of rolling wave planning techniques, Turner and Cochrane (1993) noted that \"frozen objectives become part of the definition of the quality of the project, and project managers are said to be successful if they deliver them on time and within budget, regardless of whether or not the product is useful or beneficial to the owners and users.\" (p. 94) This highlights the benefits of iterative methods, which formalize replanning of a project during execution. For example, in his review of software development methodology, Fitzgerald (1996) also reported that 50% of design activities occurred in phases other than design. Thus, the critical issue confronting managers lies in the mismatch between the desire for early specification freeze and fixed plans with the concomitant need to maintain sufficient flexibility to modify and alter project plans to address critical business needs. As we noted, the Agile movement was intended to address some of these challenges. In 2001, the \"Agile Manifesto\" was written by practitioners who proposed many of the Agile development methods. The manifesto states that Agile development should focus on four core values (Dyb and Dingsyr, 2008; www.agilemanifesto.org): Individuals and interactions over processes and tools. Working software over comprehensive documentation. Please cite this article as: P. Serrador, J.K. Pinto, 2015. Does Agile work? A quantitative analysis of agile project success, Int. J. Proj. Manag. http://dx.doi.org/ 10.1016/j.ijproman.2015.01.006 P. Serrador, J.K. Pinto / International Journal of Project Management xx (2015) xxx-xxx Customer collaboration over contract negotiation. Responding to change over following a plan Agile methods are designed to use a minimum of documentation in order to facilitate flexibility and responsiveness to changing conditions, implying that less planning and more flexibility is used in agile projects than in traditional project management (See Table 1). Agile methods have become more and more common in technology projects since their development (Lindvall et al., 2002) because they directly address the challenges so often confronted in dealing with dynamic projects in changing environments. For example, in interviews with 31 project managers from 10 varied industries (Collyer et al., 2010), found that traditional methodologies had difficulties in dynamic environments due to three major types of changes that frequently occurred in projects: 1) goals; 2) materials, resources, tools, and techniques; and 3) relationships with other related projects, services, or products. In a study of software development projects, Boehm discussed similar sorts of challenges that development teams routinely face (Boehm, 1996). A too-detailed requirements document can have the following problems: 1) specifications that do not describe a deliverable as well as the prototype, 2) early specification of requirements results in gold plating (adding more features than required) because there will be no further opportunities to add/ change functionality, and 3) solutions focus on a specific point in time although the requirements or environment are likely to change. Boehm (1996) also reported that by using a spiral model (a phase approach similar to rolling wave or agile), which included a planning phase and execution phase in each spiral, project teams experienced \"a cost improvement from $140 to $57 per delivered line of code and a quality improvement from more than 3 to 0.35 errors per thousand delivered lines of code.\" It is important to note that Agile does not abandon front-end planning as part of the project development methodology. Coram and Bohner (2005), for example, point out that Agile methods do require upfront planning. Significant communication and working with the customer is needed to provide project requirements for the first release. Indeed, the critical point is to recognize that, in many ways, more planning is performed in 3 Agile environments, though the planning is spread across the entire development cycle, rather than occurring in an up-front, one-off manner. To elaborate this point, in reviewing Agile methods and comparing them to traditional methodologies, Boehm (2002) notes that when projects have excessively specified plans: \"Such plans also provide a source of major contention, rework, and delay at high-change levels.\" (p. 65) A balance between traditional methods and Agile methods is usually appropriate for project planning. Certain factors, such as the size of the project, safety requirements and known future requirements, call for upfront planning even in Agile projects, whereas turbulent, high-change environments call for less upfront planning and a greater use of Agile methods. Boehm (2002) suggests there is a \"sweet spot,\" which is dependent on project characteristics where the effort expended in initial planning pays off in project success. Too much or too detailed planning can result in wasted effort and too much plan rework, whereas not enough initial planning can result in project failure. In analyzing 1386 projects, Serrador and Turner (2013) found an \"inverted U\" relationship between planning and project success; in terms of the effort (time) taken to plan comprehensively. That is, they found that too much effort and time spent planning can have just as negative impact on project success just as can as too little. Agile methods also depend upon early and continuous customer involvement, both in establishing goals for the project and providing feedback to progressive prototypes as the project moves through its life cycle. Thus, the iterative nature of Agile allows for frequent stakeholder interaction, adjustments made on the fly, and re-scoping project requirements in light of new information or customer requests. In a study of the impact of Scrum (an Agile methodology) on overtime and customer satisfaction, Mann and Maurer (2005) found that customers believed that the daily meetings kept them up to date and helped to \"reduce the confusion about what should be developed\" (p. 9). Dyb and Dingsyr (2008) reported that the planning game activity, which is one of the techniques used in Agile methodologies, was found to have a positive impact both in the company and with customers because it gave both groups insight into the development process. Koskela and Abrahamsson (2004) Table 1 Main differences between traditional development and Agile development after Dyb and Dingsyr (2008). Traditional development Agile development Fundamental assumption Systems are fully specifiable, predictable, and are built through meticulous and extensive planning Management style Knowledge management Communication Development model Desired organizational form/structure Command and control Explicit Formal Life-cycle model Mechanistic (bureaucratic with high formalization), aimed at large organizations Quality control Heavy planning and strict control. Late, heavy testing High-quality adaptive software is developed by small teams using the principles of continuous design improvement and testing based on rapid feedback and Change Leadership and collaboration Tacit Informal The evolutionary-delivery model Organic (flexible and participative encouraging cooperative social action), aimed at small and medium sized Organizations Continuous control of requirements, design and solutions. Continuous testing Please cite this article as: P. Serrador, J.K. Pinto, 2015. Does Agile work? A quantitative analysis of agile project success, Int. J. Proj. Manag. http://dx.doi.org/ 10.1016/j.ijproman.2015.01.006 4 P. Serrador, J.K. Pinto / International Journal of Project Management xx (2015) xxx-xxx analyzed the role of the customer in an XP project (or extreme programming, which is an Agile methodology for software development based on scenario-based requirements, testing happening as early as possible and pair programming involving two programmers at one computer). They found that most of the time was spent on participating in planning game sessions and acceptance testing, followed by retrospective sessions at the end of release cycles. This highlights the importance of a planning phase in even Agile projects which strive to reduce that amount of formal process that is used. In fact, planning took up 42.8% of customer's effort. Interestingly, although Agile project management methods are becoming increasingly popular in dynamic settings, to date, much of the analysis of their effectiveness has been anecdotal or based on small sample sizes. Further, the results of these studies have led to mixed support for the superiority of Agile methods. For example, in interviews with 48 respondents from eight companies, Magazinius and Feldt (2011) examined the variation between Agile and non-agile companies; that is, companies that have or have not adopted Agile methodologies. They reported that the success in meeting time and budget goals and the causes of failures was not significantly different between the two methodologies. They also suggest that although estimation techniques have improved over time, other factors such as the overly aggressive push to reduce costs are at work in distorting estimates. This can result in adverse outcomes such as incorrect projects being selected, overruns costing more than the original rejected estimate or projects that might have been more beneficial being passed over. 2.2. Project success As Pinto and Slevin (1988) observed a number of years ago, \"There are few topics in the field of project management that are so frequently discussed and yet so rarely agreed upon as the notion of project success.\" (p. 67) Traditional measurements of project success focused on producing a project of sufficient quality (functionality), while meeting the dual constraints of time and budget goals of a project: the so-called triple constraint (Atkinson, 1999; Kerzner, 2003). However, project success is often defined in a broader way. Munns and Bjeirmi (1996) noted that much literature to that point considered \"projects end when they are delivered to the customer. That is the point at which project management ends. They do not consider the wider criteria which will affect the project once in use.\" (p. 83). Jugdev and Mller (2005) reviewed the project success literature over the past 40 years and found that a more holistic approach to measuring success was becoming more in evidence. Researchers were increasingly measuring success by the impacts on the organization rather than success at only meeting the triple constraint. Thomas et al. (2008) state that measuring project success is not straightforward \"Examples abound where the original objectives of the project are not met, but the client was highly satisfied. There are other examples where the initial project objectives were met, but the client was quite unhappy with the results.\" (p. 106). Shenhar et al. (1997) note that of the three traditional dimensions of project efficiency: time, budget and scope, scope can have the largest role in determining project success.1 Thus, not only is scope an aspect of project efficiency, but it also has an impact on the customer and their satisfaction. Mller and Turner (2007) have defined ten dimensions of project success as part of their series of studies of project manager competencies and project success. However Dvir et al. (2003) also found that \"all four success-measures (Meeting planning goals; End-user benefits; Contractor benefits; and Overall project success) are highly inter-correlated, implying that projects perceived to be successful are successful for all their stakeholders.\" (p. 94). This is stated in a different way by (Prabhakar, 2008, p. 7) \"There is also a general agreement that although schedule and budget performance alone are considered inadequate as measures of project success, they are still important components of the overall construct. Quality is intertwined with issues of technical performance, specifications, and achievement of functional objectives and it is achievement against these criteria that will be most subject to variation in perception by multiple project stakeholders.\" A similar point was reiterated by Kloppenborg et al. (2009) who stated that all measures of project success contain the traditional success factors of time, cost and performance. Serrador and Turner found a significant correlation between project efficiency and overall project success, arguing that efficiency should not be the final measure of success but neither can it be ignored (Serrador and Turner, 2015). Current terminology continues to distinguish between project efficiency, stakeholder performance, and project success (Mller and Turner, 2007; Pinto and Slevin, 1988; Shenhar and Dvir, 1997). Therefore we will define project status in such a manner as to adopt these dimensions: Project efficiency meeting cost, time and scope goals Stakeholder success satisfying the expectations of project stakeholders who are the best judges of overall success 2.3. Moderator variables Although it is our contention that Agile methods have a direct impact on project success, as evaluated by \"efficiency,\" stakeholder satisfaction, and the success of meeting wider business goals, this relationship is also subject to some intervening, moderator impacts that must be considered as part of our study. For example, a potential moderator of our hypothesized link between Agile methods and project success is the applicability and perceived quality of the project as assessed against larger organizational goals. In other words, as a project is developed for implementation, a critical question that is asked from a portfolio level is whether or not the project is in line with and supportive of larger organizational goals, often as demonstrated by the organization's larger portfolio of projects (Martinsuo, 2013). Thus, projects that are perceived to be of higher quality or have higher congruence with firms' 1 As evidence, witness the example of the Sydney Opera House; a disaster as measured by budget and schedule overruns, yet a classic from a scope perspective. Please cite this article as: P. Serrador, J.K. Pinto, 2015. Does Agile work? A quantitative analysis of agile project success, Int. J. Proj. Manag. http://dx.doi.org/ 10.1016/j.ijproman.2015.01.006 P. Serrador, J.K. Pinto / International Journal of Project Management xx (2015) xxx-xxx strategic goals or project portfolio will be better supported and more successful. A second moderator is the perceived degree of project complexity. Complexity is defined in various ways but typically, it contains the following elements; complex projects are understood to be a function of variation - the number of varied, interrelated elements, tasks or specialists - and complicated, involved, or intricate work (Baccarini, 1996; Miller and Hobbs, 2005). Williams (1999) cited Jones and Deckro (1993) in broadening Baccarini's original two-element model to include, in addition to variety and interdependency, the construct of uncertainty, or \"the instability of assumptions upon which the tasks are based\" (p. 27). Thus, research has suggested that the degree of project complexity can have an important impact on the manner in which projects are conceptualized and managed (Ahern et al., 2014), the forms of governance models that are most appropriate (Mller, 2009), and the means by which we can argue that specific managerial actions have predictable results on project outcomes (Williams, 1999). Finally, a third proposed moderator of the \"Agile project success\" relationship is the experience level of the project team. The literature has long recognized that project team members with greater experience and background in project-based work are typically more adept at completing their assignments, working together collaboratively, and performing tasks efficiently (Pinto et al., 1993). For our study, we sought to assess whether or not project team member experience was a significant moderator of the degree to which Agile methods could improve project success. As a result, our proposed research model is shown in Fig. 1, highlighting not only the tested relationship between Agile methods and project success, but also the potential impact of various moderators on this direct effect (see Fig. 1). 5 Our a priori contention was that Agile projects are more successful than those employing traditional project management methods. As we noted, to date, there have been few large, field research studies that have examined the impact of Agile methods on project success. By analyzing of project dataset of nearly 1400 projects, we completed the largest analysis of Agile processes and success to date. 3. Research methods 3.1. Procedure Data was collected from practitioners who are members of PMI or members of LinkedIn project management groups. Invitations to fill out a questionnaire (an on-line questionnaire using surveymonkey.com) were posted on PMI communities of practice (COPs) as well as a number LinkedIn groups. A notice was also included in some groups' mailings. We sought to gather a large data set over as wide as possible a range of different types of projects (IT, new product development, process improvement, and so forth). Identifying the overall population sample pool was not possible. Though the membership numbers for the LinkedIn groups are available (typically in the 1000 s) and membership numbers in the PMI CoPs are also available (membership up to the 10,000 s), memberships in each of these groups are not mutually exclusive. There is also no way to know how many members read group postings. Respondents were asked to think of projects they had been involved with and select two: one \"more successful\" one and one that they defined as \"less successful.\" The survey was targeted at project managers but was not restricted to people who managed the projects. A total of 865 people started the survey with 859 completing at least the first portion of it which Fig. 1. Research model. Please cite this article as: P. Serrador, J.K. Pinto, 2015. Does Agile work? A quantitative analysis of agile project success, Int. J. Proj. Manag. http://dx.doi.org/ 10.1016/j.ijproman.2015.01.006 6 P. Serrador, J.K. Pinto / International Journal of Project Management xx (2015) xxx-xxx requested information on one successful project. However, not all participants entered data for two projects; therefore the total number of projects was 1539. After removal of outliers and bad data, the usable total available for study was 1386 projects. These projects were reviewed for normality over the success factors and were found to have a normal distribution. 3.2. Participants The largest percentage of respondents were from the USA (36%, or 499), but there was broad participation in the survey, with several countries having 10 or more respondents, including India (6.9%, or 96), Canada (6.7%, or 93), Australia (2.2%, or 31), Spain (1.7%, or 24), Brazil (1.3%, or 18), Singapore (1.3%, or 18) and Germany (1%, or 14). Overall, there was a good global representation with respondents from more than sixty countries. Participants reported filling a variety of roles on projects: Project Manager (304), Senior Project Manager (141), Program Manager (72), Project Coordinator (66), Project Team Member (58), Senior Manager (36), Senior Program/Portfolio Manager (22) and C Level Management (14). 146 chose not to answer. Of those who responded to the demographic questions, 85% had more than 5 years' experience. 3.3. Approach In general, the simplest relationships were examined first and then analysis continued using progressively more involved techniques. The typical progression is to subgroup analysis to understand if there is a relationship followed by linear regression to see if there is a dependent relationship. This is followed by investigating the impact of moderators, using Moderated Hierarchical Regression Analysis (MHRA). For the MHRA analysis, we were trying to discover the underlying relationship between dependent and independent variables and understand how it is impacted by moderator variables (Sharma et al., 1981). MHRA analysis enables us to explore these relationships in more detail. groups, LinkedIn and personal contacts ensured there were no convenience sample issues. Therefore monosource bias was assumed not to be an issue for this research. Surveys questions, in general used a 5 or 7 point Likert-like numeric scale (Cooper and Schindler, 2008). Pure Likert scales were not used as there were several questions where numerical responses were appropriate. The varying scale was partially due to following the scales from the existing literature, using 7 point scales to allow optimum ordinal value for numeric ranges and 5 point scales for subjective ratings. Since a variety of scales were used this ensured that item context effects as per Podsakoff et al. (2003) were not an issue. For success, three questions measured project efficiency and four measured satisfaction of various key project stakeholders. Finally, an assessment of overall project success was solicited. All scales employed either a 5-point or 7-point Likert scale. Sample questions for efficiency included, \"How successful was the project in meeting project budget goals?\" and \"How successful was the project in meeting scope and requirements goals?\" Examples of questions for overall project success, included \"How do you rate the client's satisfaction with the project's results?\" and \"How do you rate the end users' satisfaction with the project's results?\" Overall project success used questions such as: \"How do you rate the overall success of the project?\" In addition, a number of questions related to the proposed moderators were included in the survey, based on factors which could impact success. As noted previously, these moderators included level of stakeholder engagement level, Vision/Goals quality, project complexity and experience level of team. For assessing the use of Agile methods, we asked respondents to indicate the amount of effort spent planning during the planning and execution phases. To facilitate the analysis of the planning effort as a percentage of the total project development time, the following two indexes were calculated: Planning Effort Index Total effort expended on the planning phase in person days 3.4. Measures This study used a self-reporting (subjective) assessment of project success and use of Agile methods. Therefore, this paper will largely be concerned with perceived project success as reported by participants. To measure this factor, questions in the survey were largely based on a combination of the success dimensions defined in the literature, (Crawford et al., 2004; Mller and Turner, 2007; Shenhar et al., 2001). Monosource bias and other response biases can occur in self-rated performance measures as discussed by (Podsakoff et al., 2003) and (Conway and Lance, 2010). For that and privacy reasons, anonymity was allowed in the survey and company names were not captured. By targeting project managers, we intended to receive information from the individual who would have the best overall view of the project. To avoid social desirability issues related to project success, respondents were asked to provide data for both a more successful and less successful project. Finally, the use of PMI =Total project Effort in person days Agile Planning Effort Index Total effort expended on planning after the planning phase =The total project effort in person days The quantitative data on planning phase effort and budget did not lend itself to multiple measurement tools or questions. The project data collected can only be asked in one way and should be hard data (hours and dollars spent) rather than the opinions of respondents. The amount of planning done in the planning phase was considered a key data set for the survey. The planning effort index was used as the key measure of this information. Outliers with values over two standard deviations were identified, as recommended by Field (2009). Given the mean was .18 and 2*SD was .40, an approximate upper value of .60 was chosen as the cut-off for outliers. This exclusion was also based on the goal of this research to focus on projects that Please cite this article as: P. Serrador, J.K. Pinto, 2015. Does Agile work? A quantitative analysis of agile project success, Int. J. Proj. Manag. http://dx.doi.org/ 10.1016/j.ijproman.2015.01.006 P. Serrador, J.K. Pinto / International Journal of Project Management xx (2015) xxx-xxx contained a substantial execution phase and that were not canceled prior to completion. For consistency and simplicity, similar exclusion rules were used for Agile analyses: cases with their respective indexes N .6 were excluded. 0 was considered valid in this case as non-Agile project may not do any planning during execution. The total number of projects with valid Agile effort indexes was 1002. There was a consistent problem in survey responses to the Agile effort or budget questions. The question requested that all the planning throughout the Agile project including the initial planning phase be reported as one number. However, some respondents did not include the upfront planning phase. This was clear in cases where the upfront planning effort or budget was given as greater than the total planning over the entire project which should have also included the planning phase planning. To address this issue, the data was divided into two broad categories: cases where the respondent had clearly misunderstood the question and all others. Cases where the question was clearly misunderstood were easy to identify as the upfront planning effort was greater than the total planning effort. In those cases, we could find two values: planning before project execution and planning during project execution. There were adequate data in this subset at 412 to continue the analysis without additional data. The issues with this unclear question only impacted the questions on planning effort amount during execution. They did not impact any other parts of the survey. This group of projects will be identified as Group 1. The remaining 590 cases were also analyzed as a secondary confirmation of results. For this analysis, the second group was transformed by subtracting the planning phase planning effort from the total planning effort given in the second question. This transformation was only completed for projects where this would result in execution planning effort being N 20% of planning phase effort since Agile projects are assumed to have a material amount of planning in the execution phase (Boehm, 2002; Coram and Bohner, 2005; Mann and Maurer, 2005; Smits, 2006). This data was used for secondary analysis and defined as Group 2. A one-way ANOVA analysis was completed on the dependent variables to test variance between the two datasets. The results are shown in Table 2 and confirm homogeneity. To facilitate further analysis, the success measures were grouped in two success factors. These were: Efficiency factor = mean of the following three responses: 1. How did the project do in meeting project budget goals? 2. How did the project do in meeting project time goals? 3. How did the project do in meeting project scope and requirements goals? Table 2 One-way ANOVA analysis between group 1 and group 2. Stakeholder success factor F p Efficiency factor 0.000 0.946 0.113 0.737 7 Table 3 Confirmatory factor analysis of success measures. Factor loadings (Varimax normalized) Extraction: principal components (marked loadings are N.700000) Factor 1 Project sponsors and stakeholders success rating Project budget goals Project time goals Scope and requirements goals Project team's satisfaction Client's satisfaction End users' satisfaction Factor 2 0.893* 0.162 0.288 0.522 0.836* 0.916* 0.897* 0.267 0.877* 0.845* 0.524 0.299 0.256 0.202 Stakeholder success factor = mean of the following four responses: 1. How did the project sponsors and stakeholders rate the success of the project? 2. How do you rate the project team's satisfaction with the project? 3. How do you rate the client's satisfaction with the project's results? 4. How do you rate the end users' satisfaction with the project's results? Though these factors were based on the findings of previous researchers, we also conducted a confirmatory factor analysis to verify factor stability (see Table 3). Factor 1 clearly corresponded to stakeholder satisfaction; factor 2 corresponded more specifically to project efficiency metrics (\"project management\" success, as suggested by Cooke-Davies (2002)). This factor analysis confirms our selection of factors with the exception of the scope question. This question could fit with either the efficiency or stakeholder satisfaction factor. Our results are in keeping with Shenhar and Dvir (1997) who stated that scope was the most important element of the triple constraint for overall success. Since the result for scope is somewhat higher for factor 2 (efficiency), we maintained its use as part of the efficiency factor. A scale reliability analysis (using Cronbach alpha) was performed on the success factors and results were a .945 for the stakeholder satisfaction factor and .77 for the efficiency factor (see Tables 1 and 2, Appendix A), which is considered to be well into the acceptable range for research purposes, (Nunnally, 1978). Table 4 Frequency table for methodology type. Percentage agile/iterative Count Cumulative-count Percent Cumulative-percent 80-100% 60-79% 40-59% 20-39% 1-19% 0% Missing 80 152 347 162 194 451 0 80 232 579 741 935 1386 1386 5.772 10.967 25.036 11.688 13.997 32.540 0.000 5.772 16.739 41.775 53.463 67.460 100.000 100.000 Please cite this article as: P. Serrador, J.K. Pinto, 2015. Does Agile work? A quantitative analysis of agile project success, Int. J. Proj. Manag. http://dx.doi.org/ 10.1016/j.ijproman.2015.01.006 8 P. Serrador, J.K. Pinto / International Journal of Project Management xx (2015) xxx-xxx Table 5 Methodology type means with ANOVA analysis: how much of the project was done using agile or iterative techniques. Percentage Means and ANOVA agile/iterative Upfront Agile Efficiency Stakeholder Valid N planning planning factor success effort index effort index factor 80-100% 60-79% 40-59% 20-39% 1-19% 0% All Groups F p(F) 0.161 0.147 0.164 0.135 0.150 0.154 0.153 1.492 0.173 0.149 0.138 0.132 0.101 0.091 0.048 0.105 18.370 0.000 4.821 4.664 4.793 4.638 4.460 4.582 4.647 1.93 0.087 3.638 3.567 3.544 3.414 3.179 3.208 3.376 7.73 0.000 80 152 347 162 194 451 1386 These two factors, along with the single-item question put to the respondents to rate the overall success of the project, were used for the analysis. 4. Results Our study demonstrated that Agile and iterative methods had been widely adopted for the purpose of managing projects. We asked respondents to indicate what percentage of their projects included some elements of Agile methods, based on a detailed description of these activities. Thus, from Table 4, we see that nearly 6% of total projects were completely or nearly completely Agile. Further, more than 65% of the original 1386 projects reported having some Agile or iterative component (see Table 4), determined by the percentage of Agile methods they indicated occurred in their project. We found that the greater the Agile/iterative approach reported, the higher the reported project success. This finding can be confirmed through a one-way analysis of variance (ANOVA) procedure. As Table 5 demonstrates, for the two assessments of project success: efficiency and stakeholder satisfaction, for project success the ANOVA demonstrated strongly significant differences (p b .01) in project success by degree of Agile methodologies employed in those projects. In the case of the Efficiency Factor, a marginally significant difference (p b .10) was also found. As a result, we can see from the p value for stakeholder success factor that there is clearly a relationship between Agile use and the success factors. The amount of planning during execution is also significantly related to methodology as expected. Table 6 Correlation analysis between methodology type and success factors. Correlations N = 1386 Overall project success rating Methodology type Efficiency factor Stakeholder success factor -.1721 p = .000 -.0617 p = .022 -.1570 p = .000 *1 = 80-100% agile and 6 = fully waterfall. Table 7 Logit Regression analysis of methodology type vs. project success rating. Project success rating test of all effects Ordinal multinomial link function: LOGIT Degr. of freedom Wald-Stat. p 4 5 Intercept Methodology type 1422.172 43.398 0.000 0.000 In addition, the correlation between the level of Agile development and success is small but statistically significant as shown in Table 6. The negative correlation is found because the low value of 1 was coded to fully Agile and the high value was assigned to fully waterfall (that is, a \"cascading,\" fully front-loaded planning approach). What is indicated is that Agile methodologies are correlated with higher reported success for each category: overall project success, efficiency, and stakeholder success. It is interesting to note that projects with a high Agile percentage report mean upfront planning amounts similar to traditional projects and for project efficiency, there is no significant difference in the relationship between Agile use and upfront planning as shown by the p for upfront planning index. As noted by Dyb and Dingsyr (2008) and Smits (2006), if substantial planning is done during execution then Agile projects appear to do more planning overall than traditional projects. Regression analysis allows us to further understand the relationship between project success and Agile methods. Some authors (Norman, 2010) have challenged the common understanding that regression models do not give accurate results with ordinal independent variables (range variables as opposed to continuous data). In fact, logit regression allows for testing of the relationship between ordinal methodology type and ordinal project success rating (Field, 2009). Because methodology type is ordinal, we completed a logit regression against another ordinal variable, the respondent's overall success rating (see Table 7). We can see that the results demonstrate strong statistical significance; there is a predictive relationship between Agile use and reported success, suggesting that more investigation is warranted. If we now complete a standard regression of Agile methods on project success, as suggested by Norman (2010), we again find a highly significant predictive relationship, albeit with a relatively low R2 of .03 (See Table 8). Although the regression analysis shows strongly significant results, there is an underlying problem with the use of single-item measures as they can call into question the reliability of a construct scale. To avoid the issue of regression against a single item measure, a new index was created so that we can complete regression against two sets of continuous Table 8 Standard linear regression analysis of methodology type vs. project success rating. Regression project success rating R = .172 R2 = .030 Adjusted R2 = .029 Beta Intercept Methodology type B Std.Err. of B t(1384) p-level 0.172 3.846 0.124 0.085 0.019 45.3999 6.499 0.000 0.000 Please cite this article as: P. Serrador, J.K. Pinto, 2015. Does Agile work? A quantitative analysis of agile project success, Int. J. Proj. Manag. http://dx.doi.org/ 10.1016/j.ijproman.2015.01.006 P. Serrador, J.K. Pinto / International Journal of Project Management xx (2015) xxx-xxx 9 Table 9 Standard linear regression analysis of combined Agile measure vs. project success rating. Table 11 MHRA analysis for quality of vision/goals as moderator in the Agile measure versus success factor relationship. Regression for project success rating R = .138 R2 = .019 Adjusted R2 = .018 N = 1002 Variables entered Beta Intercept Combined Agile measure B Std.Err. of B tn410) p-level 0.138 0.031 3.119 0.857 0.072 0.194 0.0000 0.00001 variables. This index combined the results of the question on the degree of Agile in the project with the measure of how much planning was done in the execution phase. Replanning during execution is a feature of agile methodologies (Dyb and Dingsyr, 2008; Smits, 2006). The following measure was defined. Combined Agile measure = mean of the following two responses as a summated scale of normalized values: 1. Methodology type 2. Agile planning index Where Agile planning index is defined as follows: Agile Planning Effort Index Total effort expended on planning after the planning phase =The total project effort in person days Both items are taken as a measure of the \"Agileness\" of the project. The first is based on respondent assessment of how much Agile process is used in the project; the second on how much planning is completed during execution, which is a feature of Agile projects (Dyb and Dingsyr, 2008; Smits, 2006). We ran a regression analysis for our combined Agile measure on the project success dependent variable (see Table 9). Although significance levels (p-value) are strong, the overall model R2 values are relatively low at .019. It is likely the case that our large sample led to the highly significant results. However, all analyses Step 1 Step 2 Step 3 Main effects Combined Agile measure .875 .572 -.253 Moderators Quality of vision/goals -.513 643 Interaction terms Quality of vision/goals Combined .401 + Agile measure F for regression 19.492 89.838 60.973 0.019 0.151 0.152 + R2 p b .10. p b .05. p b .01. p b .001. + suggest that the use of Agile methods is positively associated with improved success. Next, we examined the efficacy of using Agile methods by industry. It is possible to determine some interesting patterns emerging (See Table 10). Four industries showed statistical significance when regressing methodology type versus overall success measure: high technology, health care, professional services, and the category reported by participants as \"other.\" This finding confirms earlier work that has observed that Agile is more widespread in the high tech and IT fields and, in fact, Agile was originally designed for that type of environment (Dyb and Dingsyr, 2008). On the other hand, industries where there is less reported use of Agile methodologies - such as construction, manufacturing and retail - do not show a statistically significant relationship. Finally, we conducted a MHRA analysis on the data and found mixed support for the presence of moderator effects in the link between agile and project success. Interestingly, despite literature arguing that quality of project vision/goals, complexity, and team experience (see Fig. 1) are likely to moderate the direct effect of agile on project success, we found minimal support for these effects. Specifically, for the data, only a marginally Table 10 Comparison of means and regression results for Agile success by industry. Methodology type Construction Financial services Utilities Government Education Other High technology Telecommunications Manufacturing Health care Professional services Retail All groups Efficiency factor Stakeholder success factor Valid N Regression p value vs. success 4.739 4.781 4.261 3.647 3.800 4.245 4.526 4.343 4.810 4.500 4.273 4.313 4.432 4.536 4.621 4.435 4.304 4.867 4.484 4.690 5.105 4.437 5.042 4.530 4.563 4.617 3.652 3.267 3.413 3.169 3.150 3.193 3.427 3.707 3.304 3.521 3.318 3.234 3.356 23 73 23 34 10 53 57 35 42 24 22 16 412 0.184 0.747 0.584 0.336 0.084 0.0002 0.035 0.570 0.726 0.017 0.034 0.722 0.007 Indicates statistical significance. Please cite this article as: P. Serrador, J.K. Pinto, 2015. Does Agile work? A quantitative analysis of agile project success, Int. J. Proj. Manag. http://dx.doi.org/ 10.1016/j.ijproman.2015.01.006 10 P. Serrador, J.K. Pinto / International Journal of Project Management xx (2015) xxx-xxx Table 12 Comparison for R2 from MHRA analysis between combined Agile measure and success factors. N = 1002 2 R Stakeholder success factor Efficiency factor .152 p = .089 .096 p = y083 significant moderator effect was found in the quality of vision/ goals (see Table 11). Neither project complexity or team experience were significant moderators of the agile-success linkage. The p value for the final model was p = .08, which indicates a marginally significant moderator relationship and a final model R2 of .152. Results are similar for the efficiency factor, though it was less impacted by agile methods than stakeholder success (see Table 12). The use of Agile appears to have a greater impact on success factors than straight efficiency. This is largely in keeping with the stated goal of Agile methodologies. 5. Discussion and directions for future research For a number of years now, Agile has been touted as a methodology for project planning and execution that addresses many of the failings with the traditional, cascade planning process. Out of the frustration of multiple practitioners was born the Agile Manifesto and its call to dramatically reconsider the means by which successful projects are managed in chaotic settings. While the ideas that underpin the Agile philosophy are attractive and logical, to date, what has been lacking is empirical validation. Is an Agile-managed project more likely to succeed that one that relies on traditional approaches? This paper has explored the efficacy of the Agile method through a comprehensive and large-scale empirical analysis of projects being developed with varying levels of agile approaches and their subsequent likelihood of success. Our findings suggest that there is research support for the application of Agile methodology in managing projects. That is, we found that the level of Agile used in a project does have a statistically significant impact on all three dimensions of project success, as judged by efficiency, stakeholder satisfaction, and perception of overall project performance. Our findings offer limited support for current work by Budzier and Flyvbjerg (2013), who studied a data set of IT projects and found that Agile methods appear to improve project delivery times, though they found no evidence that they have positive impact on other success metrics. We further found evidence that the quality of the vision and goals for the project can serve as a significant moderator of the relationship between Agile methods and project success. On the other hand, two other proposed moderators (experience of the team and project complexity) were not found to moderate the relationship between Agile and project success. It is interesting to note that our original regression analysis showed statistical significance but low values for percentage of variance explained (R2). However, when we conducted a follow-on moderated hierarchical regression analysis (MRHA) and entered the moderator variables into the regression, the overall R2 rose significantly. The fact that neither project complexity nor experience of the project team significantly moderates the relationship between Agile and project success is an intriguing finding because it suggests that one of the benefits of introducing Agile methods is that they allow for superior success regardless of the use of \"seasoned\" staff. Further, Agile appears to work well regardless of the perceived complexity of the project. However, complexity was found to be a moderator for group 1 however with a low R2 = .03 for a model including it. For group 2 or for the full dataset, it was not found to be a moderator. It was close in the last case at near the p = .10 level but it was decided to exclude it from further analysis. Recall that we followed the conceptualization in defining complexity as a function of variety, interdependency, and uncertainty (Baccarini, 1996; Williams, 1999). The role of project complexity in this relationship is an area that warrants further study. Although our findings offered some intriguing perspectives on the use of Agile methods and their impact on project success, there were also some limitations to the study that need to be acknowledged. First, surveys can suffer from participant errors (Cooper and Schindler, 2001). These can include non-response errors where the participants fail to respond to particular questions and response error where the participant does not give an accurate response or gives an incomplete response. These errors were of particular concern relating to the quantitative data on planning phase and total project effort and budget. This information may not be readily available for all project managers to access. In manually reviewing the data, it was clear that some participants did not have access to this data as they selected \"zero effort\" as a value. In other cases, the project totals were entered using the same value as the planning phase totals or the planning phase was entered as an unrealistically high percentage of the total. Steps were taken during analysis to remove those entries. Many of the statistical relationships reported in this study, though highly significant, were of relatively small effect in terms of variance explained (R2). This raises the question of \"rigor vs. relevance\" in regards to some of the findings of the study; that is, though the results demonstrated statistical significance, the question to be asked is whether generalizable conclusions are possible when the actual predictive effect size for some of the relationships was .03. Future research (including MHRA analysis with additional moderators) could be undertaken to discover the extent of these relationships. For example, Serrador and Turner (2015) showed that the R2 for the planning effort impact on project success increased after MHRA. Likewise, MHRA has clarified the relationship in this research by showing R2 in the final model of .15. A limitation of this research is an examination of the impact of the dynamism of the environment on the relationship between agile and success. Future research should examine this relationship and whether agile works best in dynamic environments or works well in all environments. Future research could also focus on the prevalence of \"hybrid\" Agile methods where a mix of agile and traditional methods is used. That is, focusing specifically on cases where companies formally adopt a hybridized version of Agile could yield some interesting differences in planning approaches and project outcomes. Our research has found that this is, by far, Please cite this article as: P. Serrador, J.K. Pinto, 2015. Does Agile work? A quantitative analysis of agile project success, Int. J. Proj. Manag. http://dx.doi.org/ 10.1016/j.ijproman.2015.01.006 P. Serrador, J.K. Pinto / International Journal of Project Management xx (2015) xxx-xxx the majority of projects and this phenomenon should be further investigated. Further analysis of projects where more time is spent replanning during execution is warranted to understand if they are more or less successful. To illustrate, it is interesting to note that projects with a high Agile percentage report mean upfront planning amounts similar to traditional projects. If substantial planning is done during execution, as noted by Dyb and Dingsyr (2008), Coram and Bohner (2005) and Smits (2006), then Agile projects would appear to do more planning overall then traditional projects. The impact of planning done during execution is another area for research. Does the relationship between how planning is structured impact Agile success? Further research in this area is warranted. This paper reports on one of the first empirical studies of the efficacy of Agile methods for project success (and employing the largest dataset). Although Agile has been used for project planning for a number of years now, to date, the majority of research examining its usefulness has been anecdotal, single-case studies, or based on small sample sizes in single-organization or single-industry settings. To our knowledge, this is the first study that collected a large data set across multiple industries and national settings. The likelihood of project success, as measured by multiple perspectives (efficiency, stakeholder satisfaction) was shown to improve through the use of Agile. It was also found that the quality of goals and vision of a project are important to the success of projects as expected, but particularly for Agile projects. Further, though it has been adopted in multiple industries and across national borders, our findings suggest that it has achieved best success to date within certain settings; notably, high technology, healthcare, and professional service. All of which are heavy users of software and IT. It would be useful to revisit this field in a few years to determine the ongoing diffusion rate of Agile given its record of success. Appendix A Table 1 Cronbach alpha analysis of success measure. Cronbach alpha: .945 Standardized alpha: .944 Average inter-item corr.: . 815 Mean if Var. if StDv. if Itm-Totl Alpha if deleted deleted deleted Correl. deleted Sponsors success rating Project team's satisfaction Client's satisfaction End users' satisfaction 10.112 10.151 10.081 10.126 9.651 10.490 9.671 10.227 3.107 3.239 3.110 3.198 0.886 0.818 0.915 0.849 0.921 0.942 0.912 0.933 Table 2 Cronbach alpha analysis of efficiency measure. Cronbach alpha: .769 Standardized alpha: .767 Average inter-item corr.: .529 Mean if Var. if StDv. if Itm-Totl Alpha if deleted deleted deleted Correl. deleted Project time goals 9.214 Project budget goals 9.667 Scope and requirements 9.001 goals 8.484 7.477 9.733 2.913 2.734 3.120 0.605 0.690 0.521 0.686 0.584 0.774 11 References Ahern, T., Leavy, B., Byrne, P.J., 2014. Complex project management as complex problem solving: A distributed knowledge management perspective. Int. J. Proj. Manag 32, 1371-1381. Atkinson, R., 1999. Project management: cost, time and quality, two best guesses and a phenomenon, its time to accept other success criteria. Int. J. Proj. Manag. 17 (6), 337-342. Baccarini, D., 1996. The concept of project complexity a review. Int. J. Proj. Manag. 14 (4), 201-204. Boehm, B., 1996. Anchoring the software process. IEEE Softw. 13 (4), 73-82. Boehm, B., 2002. Get ready for agile methods, with care. Computer 35 (1), 64-69 (January). Budzier, A., Flyvbjerg, B., 2013. Making sense of the impact and importance of outliers in project management through the use of power laws. Paper presented at IRNOP (Oslo). Collyer, S., Warren, C., Hemsley, B., Stevens, C., 2010. Aim, fire, aim project planning styles in dynamic environments. Proj. Manag. J. 41 (4), 108-121. Conway, J., Lance, C., 2010. What reviewers should expect from authors regarding common method bias in organizational research. J. Bus. Psychol. 25 (3), 325-334. Cooke-Davies, T.J., 2002. The real success factors in projects. Int. J. Proj. Manag. 20 (3), 185-190. Cooper, D., Schindler, P., 2008. Business Research Methods. Irwin/McGrawHill, New York, NY. Coram, M., Bohner, S., 2005. The impact of agile methods on software project management. Proceedings of the 12th IEEE International Conference and Workshops on Engineering of Computer-Based Systems. IEEE Computer Society, Washington, DC, USA, pp. 363-370. Crawford, L.H., Hobbs, J.B., Turner, J.R., 2004. Project categorization systems and their use in organisations: an empirical study. Proceedings of PMI Research Conference (London). Dvir, D., Raz, T., Shenhar, A., 2003. An empirical analysis of the relationship between project planning and project success. Int. J. Proj. Manag. 21 (2), 89-95. Dyb, T., Dingsyr, T., 2008. Empirical studies of agile software development: a systematic review. Inf. Softw. Technol. 50 (9), 833-859. Field, A., 2009. Discovering Statistics Using SPSS. Sage Publishing, London. Fitzgerald, B., 1996. Formalized systems development methodologies: a critical perspective. Inf. Syst. J. 6 (1), 3-23. Hllgren, M., Maaninen-Olsson, E., 2005. Deviations, ambiguity and uncertainty in a project-intensive organization. Proj. Manag. J. 36 (3), 17-26. Jones, R., Deckro, R., 1993. The social psychology of project management conflict. Eur. J. Oper. Res. 64, 216-228. Jugdev, K., Mller, R., 2005. A retrospective look at our evolving understanding of project success. Proj. Manag. J. 36 (4), 19-31. Kerzner, H., 2003. Project Management: A Systems Approach To Planning, Scheduling, And Controlling. 8th ed. Wiley, New York. Kloppenborg, T.J., Manolis, C., Tesch, D., 2009. Successful project sponsor behaviors during project initiation: an empirical investigation. J. Manag. Issues 21 (1), 140-159. Koontz, H., 1958. A preliminary statement of principles of planning and control. J. Acad. Manag. 1 (1(1)), 45-61. Koskela, J., Abrahamsson, P., 2004. On-site customer in an XP project: empirical results from a case study. In: Dingsyr, T. (Ed.), Software Process Improvement. vol. 3281. Springer, Berlin/Heidelberg, pp. 1-11. Lindvall, M., Basili, V., Boehm, B., Costa, P., Dangle, K., Shull, F., Zelkowitz, M., 2002. Empirical findings in agile methods. In: Wells, D., Williams, L. (Eds.), Extreme Programming And Agile MethodsXP/Agile Universe 2002. 2418, pp. 81-92. Magazinius, A., Feldt, R., 2011. Confirming distortional behaviors in software cost estimation practice. Proceedings Of The 37th EUROMICRO Conference On Software Engineering And Advanced Applications (SEAA). IEEE, pp. 411-418. Mann, C., Maurer, F., 2005. A case study on the impact of scrum on overtime and customer satisfaction. Agile Conference, 2005. Proceedings, pp. 70-79 (july). Please cite this article as: P. Serrador, J.K. Pinto, 2015. Does Agile work? A quantitative analysis of agile project success, Int. J. Proj. Manag. http://dx.doi.org/ 10.1016/j.ijproman.2015.01.006 12 P. Serrador, J.K. Pinto / International Journal of Project Management xx (2015) xxx-xxx Martinsuo, M., 2013. Project portfolio management in practice and context. Int. J. Proj. Manag. 31, 794-803. Miller, R., Hobbs, B., 2005. Governance regimes for large complex projects. Proj. Manag. J. 36 (3), 42-50. Mller, R., 2009. Project Governance. Gower Publishing, Surrey, {UK}. Mller, R., Turner, R., 2007. The influence of project managers on project success criteria and project success by type of project. Eur. Manag. J. 25 (4), 298-309. Munns, A., Bjeirmi, B., 1996. The role of project management in achieving project success. Int. J. Proj. Manag. 14 (2), 81-87. Norman, G., 2010. Likert scales, levels of measurement and the \"laws\" of statistics. Adv. Health Sci. Educ. 15 (5), 625-632. Nunnally, J., 1978. Psychometric Theory. McGraw-Hill, New York. Pinto, J.K., Slevin, D.P., 1988. Project success: definitions and measurement techniques. Proj. Manag. J. 19 (1), 67-72. Pinto, M.B., Pinto, J.K., Prescott, J.E., 1993. Antecedents and consequences of project team cross-functional cooperation. Manag. Sci. 39, 1281-1297. Podsakoff, P., MacKenzie, S., Lee, J., Podsakoff, N., 2003. Common method biases in behavioral research: a critical review of the literature and recommended remedies. J. Appl. Psychol. 88 (5), 879-903. Prabhakar, G., 2008. What is project success: a literature review. Int. J. Bus. Manag. 3 (8), 3-10. Ritter, T., 2007. Public sector IT projects have only 30% success rate. CIO for Department for Work and Pensions (Retrieved from http://www. computerweekly.com/blogs/public-sector/2007/05/public-sector-it-projectshave.html). Serrador, P., Turner, J.R., 2013. The impact of the planning phase on project success. Paper Presented At IRNOP (Oslo, Norway). Serrador, P., Turner, J.R., 2015. The relationship between project success and project efficiency. Proj. Manag. J. 46 (1). Sharma, S., Durand, R., Gur-Arie, O., 1981. Identification and analysis of moderator variables. J. Mark. Res. 291-300. Shenhar, A.J., Dvir, B., 1997. Mapping the dimensions of project success. Proj. Manag. J. 28 (2), 5-9. Shenhar, A.J., Dvir, D., Levy, O., 1997. Mapping the Dimensions of Project Success. Proj. Manag. J. 28 (2), 5-13. Shenhar, A.J., Dvir, D., Levy, O., Maltz, A.C., 2001. Pr