Budgeting for Nonprofit Organizations

Although budgeting is just as important for nonprofit organizations as for for-profit companies, the approach taken toward budgeting can be very different. In this Discussion, you will consider the budgeting process and its role in nonprofit organizations.

To prepare for this Discussion,scanthe following references to familiarize yourself with the topic:

• How and why is a realistic budget relevant to nonprofit organizations? What are some of the potential consequences throughout the organization including employee decision making of an accurate budget?
• Use one of the articles listed above to aid in formulating your response, and find at least one additional reference from the Business Source Complete database in the Walden Library to support your position. Be detailed and clear, and apply the principles explored in this course as you write your response.
• Inaddition,conductresearchonapopularbudgetingsoftwaretool.Selectonetooltodescribeinmoredetail,includingwhatonlinereviewslistasitsgreatestfeatures.Nowinyourownopinion,commentonhowthistoolwouldworkforanonprofitorganization.Woulditmeettheuniquechallengesofthistypeoforganization?Whyorwhynot? Inyourresponse,prepareyourdiscussiondistinctlyandseparatelyfromthesalespitchyouwilllikelyencounteronvarioussoftwaresitesyouwillreview.
• Please put in bracket the resources referenced from the attached once. I will do the proper referencing. However, any other material should be properly referenced. APA standard, please. Just a concise discussion.

Planning and budgeting in non-profit organizations CMA MANAGEMENT 22 March 2009 By Michel Pich, CMA CMA MANAGEMENT 23 March 2009 Unless the process is supported by well-defined policies and guidelines, clear strategic goals and operating priorities, it can become quite chaotic and confusing. Although it's often seen as a cumbersome process by management, planning and budgeting is an essential activity for any organization. Most accountants dread this time of the year almost as much as the traditional year-end closing and audit marathon. It sets manager against manager, program against program, with everyone having to justify existing resource levels or requests for new funding. The alignment of strategic goals and budgets for public and non-profit organizations can take many twists and turns. In participation with l'Ecole Nationale d'administration Publique, the Secretariat in Public Fund Management (SPFM) has put together \"best practices\" for planning and budgeting in these types of organizations. These policies incorporate organizations' formal and informal practices and advocate a goal-driven approach to budgeting, which spans the planning, development, adoption, and execution phases of the budget process. The ISPFM's work emphasizes that budgeting must have a longrange perspective and not simply be an exercise in balancing annual revenue and expenditures. overseen by a complex system of local and national governance and involves volunteers to set the organization's strategic direction, approve budgets, and monitor performance. After reviewing documents and meeting with staff and volunteers, the CFO could not understand how the organization was able to combine all of its programs and projects into a budget document. There was no specific operating plan that linked the budget to the strategic goals, budget policies were almost nonexistent, guidelines dealt mostly with accounting matters, performance indicators were nowhere to be found, and the previous year's budget document read more like a statement of intent. The CFO realized that, although the board spent considerable time and efforts coming up with a multiyear strategic plan, this plan had little influence on the activities performed in the field. The Organization's programs had been in place for many years and continued year after year with minimal changes. Budgeting was essentially a cost review exercise with efforts spent finding new revenue to fund increasing expenses. It seemed that only a crisis could justify objective review and analysis of programs. Regardless of the challenge, the CFO put together budget guidelines based on more than 20 years of experience. He developed basic financial and budgeting policies \"on the run\" to provide for a more rational basis to assess ever increasing demands. These policies were focused primarily on setting expected operating outcomes, price and cost increases, and capital expenditure limits. As the budget process moved forward, it became obvious that extensive negotiations would be required to address the managers' many \"wants and needs.\" Despite the frequent communication with management, staff, and volunteers, the numbers that vastly exceeded the organization's funding capacity. Furthermore, there was minimal alignment of the budget requests and the organization's strategic goals. The CFO quickly realized that his most pressing challenge was to find a way to impose order in the CASE STUDY The Humanitarian Organization A chief financial officer (CFO) started with The Humanitarian Organization in the midst of the organization's planning and budgeting cycle. He had the immediate task of issuing budget guidelines to managers in order to start the process. Although he didn't know much about the organization and its programs, he spent considerable time gathering information on strategies, operating plans, investment projects, financial policies, etc., including a detailed review of the previous year's budget document. The Humanitarian Organization is a large aid organization that operates a multitude of programs. The organization depends on donors' contributions for its funding, but is also involved in many fee-for-service activities to help generate additional revenue. It is CMA MANAGEMENT 24 March 2009 process while being respectful of the humanitarian culture of the organization. After months of intense negotiations, the CFO thought that he had achieved an acceptable balanced budget with appropriate capital spending. The board's review meeting was fast approaching and a final budget document had to be completed and distributed. Unfortunately, despite the apparent consensus reached with managers, change requests that reflected latest economic conditions or additional programs' needs kept arriving. In order to meet the board's deadline, the CFO had to make last minute arbitrary decisions, some of which were not well accepted. Discussions with managers and volunteers continued right up to the morning of the Board of Directors meeting. The budget document was finalized within hours of the board meeting. It was a balanced budget with acceptable capital expenditures, but lacked specific operation's targets and had minimal alignment with the Organization's strategic goals. Although there was a separate operating plan presented to the board, it was generic and could not be linked to the financial numbers. The Board of Directors did approve the budget document but not without concerns. Board members wanted to understand how resources allocated to the various programs related to the strategic goals.They also wanted to see key operating targets and performance indicators that would show whether the Organization was using its resources effectively and efficiently. After this humbling experience, the CFO summarized his assessment of the planning and budgeting process. He noted the following: 1. 2. 3. 4. 5. 6. 7. There were few planning and budgeting policies, rules had to be made on the run. Budgeting was bottom-up without clear focus, based on local wishes. National priorities were neither well understood nor communicated by the regions, and mostly viewed as a hindrance. There was no overall attempt to align operating priorities and strategic goals with resources. There was absence of relevant operating targets and performance indicators. The operating plan and financial budget were prepared simultaneously creating confusion with managers. The final budget document presented to the board contained mostly financial data with very little strategic, operational, or risk assessment information. The alignment of strategic goals and budgets for public and nonprofit organizations can take many twists and turns. The CFO proceeded to look for a plan and budget model that would provide a more effective and efficient process. His research brought him into contact with the Secretariat in Public Fund Management, where he was able to learn about \"best practices\" in planning and budgeting. Figure 1. Annual Planning and budget cycle Level April May June July/August September Strategic Orientations adaptation Performance Beginning of measurethe Fiscal ments (KPI Year Indicators) Formulation of Budget Hypothesis Development and Implementation of Strategic Actions Financial Statements Second Quarterly Review First Quarterly Review CMA MANAGEMENT 25 November December January February/ March Confirmation of Orientations and Action Plan Tabling of Orientations and Action Plan Board/Mana- Sessions on Presentation to the Board Strategic gement Orientations Committee Permanent Personnel Division/ Regions October March 2009 Budget Approval Budget Preparation Third Quarterly Review Figure 2. Management and financial policies FINANCIAL PLANNING POLICIES REVENUES POLICIES 1. Strategic and financial long-range planning. 5. Revenue diversification. 2. Policies validation and investment projects. 6. Fees and charges. 3. Structurally balanced budget. 7. Use of one-time revenues. 4. Asset inventory, their valuation, and sustainability. EXPENDITURE POLICIES 9. Reserve or stabilisation accounts and preparedness 8. Use of unpredictable revenues. in case of disaster. 10. Operating / capital expenditure accountability. BEST IMPLEMENTATION PROCESS q Strategic planning process (forward looking). q Efficient internal control, audit committee and risk management. q Performance management and evaluation for decision making (performance indicators). q Value management (VM). BENEFITS q Financial and budget management oversight. q Efficient use of revenues. q Improved program management q Enhance techniques and provide clear and relevant q Greater transparency and citizen involvement. support and resources allocation. q Better infrastructures. Secretariat in Public Fund Management (SPFM) Founded in 2005, the SPFM is a non-profit organization whose main purpose is the advancement, promotion and implementation of best management practices in public and non-profit organizations. The SPFM works closely with the Government Finance Officers Association (GFOA) in Canada and the United States to develop practical models that can help organizations manage and monitor the use of public/donor funds. Having gone through a chaotic planning and budgeting cycle, the CFO recognized the importance of formalized financial and budget policies and turned to the SPFM for help. Best practices An annual budget cycle should include a broad scope of planning and decision making based on four key principles: q Establish broad goals to guide the organizations decision making. q Develop approaches to achieve these goals. q Develop a budget consistent with the approaches to achieve the goals. q Evaluate performance and make adjustments. information about effective strategies. A good budget process should: Incorporate a long-term perspective. q Establish linkages to broad organizational goals. q Focus budget decisions on results and outcomes (budgeting for outcomes). q Involve and promote effective communication with stakeholders. q Provide incentives to the organization's management and employees. As per the SPFM, the budget process must help decision makers make informed choices about the selection of programs and services and the allocation of resources. The four principles are mapped into an annual planning and budgeting cycle that supports the preparation of a comprehensive budget document covering critical financial and budget policies. An example of the model is presented in Figure 1. q Financial and budget policies Having gone through a chaotic planning and budgeting cycle, the CFO recognized the importance of formalized financial and budget policies and turned to the SPFM for help. Through its research, the SPFM developed financial and budget policies based upon principles recommended by the National Advisory Council on State and Local Budgeting (NACSLB) and the GFOA. The SPFM key financial and budget policies are summarized in Figure 2. They are considered fundamental to the planning and budgeting process of public and nonprofit organizations. The SPFM financial and budget policies are categorized in three groups: CMA MANAGEMENT 26 March 2009 Planning policies Financial planning policies help organizations establish a more comprehensive long-term view and create a vision towards a sustainable future. They comprise: 1. Long-range strategic and financial planning. 2. Validation and investment projects. 3. Structurally balanced budget. 4. Asset inventory (and management), their valuation, deferred maintenance and sustainability. Revenue policies Understanding the revenue stream is essential to sensible planning. Most of the following revenue policies seek stability to avoid potential service disruptions caused by revenue shortfalls. The organization needs to develop policies focusing on revenue diversification, fees and charges, use of one-time revenue, and use of unpredictable revenue. 5. Revenue diversification. 6. Fees and charges. 7. Use of one-time and unpredictable revenue. Expenditure policies The expenditures of public and non-profit organizations reflect their ongoing service commitment. Strategic expenditure planning and accountability helps ensure fiscal stability. It is recommended that organizations adopt debt capacity, issuance and management, reserve or stabilization accounts, and operating/capital expenditure accountability policies. 8. Debt capacity, issuance, and management. 9. Reserve or stabilization accounts. 10. Operating/Capital expenditure accountability. The financial and budget policies listed above are supported by a comprehensive guide developed by the ISPFM, which provides details on their use. This guide can be adapted to every organization following an initial assessment of the entity's planning and budgeting process. The ISPFM guide is a work in progress that continues to be updated based on development from various groups, including the GFOA. The implementation of the ten ISPFM recommended financial and budget policies will help the organization: q Improve its financial management process. q Reassure stakeholders about the administrative and organizational planning. q Make the most efficient use of resources in achieving long-term strategic and financial goals. The use of the recommended financial and budget policies will encourage the development of organizational goals and plans to achieve these goals, and the allocation of resources through a budget process that is consistent with such goals, policies, and plans. Given the evolving nature of CMA MANAGEMENT good management, budgeting and finance, these practices are not intended as mandatory prescriptions for the organization. Rather, they are practices that provide a milestone for the organization to make improvements to its financial and budget processes. Implementation of these practices is expected to be an incremental process that can take a number of years. After gaining a good understanding of the concepts and application of these best practices, the CFO began to apply them within The Humanitarian Organization. He soon discovered that it was a much more difficult task than he had initially expected. Board members and staff could not understand why all this work was necessary and why planning and budgeting had to be an ongoing process. There were also some internal struggles to understand the meaning of the ten financial budget policies and subtle resistance to the need for increased transparency and accountability. As per the SPFM, the budget process must help decision makers make informed choices about the selection of programs and services and the allocation of resources. It became clear that these policies would have a considerable impact on the Organization's planning and budgeting practices, but also in many other aspects of its financial and program management. Most importantly, the board and senior management had to be convinced to adopt these policies as part of their ongoing governance role. The CFO spent considerable time selling these \"best practices\" to board members all the way down to operations' managers (including accounting staff). It took many years of hard work, but in the end, the CFO was able to develop a planning and budgeting process which provided alignment between the organization's strategic goals and operational priorities, enabled effective allocation of capital and program investment resources, and, provided clear performance measurement indicators. More importantly, it presented key stakeholders with an objective view of the organization's planning and budgeting process. s Michel Pich (mpiche2@netscape.net), CMA, CIA, M.P.A., is a finance and corporate services executive with 30 years' experience in public and private organizations. He spent the last 12 years of his career as CFO in diverse sectors including: an international non-profit organization, global mining services group, and telecommunications service provider. 27 March 2009 The Symphony Of Southeast Texas In 2010: Managing A Regional Orchestra In Modern Times Escamilla, Craig;Venta, Enrique Henry R Journal of Business Case Studies; Nov/Dec 2010; 6, 6; ProQuest Central pg. 85 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. J. o r PUBLIC BUDQETIMQ, ACCOUnTif^Q fllWlCIAL MAMAGEMEMT, 20 (3), 324-354 FALL 2008 AN APPROACH TO EVALUATING RELATIVE EFFECTIVENESS IN NON-PROFIT INSTITUTIONS Yaw M. Mensah, Kevin C. K. Lam and Robert H. Werner* ABSTRACT. We present, in this study, a method for comparing the relative effectiveness of different non-profit institutions with similar objectives. In addition, we show how this measure of relative effectiveness is related theoretically to their relative efficiency. Relative effectiveness is shown to be a product of the efficacy with which potentially utilizable resources can be converted into usable inputs, and the efficiency with which the inputs are converted to outputs or outcomes. Finally, drawing on developments in data envelopment analysis, we illustrate the new methodology using data from 109 institutions of higher education. INTRODUCTION I The evaluation of organizational effectiveness has a long history that cuts across the disciplines of economics, operations research, personnel and organization management, and strategic management, as well as accounting. Economists, operations researchers and accountants have tended to focus on organizational efficiency, while personnel management, organizational and strategic *Yaw M. Mensah, Ph.D., is Professor and Research Director, Center for Governmental Accounting, Education and Research, Rutgers University at New Jersey. His research interests are in governmental accounting, international accounting, and the economic measurement of productive efficiency. Kevin C.K. Lam, Ph.D.. is Associate Professor. School of Accountancy, Chinese University of Hong Kong. His teaching and research interests are in accounting and management control. Robert H. Werner, Ph.D., is Associate Professor of Accounting and Director, Center for Governmentai Accounting, Education and Research, Rutgers University at New Jersey. He is also Director of the MAccy Program in Governmental Accounting at Rutgers University. His teaching and research interests are in governmental accounting. Copyright 2008 by PrAcademics Press An APPROACH TO EVALUATiriQ RELATIVE EFFECTIVEnESS Id nOPi-PROFiT ifiSTimiOnS management researchers have tended to specialize measurement of organizational effectiveness. 325 in the While most scholars of organizations will agree that organizational performance encompasses both effectiveness and efficiency, the tendency for the different disciplines to focus on different aspects of performance is understandable for a reason that becomes apparent once the terms are defined formally. The commonly accepted definition of effectiveness is that it measures the degree of success with which organizational goals are achieved (see, for example, Fink, Jenks and Willets, 1983). Typically, efficiency is defined as "the attainment of a desired outcome or goal with a minimum of effort, cost or waste" (Reed, 1991). Thus, while efficiency involves the quantitative ratio of inputs to outputs, effectiveness involves the softer concept of organizational goals versus organizational achievements. The discipiines where optimization concepts are emphasized can thus easily deal with the probiems of efficiency measurement but cannot easily handie the evaluation of effectiveness, particularly in areas like the non-profit sector where output/outcome measures are often not quantifiable. The management disciplines have filled this void with softer constructs which, unfortunately, do not lend themselves easily to quantification. Thus, the exact theoretical linkage between effectiveness and efficiency is lacking in the literature. The purpose of this paper is to advance a quantitative measure of relative effectiveness of different organizations with similar goals, and to show how effectiveness is iinked theoretically to efficiency. We illustrate the concepts presented by employing the data for 109 private institutions to show the relationship between the concepts. In the sections that follow, we first review the literature on the measurement of effectiveness and efficiency in the different literatures. Second, we present our conceptual model that shows how reiative effectiveness of non-profit institutions can be measured quantitatively. This construct is then related to relative efficiency. Third, we provide empirical estimates of the relative effectiveness and efficiency of a sample of private IHEs to illustrate the feasibility of the approach developed in the paper. Finally, our conclusions are presented. 326 MEnSAH. LAM AND WERNER EFFECTIVENESS AND EFFICIENCY MEASUREMENT We present in this section an overview of the literature on the measurement of organizational effectiveness and efficiency, and identify the areas where this study can make a contribution. Effectiveness It is a mild understatement to state that few topics that are of such fundamental importance and have attracted such attention from both academics and professionals are still subject to as much confusion as the concept of effectiveness. It is perhaps indicative of the confusion surrounding the topic that, in a much-cited synthesis of extant work on the subject, which led to yet another measurement approach, the competing values approach, Quinn and Rohrbaugh (1983, p. 374) state in their conclusions: "Organizationai effectiveness is not a concept It is a socially constructed, abstract notion carried about in the heads of organizational theorists and researchers." The unsatisfactory state of affairs led many researchers in the past to call for a moratorium on organizational effectiveness studies (Goodman, Atkin & Schoorman, 1983; Hannan & Freeman, 1977). As pointed out by Cameron (1986), however, the evaluation of organizational effectiveness is far too important to everyday usage for the failure of academic researchers to agree on measurement criteria to result in its abandonment. Furthermore, the problem of lack of appropriate criteria is acute only in the nonprofit sectors of the economy; since in the for-profit economic sectors, many observers will agree that profitability and customer satisfaction constitute acceptable dimensions on which effectiveness can be measured (Joys, 2001). A brief summary of the most important operational constructs developed to measure organizational effectiveness are the following four major approaches: (1) the goal model of effectiveness (Campbell, 1977; Scott, 1977); (2) the system resource approach (Pfeffer & Salancik, 1978; Yutchman & Seashore, 1967); (3) the muitipie constituencies approach (Connolly, Conlon & Deutsch, 1980; Zammuto, 1982); and the spatial modei/competing values framework (Lewin & Minton, 1986; Quinn & Rohrbaugh, 1983).^ ATI APPROACH TO EVALUATinQ REUTIVE EFFECTIVENESS IP! MOn-PROFIT inSTITUTiOnS 327 The "goal modei of effectiveness" approach adopts the traditional view that organizational effectiveness should be evaluated with reference to the declared objectives of the organization. The "system resource" approach assesses effectiveness by the extent of resources the organization is able to marshal. The "multiple constituencies" approach recognizes that an organization has multiple constituencies who are iikely to use different criterion to assess its effectiveness. Finally, the "spatial/competing values" approach is a synthesis of four different approaches to effectiveness measures: the human-relations model, the open-system model, the internal-process model, and the rational-goal modei. Out of this synthesis emerges the view that these diverse criteria relating to productivity/efficiency, stability/control, resource acquisition/ external support and the value of human resources be combined. This requires that values be articulated, weights placed on the values, and a formula be developed for combining the weighted scores on each criterion. One difficulty with the existing models of efficiency is that the concept of relative effectiveness is left fuzzy and ill-defined. Quite clearly, in making judgments of institutions, individuals do comparative evaluations of different institutions. None of the proposed criteria for evaluating organizational effectiveness, however, provides the basis for making comparative judgments of relative effectiveness. If an organization's effectiveness is judged solely by reference to its declared objectives, the perceptions of its multiple stakeholders, the resources it is able to acquire, or a complex combination of the above, comparisons across different organizations with even the same objectives become problematic. An organization can choose objectives that fall short of (or far exceed) what is attainable given its potential, and stakeholder perceptions of its performance may not reflect a comparative knowledge of what is attainable with the resources provided. These deficiencies in the existing state call for a new approach, one that attempts to introduce more rigorous comparative methodology into the assessment of relative effectiveness. To do so, a review of the development of the concept of efficiency as a measurement construct is first necessary. This review is provided in the next section. 328 MEriSAH.LAMANDWERTIER Efficiency Essentially, efficiency is a question of relating actual inputs used to the actual outputs/outcomes generated. The conceptual basis of the measurement of efficiency may be said to have originated with the development of "activity analysis" by Koopmans (1951), and its empirical operationaiization by Farrell (1957). The specific identification of possible inefficiencies in production originated with Debreu (1951). The linear programming reformulation of activity analysis was demonstrated by Charnes and Cooper (1957, 1961), and the final evolution to data envelopment analysis (DEA) by Charnes, Cooper and Rhodes (1978,1981). With the hundreds of articles its various extensions, it is fair relative organizational efficiency What remains to be resolved is effectiveness. that have been devoted to DEA and to state that the measurement of is no longer a conceptual problem. exactly how efficiency is related to A start in this direction has been made by Reed (1991) when, in the context of diversification, he argued that bimodality in measures of diversification is due to organizations trading off efficiency against effectiveness. He argued that both efficiency and effectiveness in diversification are subject to the laws of diminishing returns approached from different directions, and thus require joint optimization. Unfortunately, Reed's model does not provide any basis for resolving the difficulties in the measurement of effectiveness discussed earlier. Nevertheless, his model provides one way to visualize how effectiveness and efficiency are related. In the model presented below, we provide an alternative way in which efficiency is embedded in the concept of effectiveness. A MODEL FOR MEASURING RELATIVE EFFECTIVENESS in non-profit environments, three key characteristics frequently play a dominant role in organizational performance. The first characteristic relates to the organization's resource-acquisition ability, the second to the attainability of the organization's goals, and the third to organizational efficiency. Regardless of whether the organization charges for its services or not, because of the usual requirement of long-run financial breakeven operations, the quantity and quality of the organization's output/outcomes depend vitally on An APPROACHTOEVALUATinO RELATIVE EFrECTiVEMESS IM nOM-PROriT INSTITUTIOMS 329 the organization's ability to raise revenue or otherwise acquire resources that it can then convert into inputs. At the same time, the open-ended or closed-ended nature of the organization's goals is an important determinant of the effort required of the organization. If the goals are closed-ended, their achievement caps the effort required, and the issue of relative organizational effectiveness is then moot. If the goals are openended, then it reduces to one of output/outcome maximization subject to the resources available to achieve those goals. This is a much richer situation, and the discussion below assumes such to be the case. Measuring Relative Effectiveness The fundamental shift in thinking required to develop the notion of relative organizational effectiveness is the idea that, simitar to efficiency, relative effectiveness requires a comparison of resources to outcomes. There are, however, two key differences. The first difference is that, unlike the case of efficiency, for the measurement of relative effectiveness, the resources under consideration are not the actuai resources used by the organization, but ail potentiaiiy usabie resources. By "potentially usable resources" {hereinafter called PURs), we mean potential assets (including intangible ones) that, by creative thinking, could be converted to inputs that are utilizable by the organization. The second key difference relates to achievable outputs/outcomes (for relative effectiveness) versus actual outputs/outcomes for relative efficiency. By "achievabie outputs/outcomes" we mean the outputs/outcomes that an effective and efficient organization can generate if all the potentially usable resources were deployed and utilized in pursuit of the objectives of the organization. The rationale for substituting "achievable output/outcome" for the actual targeted objectives of the organization is that the measurement of relative effectiveness should be independent of any systematic biases in judgments that organization may make in selecting their targeted levels of output. If this were not the case, an organization with vast resource endowments that (by deiiberate choice or failure to gauge its potential) chooses low targeted outcomes would be rated effective merely because it achieved those objectives. At the same time, a 330 MEPiSAH.LAMANDWERTiER more ambitious organization with higher targets relative to its resource base would be rated "ineffective" because it failed to achieve those lofty goals. A schematic representation of the proposed model of relative effectiveness and its relationship to efficiency is presented in Exhibit 1. As shown in Exhibit 1, the PURs (Z,) have to be transformed into inputs that are used by the organization. In the nonprofit situation, revenue-generation is a primary consideration, so the level of revenues and other resources that can be raised from the PURs is the EXHIBIT 1 Diagrammatic Representation of Relative Effectiveness versus Relative Efficiency Poteatially Uable Resources based Input Reiourc Uied W Institution (X*. ' Relativ = -\\ctual Output Generated Poteiialj' .\\chievable Ouqaul ffven Polenlially Usable Resources Relative EffkieiKy = Actual Output GeneratedMaximum Output Actual Inpits Used Current Ac tual Ou^ut Geneialed (Y* J and Change in Fiiundal Reserves (RJ Potemiillj- Achievable Output Outcomes ytl^ PotertiaHy Usable Rourc Explanatory Notes: Organization's Goals/Objectives defines the Cohort/Peer group for relative effectiveness and efficiency evaluations. Only entities with similar goals/objectives are to be evaluated in a comparative mode. An APPROACH TO EVALUATiriQ RELATIVE EFFECTlVEriESS in fiOfi-PROFIT ItiSTITUTiOMS 331 primary issue. The revenues and other resources are inputs that are used by the organization to generate outputs/outcomes. The rate at which the revenues, once raised, are converted to current outputs determine the organization's efficiency rating. One important factor here relates to the relationship between revenues and costs. Although, in the long term, revenues raised must equal cost expenditures for nonprofit institutions, in the short term, this equality need not hold. This is because, to survive for the iongterm, a nonprofit institution must build financial reserves (i.e., a 'rainy day' fund) to offset temporary imbalances between revenues raised and current expenditures needed to sustain operations at desired levis. Thus, whiie the PURs must be related to revenues and other input resources raised (denoted X^A), the determination of organizational efficiency Invoives relating costs and other inputs (denoted X^A) to the current level of outputs (denoted Y^A). Differences between X^A and X^A lead to the build-up or draw-down of reserves (denoted ARA). Actual output (YA) then consists of Y^A plus ARA. The organization's own goals or objectives can be used to identify the appropriate cohort or peer group. Once this peer group is identified, the organization's reiative effectiveness (compared to the peer group) can be evaluated by the degree to which its actual output compares to the maximum output (YMAX) that could be generated given the potentiaily usable resources of the organization. YMAX is again defined to the level of maximum current output (Y^MAX) plus maximum buildup (or minimum drawn-down) of financial reserves (ARMAX) To determine the PUR domain, a researcher has to identify all resources that, conceptuaiiy, could be converted into usable inputs (given sufficient creativity). Some guidance on this can be provided by a study of the activities of the peer group in obtaining inputs (a kind of benchmarking). An effective non-profit organization (with openended goals) is one that is able to convert as many PURs as possible into usable inputs in order to bring its actual output level to the maximum output level. This implies that organizations are seeking to maximize output levels, subject to PUR constraints. In summary, the concept of reiative effectiveness implies output/outcome maximization subject to PUR constraints. In situations where the organization has specifically limited output 332 MEnSAH, LAM ATID WERNER objectives, and existing resources are sufficient to achieve those objectives (regardless of efficiency), relative effectiveness is not a meaningful concept In such situations, only relative efficiency is pertinent. In the more general situation where an organization is faced with unlimited demand for its services/products but has limited PUR, its relative effectiveness can be evaluated by the actual output it is able to generate relative to its maximum achievable output. Since maximum achievable output is defined relative to PUR, effective organizations (in resource-constrained environments) are those that are able to transform as much of these PURs into usable inputs in order to push its output/outcomes to the limit. Since the achievement of high relative effectiveness (in resourceconstrained environments) involves the conversion of untapped resources to usable inputs in order to increase the output capability of the organization, returns to scale and scope are particularly important. Naturally, one would expect that the resources that are most easily exploited are the ones that are initially tapped. Thus, one would expect the usual increasing returns to scale and scope in the early ranges of the input/output transformation process. As these resources are more fully exploited, the constant-returns-to-scale ranges are reached. Attempts to further exploit the unutilized PUR then get an organization into the decreasing-returns-to-scale and scope regions. This performance degradation, once certain levels of output are achieved, is to be expected, as less desirable resources are forced into use. A graphical representation of the expected relationships is depicted in Exhibit 2. In Exhibit 2, the three curvilinear lines shown are VRS (variable returns to scale) production functions for Most Efficient (ME), Average Efficiency (AE), and Least Efficient (LE) organizations. The straight line from the origin is the CRS (constant returns to scale) production function for the ME organization. Points ME, AE and LE denote the maximum outputs that can be generated by the ME, AE and LE entities, respectively, when the potentially usable resources are fully deployed. Only the most efficient organizations can achieve the YMAX output level. The relationships between relative ineffectiveness, relative inefficacy, and relative inefficiency are apparent from Exhibit 2. Relative ineffectiveness is measured by the distance between YMAX and YA'^E assuming a firm operates along the production function m APPROACH TO EVALUATinG RELATIVE ErFECTIVEriESS Ifi riOn-PROriT IPISTITUTIOMS 333 represented by AE. The distance from YA*^ to YA^^^ measures the traditional output-oriented inefficiency. Relative inefficacy for an organization operating along the production function AE is therefore measured by the distance from YA^^^ to YMAX.. XA represents the actual set of inputs used, and the differences in output levels (YA"^, YA*^ and reflect the differences in organizational efficiency. EXHIBIT 2 Graphical Representation of the Relationship between Relative Effectiveness and Relative Efficiency {Graph of Production Functions) CRS production INPLTS (X> POTE-VnULY US.\\BLE RESOURCES (Z) Explanatory Notes: The superscripts ME, AE and LE represent Most Efficient, Average Efficiency and Least Efficient, respectiveiy. X stands for Inputs. Z for Potentially Usable Resources, and Y for Outputs. The subscripts A and MAX represent Actual used (for inputs) or generated (for outputs) and Maximum Feasible, respectively. Relative Ineffectiveness = 1.0 - Relative Effectiveness; Relative Inefficiency = 1.0 - Relative Efficiency; and Relative Inefficacy = 1.0 Relative Efficacy. These follow the usual nomenclature in the productivity literature. ^^ MEnSAH.LAMAnDWERTIER Thus, if an organization operates along the production function ME, there will be no relative inefficiency, and relative ineffectiveness is then equal to relative inefficacy. Therefore, an organization operating along production function IVIE that is also able to equate XA to XMAX will attain zero inefficiency, zero inefficacy, and thus zero ineffectiveness. Converseiy, a firm operating along production function LE will have a higher level of relative inefficiency, and consequently higher ineffectiveness, although the level of inefficacy will not be affected as long as there is no shift in XA. It is apparent that the more an organization is able to tap its potentially utiiizabie resources, the more relatively effective it is, with efficiency held constant. In conclusion, to obtain a perfect relative effectiveness score of unity, an organization has to fuiiy exploit ail potentially usable resources inputs and also be efficient. An efficient organization does not have to be perfectly effective, and a relatively effective organization is not necessarily the most efficient. A perfect score on the relative effectiveness scale (in a sufficiently large sample), however, requires that the organization also be most efficient (within the context of a variabie-returns-to-scale world). Operational Assessment of Relative Effectiveness Based on the discussion above, the following six steps are required to evaluate the relative effectiveness (RE) of a non-profit organization against its peers: Step 1- Identification of PURs: A necessary first step in assessing the RE of an nonprofit organization is to valdate the PURs that are initially hypothesized. This will generally require multi-equation regression (or canonical correlation analyses) in which the set of hypothesized PURs (Z) are related to the set of input resources actuaiiy used by the organizations (X^^A) to generate their outputs (YA) (Including the buildup of financial reserves as needed). Step 2 - PUR Efficacy Conversion Rating: The efficacy with which the organization has been abie to convert the set of statistically significant PURs into usable input resources must be assessed in the next step. Because we theorize the existence of variable returns to scale, the BCC (Banker, Charnes & Cooper, 1984) version of the Data Envelopment Analysis model is suggested as being most appropriate. The BCC DEA model must be implemented, however, with due An APPROACH TO EVALUATiriQ RELATIVE EFFECTIVENESS in nOfi-PROFIT IfiSTITUTlOfiS 335 consideration for non-zero slacks. That is, Pareto-Koopmans' efficiency rather than the Farreli efficiency measure is called for in this situation {see Cooper, Seiford & Tone [2000, pp. 45] for a discussion of the differences between the two implementations of DEA efficiency). In this DEA model, the Zs are reiated to X^A. Since Pareto-Koopmans' Efficiency is being implemented, the usual dichotomy between input and output orientation in DEA anaiysis is immaterial. We denote the PUR efficacy rating of organization i as YI [Z.X] Step 3 - Derivation of XMAX: The PUR efficacy rating provides the basis to project the actual input resources currently used by the organization (XA) into the maximum set of inputs that the organization could theoretically generate given its set of PURs (XMAX). Given the PUR efficacy rating (y,), that the relationship can be presented as: XMAX,, = Yi [Z.XA,,] (D Step 4 - Efficiency Rating: In this step, the usuai DEA efficiency measuring the rate at which the organizations converted their inputs {XA} into outputs {YA) is derived. Since variable returns to scale is assumed, the analysis uses the BCC-DEA model and ParetoKoopmans' Efficiency concept^. We denote the efficiency rating of organization i as iI>i[X] Thus, YA, = , [XA,.] (2) Step 5 - Derivation of YMAX: In this fifth step, the XMAX determined in Step 3 along with the input-output efficiency frontier defined in Step 4 are used to derive the set of maximum outputs that could have been generated by each organization from its XMAX. Again, we assume the possible applicability of variable returns to scale, so the BCC DEA model and Pareto-Koopmans' efficiency are applied. The solution to this problem is:^ YMAX.. = Oj [XMAX,1 = a>i [Yi [Z,XA,I]] (3) (4) Step 6 - Derivation of Relative Effectiveness (RE): The final step is the computation of RE and related concepts. RE is the projection of YA to the frontier defined by YMAX, Thus, in vector notation. 336 MEriSAn.LAMATDWCRTiER REi = | Y A , | / |YMAX,| (5) Operationally, if the existence of possible scale effects in projecting from XMAX is ignored, Equation (5) can be reduced to: YMAX REi = (Di [XA.] * Yi [Z,XA, ] (6) Thus, the relative effectiveness of an organization can be measured approximately as the product of its relative efficiency and its relative efficacy in converting PURs into inputs for its production function. Estimation Approaches The estimation ofthe relationships modeled in this paper draws upon both the econometric and operations research approaches. As suggested by Cooper, Seiford and Tone (2000), regression/canonical correlation techniques can be used to determine if the theorized relationships are valid. Subsequently, DEA can be used to derive the relative effectiveness and efficiency estimates. ; Regression techniques are needed to validate the potentially usable resources as indeed resources that can be converted into usable inputs by an organization. This issue is particularly important for nonprofit institutions because the PURs are frequently in forms that are likely to be very different from the actual inputs used. For example, for public agencies the degree of political support from key stakeholders may constitute a very important PUR since that may be converted to the level of funding obtainable from a legislature. Similarly, for private institutions of higher education, key PURs may be factors like the size ofthe alumni body and its relative wealth, the location of the institution, and the past prestige of the institution. Current inputs, however, may be factors like the total operating costs, size and condition of the physical facilities, and the drawing power of the institution in terms of the entry level Scholastic Aptitude Test (SAT) scores. Regression techniques are again needed to establish the linkage between the theorized inputs and outputs of the organization. This is also a critical step in the analysis of the pert^ormance of non-profit institutions because the definitions of inputs and outputs may be subject to differences of opinion, and there may be conditioning variables that need to be considered. A good example of this conditioning through regression methods can be found in Arnold, Bardhan, and Cooper (1997) where it was found through regression Ail APPROACH TO EVALUATIfiQ RELATIVE ErFECTlVEPiESS IM non-PROFIT iflSTITUTIOMS 337 that three input variables had negative effects on an outcome variable, and thus had to be treated differently from the other inputs. EMPIRICAL ILLUSTRATION Data Source and Variables Used To demonstrate the feasibility of the relative effectiveness measurement approach, the higher- education institutional setting was selected. This selection was motivated by two factors. First, institutions of higher education (IHEs) have sufficient similarities in mission and general objectives (when grouped by the classification criterion advanced by the Carnegie Foundation for the Advancement of Teaching [1973]) to meet the requirement for an adequate sample size. Second, IHEs have been frequently used in the application of DEA techniques, so the input-output relationships required for estimating relative efficiency have been well specified for some time. Thus, this study extends the literature by introducing relative effectiveness as an additional dimension of performance evaluation. The study also provides some degree of continuity in the literature by focusing attention on the notion of potentially utilizable resources, and their rote in evaluating institutional performance.'* The sample chosen for this study was used in a previous study by Mensah and Werner (2003) in which they showed a negative relationship between financial flexibility and cost efficiency. The sample was generated randomly from a list of colleges and universities covered in the 1997 and 1998 US News and Worid Report Coiiege Survey. Ofthe 200 institutions contacted initially, 1 3 1 institutions responded and were used in that study. For this study, the sample was further reduced to the 109 private colleges and universities with data available for both years. Based on the previous literature (Ahn, Arnold, Charnes and Cooper, 1989), three variables were chosen to measure the inputs of IHEs: Total Operating Expenses (TOPEXP); Total Plant, Property and Equipment at book value (PPE), and the Average SAT score of entering first-year undergraduate students (SAT). TOPEXP and PRE were gathered from the financial statements of the IHEs for 1997, while SAT was obtained from US News and World Report College Survey. These variables constitute the set of inputs we referred to earlier as the cost-based inputs (X^A). 338 MEdSAH, LAM AHD WERflER For the current output measures (Y^A), five variables were initially selected based on the prior literature. These variables were the number of full-time equivalent graduate students {GRAD), the number of full-time equivalent undergraduate students {UGRD), the amount of external research grants obtained by the faculty as reported in the IHE's 1997 annual report {RESC), the average graduation rate of undergraduate students {GRATE), the academic reputation rating of the institution {ACRER), and the alumni giving rate {ALMGR). GRAD and UGRD capture the volume of the IHE's output, while GRATE captures the effect of inter-institutional differences in the throughput rate. ALMGR is used as a proxy for the level of alumni satisfaction with the institution (reflecting both the level of identification and interest in the furtherance of the institution's objectives). RESC proxies for the level of faculty research output, and ACRE? reflects the reputation that the institution has garnered among its peers for the quality of its academic activities. Except for RESC, all the other variables were obtained from the US News and Worid Report Coiiege Survey. Because ACREP scores are reported by US News and Worid Report by college classification, we introduced the four classes under which the ratings are classified; national university {NU), national liberal arts college (NLA), regional university (RU), and regional liberal arts college (RL4). Six variables were initially identified as suitable PURs (Z) for the IHE setting. The age of the institution {AGE) was theorized to be a potentially utilizable resource because of the belief that, the older the institution is, the greater the opportunity the institution has had to build links in the community, build up a reputation, and establish a strong alumni network. Thus, AGE was theorized to be positively associated with all the INPUTS. The LOCATION (by region) of the institution was theorized to be another PUR, with institutions in the EAST expected to have higher advantage because of the higher population density than those in the WEST, MiDW (Midwest), and SOUTH. Furthermore, LOCATION SETTING was also theorized to be another PUR. Institutions in RURAL settings were theorized to have a comparative disadvantage relative to those in URBAN or SURB (suburban) locations because of the lower density of the immediate population that they can draw upon. This potential disadvantage is offset by the lower cost of living in rural areas, and hence, lower operating costs. IHEs in SURB locations were theorized to have an advantage over URBAN locations in attracting ATI AFFROACH TO EVALUATIPIQ RELATIVE EFFECTiVEnESS IM nori-PROFlT IMSTITUTlOnS 339 the best students because of the greater attractiveness of suburban locations, although the lower population density may offset this advantage. The other three PURs are the estimated maximum number of living alumni (MAXALM), the level of endowment from the previous year {EDW), and the previously-established level of demand for the institution's services (DEMD). MAXALM was estimated as the total number of students enrolled in the institution multiplied by the lesser of 30 or the age of the institution. In general, the higher MAXALM, the greater the resources that can potentially be generated. EDW reflects the resources carried forward from previous periods that can be used to support the demand for services. DEMD is computed as the complement of the student selectivity ratio (i.e., the number of students rejected to the total number of applications received). It therefore reflects the degree to which institutional reputation for quality of services relative to cost has already been established from past history. For the purpose of relating the PURs to the Inputs, TOPEXP was replaced by total revenue {TREV), The rationale is that, for measuring efficacy ratings, the focus has be to shifted to the ability of the institution to raise revenues, and using TREV instead of TOTEXP accomplishes this objective. Thus, the X^A variables consisted of TREV, PPEan SAT. To formally demonstrate that the linkages linked theoretically are valid empirically, the following two sets of regression equations were estimated. The first set of equations was designed to determine which hypothesized PURs were related to the three revenue-based inputs. These equations can be written as: logTREVt = aQ + a^ MIDW + Clf^ WEST + a^ SOUTH + y^ RURAL + y^ SUBRB + /L, log A G M + / log MAX4.Mt-l +A}log EDWt-l +/l^ DEMANDi-i (7) logPPFt = a(, + a^ MIDW + aj WEST + a-^ SOUTH + ;K, RURAL + y 2 SUBRB + + Z^ log MAXALM t-l + X, log DlVt-l +^ DEMAND t-i (8) \\ogSATi = a j j + a , MIDW + 02 WEST + a^ SOUTH + y^ RURAL + / ^ SUBRB + X^ log AGEt-i + Xj log MAXALM t-1 + ^ log EDVVt-i + X^ DEMAND M (9) 540 Based on the previous discussion, all the X parameters in Equations (7) to (9) are expected to be positively signed. The signs of the a coefficients (except the intercept) are expected to be negative, reflecting the expectation that, compared to the institutions in the densely-populated East (which represents the base), the institutions in the other regions will be slightly disadvantaged. The signs of the Y coefficients, however, could be either positive or negative (with URBAN as the base), depending on the relative advantages enjoyed by the urban location vis--vis rural and suburban locations. To determine if the cost-based inputs (X^A) and current outputs (Y^A) selected are related to each other in the directions expected, the following regression equations were estimated: logTOPEXPt = ao + a, NLA +a2RU+ a^ RLA + ^ logGRADt + P- log UNDGi + /?3 GRATEx + ^ log RESCt + ^ ACREPt + ^ ALMGRx logPPEt = 0 + a , NLA -^ajRU {10) -^ 3 RLA + yff, lOgGR^^Di + ^ log UNDGt + ^ GRAJEx + ^ log RESCt + ^ ACREPt + ^^ ALMGRx (H) logSvATt = a ^ + a , NLA + 02 RU + a-^ RLA + ^ \\ogGRADx + ^ log UWDGt + /?3 GRATEx + ^ log RESCt + ^ ACREP\\ + ^ ALMGRx ( 12) Based on the previous discussion, the signs of all the parameters in Equations (10) to (12) are expected to be positive, reflecting the expectation that the generation of these outputs consume resources, on average. The signs of the a parameters are negative, since with NU as the base, one would expect the other types of institutions to have lower costs, given the lack of the heavy administrative overhead costs associated with comprehensive nationai universities. Results of Validation of Variables To facilitate future referencing, the list of variables used in the empirical part of the study is presented in Table 1. Summary statistics on the distribution of the sample by Carnegie Foundation classification and geographic location and setting are given in Tabie 2. In general, the sample is wideiy dispersed over the spectrum of Carnegie Foundation classifications, with 32 NLA colleges, 3 1 RLA colleges, 28 NU institutions, and 18 RU institutions. To reduce the m APPROACH TO EVALUATiriQ RELATIVE ErrECTIVEPiESS iri PiOPI-PROriT IfiSTITUTlOnS TABLE 1 List of Variables Used in the Empirical Part of Study EAST MIDW WEST SOUTH RURAL SUBRB URBAN NU NU RU RLA Log AGE Log MAXALM Log EDW96 DEMD96 Log RVPS97 Log TEXP97 Log PPE97 Log SAT97 Log GRAD97 Log UNDG97 GRATE97 L o g RESC97 ACREP97 ALMGR97 = Dummy variable for institutions located in the NorthEast = Dummy variable for institutions located in the Midwest = Dummy variable for institutions located in the West = Dummy variable for institutions located in the South = Dummy variable for institutions located in a rural setting = Dummy variable for institutions located in a suburban setting = Dummy variable for institutions located in an urban setting = Dummy variable for institutions classified as National university = Dummy variable for institutions classified as National Liberal Arts College = Dummy variable for institutions classified as Regional University = Dummy variable for institutions classified as Regional Liberal Arts College = Natural log of the age of the institution = Natural log of the estimated maximum size of alumni body = Natural log of the total endowment of the institution in 1996 = Natural log of the total demand for the institution in 1996. estimated as the ratio of students rejected to total applications. = Natural log of total revenue per student enrolled In 1997 = Natural log of total operating expenses in 1997 = Natural log of total property, plant and equipment in 1997 = Natural log of average SAT score of entering undergraduate students in 1997 = Natural log of total graduate students enrolled in 1997 = Natural log of total undergraduate students enrolled in 1997 = Average graduation rate of undergraduate students in 1997 = Natural log of total external research grants generated in 1997 = Academic reputation score from US News and World Report survey in 1997 = Alumni giving rate (as proxy for alumni satisfaction) in 1997 skewness in the distribution of some of the key variables and also to avoid heteroscedasticity in the estimation of Equations (7) to (12), natural log transformations were applied to ten of the 14 continuous variables. The summary statistics reported are based on the original variables before the transformation. To enable the log transformation for RESC and GRAD, we added unity to all the scores for these two variables prior to the log transformation. Since the objective of this empirical part of the paper is merely to demonstrate the feasibility of the relative effectiveness 342 TABLE 2 List of Variables Used in the Empirical Part of Study {Sample Size = 109) Variable Mean Panel A. PURs AGE Standard Deviation Minimum Maximum 119 2 MAXALM 60,476 EDW96 (in $1,000} $ 124,243.67 DEMD96 1.39 Panel B. Inputs TREV97(m$K) $ 76,879.92 TEXP97 (r) $K) $ 70,262.96 PPE97(in $K) $ 63,576.55 SAT97 1,107.65 Panel C. Outputs/Outcomes GRAD97 142.59 UNDG97 2.143.08 GRATE97 0.68 RESC97(m$K) $ 837.15 ACREP97 2.97 ALMGR97 0.29 3 5.26 1.25 $ 3.82 3.53 3.78 1.17 $ 1.05 $8.433.78 $3,071.74 735.10 $ 2,191.287.88 $ 1.425,452.69 $ 1,848,712.48 1,465.57 26.84 2.18 0.17 40.04 0.62 0.15 298.87 0.27 15,835.35 16,983.54 0.96 744.151.56 4.00 0.68 $ $ $ $ $ 18 8,022 943.88 $ 299 573.779 $ 6,517.490.71 2.39 $ 1.26 0.02 Panel D. Regional Location EAST Total sample size Hanel b. Location Setting MIDW WEST 63 14 15 SOUTH 16 RURAL Total sample size SUBRB URBAN 22 37 50 Manel h. Class Total sample size NU 28 NLA 32 RU 18 RLA 31 techniques advocated here, this transformation is presumed to be innocuous in its effect. For the same reasons, the log-transformed variables were used in both the regression analyses and the subsequent DEA applications. Table 3 presents the results of estimating Equations (7) to (9). This is an attempt to determine if the PURs hypothesized are, in fact, A1 APPROACH TO EVALUATIH RELATIVE ErrECTIVEMCSS IM MOn-PROF!T ifiSTlTUTlOnS 343 related to the level of inputs. Examining first the results for TREV in Panel A of Table 3, the results show that MIDW has a positive coefficient relative to EAST. This implies a relative revenue advantage to location in the Midwest relative to the East. The other two geographic dummy variables are not significant, so location in the West or South is not a relevant factor. Furthermore, none of the Location Setting dummy variables are significant, indicating no effect of the location setting of the institution. The coefficient for AGE is not TABLE 3 Multiple Regression Analyses of PURs (Independent Variables) and Inputs (Dependent Variable) As Test of Validity of PURs as Determinants of Levels of Inputs Available logSAr97 logPPE97 log TREV97 Independent Expected Sign Variables Coeffic t-value Coeffic t-value Coeffic t-value Intercept 1.862 -3.09 -1.482 -2.46f 6.378 44.65 Regional Location EAST (base) MIDW WEST SOUTH 0,265 0.094 0.160 2.34 1.56 0.109 -0.062 0.011 0.96 -0.57 0.1 -0.113 -0.005 -0.058 -4.21 -0.18 -2.38" 0.033 0.041 0.32 0.50 -0.001 0.081 -0.01 0.97 -0.025 0.028 -1.04 1.39 0.002 0.242 0.0391 0.881 -0.20 15.19a 11.183 3.97 0.932 -0.011 0.643 0.454 0.477 -0.11 13.15 12.96 2.31" 0.931 0,048 -0.033 0.062 0.161 0.88 Location Setting RURAL SUBRB URBAN (base) PURs Log AGE Log MAXALM Log EDW96 DEMD96 Adjusted Rsquared F-ratio/ Significance level + + + + 165.55 161.66 2.131= -2.83= 7.52 3.28 0.716 31.31 Notes: All significance levels are one-tailed for coefficients with expected signs; otherwise they are two-tailed, a = Significant at probability of 0.001; " = Significant at probability of 0.01; " = Significant at probability of 0.05. Statistically significant, while those for EDW, DEMD and MAXALM are ail positive and significant {as expected). Moving on to Panels B and C of Table 3, similar results are observed. With PPE as the dependent variable (Panel B), MAXALM, EDW and DEMD all have positive coefficients, as expected. However, AGE is not statistically significant here as well. With SAT as the dependent variable (Panel C), AGE, EDW and DEMD all have positive and statistically significant coefficients, consistent with prior expectations. MAXALM, however, has a statistically significant negative coefficient. Since these are mere associational tests (rather than cause-and-effect relationships), the negative sign may indicate that high SAT entry-score standards tends to contract the pool of alumni of an institution. In other words, for the purpose of increasing the entry level SAT scores, MAXALM cannot contribute directly. Overall, the primary conclusion from these analyses is that the suitable PURs for inclusion in the next- stage DEA analyses are the dummy variables for Regional Location, AGE, MAXALM, EDW and DEMD. The negative sign for MAXALM in the regression with SAT as the dependent variable suggests the need for sensitivity analyses to see how dropping MAXALM in the second-stage DEA analyses might affect the relative effectiveness scores. The regional variables for Location Setting can be dropped from any further analyses since they are not significant in any of the regressions. AGE, however, is retained because it has the hypothesized effect on SAT. i Table 4 presents the results of estimating Equations (10) to (12). Panel A of Table 4 shows that, with TOPEXP as the dependent variable, all variables (except ALMGR) are significant and signed as expected. Relative to NU, the operating costs of NM, RU, and RLA institutions are lower. All the hypothesized output factors (except ALMGR) have positive coefficients. Panels B and C presents the results with PPE and SA7 as the dependent variables. In both panels, all the continuous output variables except RESC and ALMGR are positive and statistically significant. Since ^LMGR is not statistically significant in any of the regression equations, it is reasonable to conclude that It is not a valid output characteristic and should be excluded. Afi APPROACH TO EVALUATlfiQ RELATIVE EFFECTIVENESS in PiOfi-PROFIT IMSTITUTlOriS 345 TABLE 4 Regression Analyses of Outputs (Independent Variables) a n d Inputs (Dependent Variable) as Test of Validity of Outputs as Determinants of Levels of Inputs Used logSAT97 log TEXP97 logPPE97 Independent Expected Sign Variables Coeffic t-value Coeffic t-value Coeffic t-value Intercept 4.625 8.94 4.207 5.48 6.755 48.69" CLASS Dumniy Variables NLA RU -0.817 RLA Output/Outcomes -1.106 Log GRAD97 LogUNDG97 GRATE97 Log RESC97 ACREP97 ALMGR97 Adjusted Rsquared F-ratio/ Significance level -1.100 + + + + + 0.046 0.634 1.049 0.021 0.458 -0.128 6.17 -9.86 -7.95 -0.716 -1.102 3.65 6.66a -1.129 -5.47" 2.95^ 0.063 0.645 1.141 0.002 0.475 2.76^ 8.88 3.50 1.87 -3.378 0.004 0.032 0