U.S. Army Cost Benefit Analysis Guide 12 JANUARY 2010 Prepared by Office of the Deputy Assistant Secretary of the Army (Cost and Economics) Version 1.0 U.S. Army Cost Benefit Analysis Guide - V 1.0 2 Table of Contents Purpose ......................................................................................................................................................... 6 Introduction .................................................................................................................................................. 6 What is a cost benefit analysis? ............................................................................................................... 8 Quick Review ............................................................................................................................................ 9 Cost Benefit Analysis Steps - A Short Summary ......................................................................................... 10 STEP 1 - Develop the Problem Statement, Define the Objective and the Scope ....................................... 12 Problem Statement ................................................................................................................................ 12 Objectives ............................................................................................................................................... 12 Scope ...................................................................................................................................................... 13 Quick Review .......................................................................................................................................... 13 STEP 2 - Formulate Assumptions and Identify Constraints ........................................................................ 15 Formulate Assumptions ......................................................................................................................... 15 Identify Constraints ................................................................................................................................ 15 Quick Review .......................................................................................................................................... 16 STEP 3 - Document the Current State (the Status Quo)............................................................................. 17 Define the Status Quo Alternative ......................................................................................................... 17 Document the Status Quo Alternative ................................................................................................... 17 Quick Review .......................................................................................................................................... 18 STEP 4 - Define Alternatives with Cost Estimates ...................................................................................... 19 Define Alternatives................................................................................................................................. 19 The Cost Analysis Process ...................................................................................................................... 20 Cost Analysis Preparation .................................................................................................................. 21 Ground Rules and Assumptions (GR&A) ........................................................................................... 21 Data collection and analysis ............................................................................................................. 22 Work Breakdown Structure (WBS) .................................................................................................... 23 Cost Estimate ..................................................................................................................................... 23 Accuracy/Reasonableness ................................................................................................................. 23 Data Sources........................................................................................................................................... 24 Cost Strategy .......................................................................................................................................... 25 Organizing Cost Data .............................................................................................................................. 25 Quick Review .......................................................................................................................................... 26 U.S. Army Cost Benefit Analysis Guide - V 1.0 3 STEP 5 - Identify Quantifiable and Non-Quantifiable Benefits .................................................................. 27 Benefits Analysis Overview .................................................................................................................... 27 Types of Benefits .................................................................................................................................... 28 Quantifiable Benefits ......................................................................................................................... 28 Non-Quantifiable Benefits ................................................................................................................. 29 Military Benefits Analysis (MBA) ............................................................................................................ 29 Identify, Estimate, and Evaluate Benefits .............................................................................................. 30 Identifying Benefits ................................................................................................................................ 30 Benefit Categories .................................................................................................................................. 31 Estimating Quantifiable Benefits............................................................................................................ 32 Evaluating Non-Quantifiable Benefits .................................................................................................... 32 Types of Quantifiable and Non-quantifiable Benefits............................................................................ 33 Quick Review .......................................................................................................................................... 34 STEP 6 - Define Alternative Selection Criteria ............................................................................................ 35 Alternative Selection Criteria ................................................................................................................. 35 Quantitative Methods ....................................................................................................................... 35 Non-Quantitative Methods ............................................................................................................... 39 Quick Review .......................................................................................................................................... 40 STEP 7 Compare Alternatives .................................................................................................................... 42 Compare Costs and Benefits .................................................................................................................. 42 Alternative Comparison Decision Matrix ............................................................................................... 42 Define Trade-Offs and Billpayers ........................................................................................................... 43 Describe Second and Third Order Effects (Cause and Effect) ................................................................ 44 Perform Sensitivity Analysis and Risk Assessment ................................................................................. 45 Sensitivity Analysis............................................................................................................................. 45 Risk Assessment................................................................................................................................. 46 Quick Review .......................................................................................................................................... 47 STEP 8 - Report Results and Recommendations ........................................................................................ 48 Documenting the CBA ............................................................................................................................ 48 Briefing the Results of the CBA Using the Template .............................................................................. 50 Appendix A .................................................................................................................................................. 51 References .............................................................................................................................................. 51 Appendix B .................................................................................................................................................. 53 Acronyms................................................................................................................................................ 53 Appendix C .................................................................................................................................................. 55 U.S. Army Cost Benefit Analysis Guide - V 1.0 4 Glossary .................................................................................................................................................. 55 Appendix D .................................................................................................................................................. 63 Cost Estimating Models and Tools ......................................................................................................... 63 Appendix E .................................................................................................................................................. 71 Cost Benefit Analysis Case Study............................................................................................................ 71 Appendix F .................................................................................................................................................. 93 Template User Instructions. ................................................................................................................... 93 U.S. Army Cost Benefit Analysis Guide - V 1.0 5 Purpose The U.S. Army Cost Benefit Analysis Guide is provided for use by analysts and agencies as they perform cost benefit analysis (CBA) to support Army decision makers. The purpose of the CBA Guide is to assist analysts in identifying, quantifying, and evaluating the future costs and benefits of alternative solutions. Based on a structured process, this Guide will assist analysts in recommending the optimum course of action for decision-making purposes. CBAs are prepared because decision makers need reliable, objective assessments of alternative courses of action. In some functional areas, such as the world of weapon systems acquisition, guidance has already been published regarding cost estimating and cost-benefit analysis. This Guide is intended for more general use, and is aimed at functional areas where guidance does not exist. Introduction In today's resource-constrained environment, the Army must exercise wise stewardship of every dollar it manages. A key element in that stewardship is to develop and use sound CBA practices throughout all requirement/resourcing processes. For every proposed program, initiative or decision point that is presented to decision makers, it is important to provide an accurate and complete picture of both the costs to be incurred and the benefits to be derived. The Senior Leaders of the Department of the Army have directed that any decisions involving Army resources be supported by a CBA. To support this objective, the Office of the Assistant Secretary of the Army (Financial Management & Comptroller (OASA (FM&C)) has developed this Guide. The Guide is applicable to a wide range of requirements, issues, tasks, and problems that require a deliberate analysis to arrive at the optimum course of action. As stated above, the U.S. Army Cost Benefit Analysis Guide is designed to support the Army leadership decision-making process. The Guide separates the CBA process into eight major steps. The eight steps (which are discussed in detail in subsequent sections of this Guide) are: 1. 2. 3. 4. 5. 6. 7. 8. Develop the Problem Statement, Define the Objective and the Scope Formulate Assumptions and Identify Constraints Document the Current State (the Status Quo) Define Alternatives with Cost Estimates Identify Quantifiable and Non-Quantifiable Benefits Define Alternative Selection Criteria Compare Alternatives Report Results and Recommendations A short description of each step may be found at the end of this section (Pages 8-9). U.S. Army Cost Benefit Analysis Guide - V 1.0 6 Cost Benefit Analysis Cost Benefit Analysis???Making the case for a project or proposal: Weighing the total expected costs against the total expected benefits over the near, far, and lifecycle timeframes from an Army enterprise perspective BENEFITS 1. Develop the Problem Statement, Define the Objective and the scope COSTS The total of quantifiable and non-quantifiable costs Quantifiable costs Salary and benefits Procurement Sustainment Other costs 2. Formulate Assumptions and Identify Constraints 3. Document Current State (Status Quo) 4. Define Alternatives with Cost Estimates The total of quantifiable and non-quantifiable benefits Quantifiable benefits Cost Savings Cost Avoidances 5. Identify Quantifiable and Non Quantifiable Benefits Non-quantifiable benefits Greater capability Faster availability Non-quantifiable costs 7. Compare Alternatives Better quality Opportunity costs Improved morale 8. Report Results and Externalities Recommendations Other? BEN EFITS MUST BALAN CE OR OUTWEIGH COSTS 6. Define Alternative Selection Criteria The Cost Benefit Analysis Process When we refer to the Army enterprise, we mean that initiatives should be evaluated based on the benefits they provide to the Army as a whole, not to any individual organization. Army elements are connected organizationally and what happens even at the lowest levels within the Army can impact/influence higher level organizations. This Guide also provides a briefing template designed to present the CBA results to decision makers. To learn more about the template, see Step 8, \"Briefing the Results of the CBA Using the Template\" on page 44 of this Guide. Appendix E, beginning on page 68of this Guide, contains an example of completed CBA and template that follows the 8-Step CBA process. The template as well as this Guide may be found at the following location: http://asafm.army.mil/offices/LinksDocsOffice.aspx?OfficeCode=1400 Our goal is to make this Guide and the CBA process as clear and as user-friendly as possible. In addition, OASA (FM&C) will review and update the CBA Guide as necessary. Comments and questions from users are encouraged and should be submitted to: cbaguidebook@conus.army.mil U.S. Army Cost Benefit Analysis Guide - V 1.0 7 Finally, a CBA is a living document. A CBA should be updated as needed, especially as the recommendations it supports work their way through an organization's approval process and assumptions change. For example, from the time a CBA is initiated to the time it is scheduled to be briefed to a senior decision maker certain data elements, such as costs or funding tradeoffs, may have changed. Therefore, it is important for the preparer to keep the CBA updated so that the ultimate decision maker can make an informed decision based upon the best available information. What is a cost benefit analysis? All CBAs provide decision makers with facts, data, and analysis required to make an informed decision. In its most basic form, the CBA is a tool to support resource informed decision making. There is no prescribed length to a CBA. All that is required is that it fully supports the recommendation. Therefore, quality is genuinely more important than quantity. A CBA: Is a decision support and planning tool that documents the predicted effect of actions under consideration to solve a problem or take advantage of an opportunity. Is a structured proposal that functions as a decision package for organizational decision makers. It defines a solution aimed at achieving specific Army and organizational objectives by quantifying the potential financial impacts and other business benefits such as: o Savings and/or cost avoidance o Revenue enhancements and/or cash-flow improvements o Performance improvements Considers non-financial or non-quantifiable benefits of a specific course of action (COA). This feature is important because although the financial data may favor one COA over another, there may be situations where the non-financial data/information is considered more important to the analyst or senior decision maker. Furthermore, the non-financial criteria and conclusions may support something other than what the financial data favors. Includes an analysis of business process performance and associated needs or problems, proposed alternative solutions, assumptions, constraints, a risk analysis. Is process oriented. It will not only develop a set of choices that will be analyzed but it will also lead the analyst to a recommended choice. Provides an evaluation and justification of a proposed solution (including any associated expenditures) before a significant amount of funds are invested. Documents the reasons for the investment and the options available and describes how the investment helps the organization reach its goals. Guides the decision maker to focus on the major issues surrounding the recommended solution and to not spend time on minor issues. U.S. Army Cost Benefit Analysis Guide - V 1.0 8 Some characteristics of a CBA are: It must be tailored to fit the problem. It will not produce a result that is more valid than the input data. It will not make a final decision; that will be the responsibility of the decision maker/leadership. It will not act as a substitute for sound judgment, management, or control. Quick Review The primary objective of developing a CBA is to identify and obtain approval of the optimum way to solve a specific problem or capitalize on a specific improvement opportunity. The following should also be kept in mind to increase the chance of success. Keep it clear and concise. Minimize jargon and conjecture. Communicate all facts as part of the overall story. Demonstrate the value that the initiative will bring to the organization and the enterprise (key stakeholders). U.S. Army Cost Benefit Analysis Guide - V 1.0 9 Cost Benefit Analysis Steps - A Short Summary 1. Develop the Problem Statement, Define the Objective and the Scope The problem statement clearly defines the problem, need, or opportunity that requires a solution and describes what the effort intends to accomplish. The objective of the effort is to improve, reduce, or increase some aspect of a process, procedure, or program. Objectives should be measurable, realistic, achievable, and results-oriented. Simply put, objectives are measurable outcomes. Scope defines the range of coverage encompassed by an initiative or proposal along specific dimensions like time, location, organization, technology or function. 2. Formulate Assumptions and Identify Constraints Assumptions are factors or conditions that are essential to the success of the solution and are beyond the control of the organization. Assumptions define the ground rules and accepted statements in order to limit the scope of the CBA. They are explicit statements of conditions on which the CBA is based. Constraints usually refer to limits placed on resources to be devoted to the project. Constraints or barriers are normally beyond the control of the analyst and provide limitations within which analyses take place. 3. Document the Current State (the Status Quo) This defines and assesses the current state/ condition. This should include a presentation of the estimate of costs associated with the status quo. The status quo alternative of the CBA is the \"baseline\" program or system against which the costs and benefits of all feasible alternatives are compared. 4. Define Alternatives with Cost Estimates Alternatives are potential solutions to the problem statement which will be evaluated in the CBA. Alternatives should reflect a review of the mission and strategic goals to verify that the alternative's objectives are consistent with the problem statement. A cost estimate captures the total cost of each alternative over its entire life cycle and is a summation of all relevant cost elements. 5. Identify Quantifiable and Non-Quantifiable Benefits Benefits are results expected in return for costs incurred for a chosen alternative. They are the quantitative and qualitative improvements expected or resulting from the implementation of an alternative. Quantifiable benefits are benefits that can be assigned a numeric value such as dollars, physical count of tangible items, or percentage change. Non-quantifiable benefits are subjective in nature and can make a positive contribution to the analysis. Some examples of non-quantifiable benefits are improvement in morale and customer satisfaction. U.S. Army Cost Benefit Analysis Guide - V 1.0 10 6. Define Alternative Selection Criteria Alternative selection criteria are those standards/bases on which a decision will be based. CBAs must contain documentation that outlines decision criteria and identifies the extent to which each alternative satisfies each of the criteria. 7. Compare Alternatives a. Compare Costs and Benefits The essence of the CBA process is in comparing the costs and benefits of two or more alternatives (including the status quo) in order to select the preferred alternative. As a general rule, the preferred alternative is the alternative that provides the greatest amount of benefits in relation to its cost. b. Define Trade-offs and Billpayers Trade-offs / billpayers are the funding sources that have been identified which will cover (partially or entirely) the costs of an alternative. c. Identify Second and Third Order Effects (Cause and Effect) Second and third order effects are the results (consequences and/or impacts) stemming from a decision. They include the opportunity costs of pursuing one alternative over another. Second and third order effects identify what a decision maker can do or not do as a result of a decision. d. Perform Sensitivity Analysis and Risk Assessment Sensitivity analysis explains what the effect is on the cost/benefit model should assumptions change, risks become issues and/or dependencies not be met. Risk assessment describes all risks that can impact the achievement of stated benefits or the cost of solving the business problem. Each risk has an associated mitigation strategy and an assessment of likelihood of occurrence. 8. Report Results and Recommendations Results and recommendations summarize the findings of the analysis and make conclusive statements about the comparisons of alternatives. The conclusions should demonstrate the cost/benefit relationships between each alternative. The results address how the alternatives were ranked using the criteria developed in Step 6. Following a clear statement of the conclusions, there should be a firm recommendation regarding the preferred alternative. Identify Supporting Documentation All data and other information used in Steps 1-8 must be adequately documented. Supporting information should be identified so decision makers and analysts can understand how Steps 1-8 were developed. U.S. Army Cost Benefit Analysis Guide - V 1.0 11 STEP 1 - Develop the Problem Statement, Define the Objective and the Scope This section discusses three areas: Define the initiative or proposal using a problem statement Define the objective/goal Define the scope of the analysis Problem Statement The first step of the CBA process and one of the most important is defining the initiative or proposal using a problem statement. The problem statement clearly defines the problem, the mission need, or required capability. When developing a problem statement, the key is to state the problem in terms of the organization's mission. What required performance or outcome is not being achieved? Who and what are impacted by this problem? Specifically, who are the customers or stakeholders? Briefly describe the process for providing the procedure, product or service where the problem or improvement opportunity occurs and how and why it occurs. The following two examples were adopted from the Lean Six Sigma Green Belt program of instruction (July 6, 2009): Example of a weak problem statement: The CAC, Common Access Card, Issuing Process needs to be improved. We've received numerous complaints from DA Civilians and Soldiers. Why? It is vague and does not identify the problem and it includes a goal. Example of a strong problem statement: The DA Civilians and Soldiers expect the CAC Issuing Process lead time not to exceed 2 hours and the current process lead time has averaged 6.2 hours. The CAC process has shown a steady increase in lead time since January 2006 at Ft. Washington. Why? It identifies a problem in real terms. Is states when and where the problem started and who is impacted. Objectives The objective describes what the effort intends to accomplish. The objective should address why the issue is important to the organization and who will benefit from the courses of action. The objective/goal of the effort may be to fill a capability gap or improve some aspect of a process, procedure, or program. In defining objectives, various elements must be considered: U.S. Army Cost Benefit Analysis Guide - V 1.0 12 mission needs, costs, level of effort, time schedules, allowable operational changes, and ease of future modification and expansion. Objectives should be defined in a clear, specific, and measurable manner. Objectives should be realistic, achievable, and results-oriented. The more precisely the objective can be defined, the greater the likelihood that the analysis will meet the needs of the decision maker. The objective statement sets the tone and expectation for the CBA. The statement of objectives should be carefully developed and clearly stated because it will become the authoritative source for lower-level (derived) requirements. It should discuss the goals of the proposal and its relationship to the Army's strategic goals. Some objectives may be related to the correction or improvement of a specific challenge or difficulty which the Army has encountered. Other objectives may involve improvements in the quality, accuracy, and/or timeliness of programs and processes. Some examples of objectives that may be appropriate: o o o o Reduce number of man-hours of effort required for a mission by a minimum of X%. Increase output produced by the organization by no less than X units per month. Improve product quality against a given standard of X or less errors per page. Provide a new, previously unavailable product or service at a reasonable cost. The objective statement in the CBA can and should be short and succinct. It is important to ensure that the descriptions for all objectives are easily related to the goals of the CBA. The objective should be evaluated to ensure that it aligns with the mission and strategic goals of the organization. While defining initiative goals, ensure that they are verifiable through formal measurement. Scope The scope of the analysis defines the range of coverage encompassed by the project along specific dimensions such as time, location, organization, technology or function. Defining the scope of the CBA is critical because it keeps the CBA focused on the things that matter. Quick Review The problem statement focuses the CBA. The problem statement describes the symptoms that the CBA is to diagnosis and treat. Four attributes of a good problem statement: U.S. Army Cost Benefit Analysis Guide - V 1.0 13 o Defines the problem o Identifies where the problem is appearing o Describes the size of the problem o Describes the impact the problem is having on the organization Objectives should be specific and measurable. The Scope should consider dimensions such as time, location, organization, technology or function. U.S. Army Cost Benefit Analysis Guide - V 1.0 14 STEP 2 - Formulate Assumptions and Identify Constraints This section discusses two areas: Formulate Assumptions Identify Constraints Formulate Assumptions Assumptions define and reasonably limit the scope of a CBA. They are explicit statements of conditions on which the CBA is based. An assumption refers to an event or condition that is essential to the success of the proposed solution but is beyond the organization's control. Because an assumption is a hypothesis related to unknowns (as opposed to a "fact") or to a future occurrence, it involves a degree of uncertainty. Assumptions play a critical role in explaining the business case results, in building credibility for the case, and in reducing and measuring uncertainty in projections. For this reason, regardless of the impact on the analysis, identify all pertinent assumptions. Do not confuse assumptions with facts or statements that, with research, could be presented as factual data. Here are two examples of assumptions: If a landfill is being considered as an alternative to solving a disposal problem stemming from increased waste, the study might include the assumption that, \"sufficient land for the operation is available within a 20-mile radius of the installation.\" In this particular instance, however, there may have been no reason why this assumption could not be verified with research and presented as a fact. If the organization is considering a solution that would require a change to a federal law, the analysis might include an assumption that any required legislative changes would be approved by higher headquarters and enacted by Congress. This is something that is clearly beyond the local organization's ability to control or to know for certain. Identify Constraints All managers are faced with certain constraints within which they operate. Constraints usually refer to limits placed on resources to be devoted to the project. Constraining organizational policies or procedures, funding considerations, physical limitations, and all time-related considerations need to be addressed in the CBA. External constraints or barriers are normally beyond the control of the analyst and provide limitations within which analyses take place. Assumptions and constraints should be established and fully documented early in the process, in order to preclude a recommendation that is not feasible or cannot be implemented due to U.S. Army Cost Benefit Analysis Guide - V 1.0 15 factors beyond the control of the implementing organization. An alternative is feasible only when it satisfies all the restrictions. Quick Review Assumptions are statements used to describe conditions over which the organization has no control and which are essential to the success of a given solution. Constraints are factors that limit the number of potential alternatives (i.e. solutions to the problem statement). Constraints may come from outside the organization or may be established by the organization's leadership. . A CBA should formulate assumptions and identify constraints before defining alternatives. U.S. Army Cost Benefit Analysis Guide - V 1.0 16 o st? STEP 3 - Document the Current State (the Status Quo) This section discusses two areas: Define the Status Quo Alternative Document the Status Quo Alternative Define the Status Quo Alternative Defining the current state (hereafter referred to as the status quo) is the method of identifying system characteristics (current process or state of operations), users, and stakeholders, as well as problems with the current system. The definition should be detailed to a level where all stakeholders can understand and support conclusions drawn from the analysis. Additionally, definition of the status quo must be detailed enough to assign costs and link performance measures. The status quo alternative of the CBA is the \"baseline\" program or system against which the costs and benefits of all feasible alternatives are compared. The status quo is the existing operational capability as of the program start date. All expenditures (such as managing, supporting and maintaining day-to-day operations) for the existing operational capability will be included in the status quo cost estimate. The cost estimate for the status quo is an extrapolation of the current level of costs and effectiveness that would accrue without changes. Document the Status Quo Alternative Identify and document the status quo alternative and all resources that have been required to meet the mission objective. Some potential sources of documentation are historical Government/contractor cost data, programmatic, financial and budgetary data/reports, tables of distribution and allowances (TDA), tables of organization and equipment (TOE), and modernization plans. Other sources are audit reports, operating procedures, field manuals, and Army publications. Review procedures and identify tasks and critical decision points within all appropriate organizations. Note that the parameters identified for the status quo must directly relate to, or closely parallel, those defined by the statement of objectives. If enhancement of the status quo to meet all or part of the objective is an alternative, estimate costs for the enhanced status quo as an alternative, in addition to the status quo. The cost of operating the status quo until the new system or project is fully operational (known as parallel operations) will be a part of the cost of all other alternatives in the cost-benefit analysis. These costs are referred to as Phase-out costs. U.S. Army Cost Benefit Analysis Guide - V 1.0 17 A cost benefit analysis that does not include the status quo (with applicable cost estimates) must be fully justified to the organizations reviewing the documentation. Usually, the status quo alternative is used to compare costs with other alternatives and to determine the quantifiable benefits. Without the status quo costs, it is very difficult to evaluate the benefits associated with the new program. Where a status quo exists, omitting it from the cost benefit analysis will reflect negatively upon the analysis and the credibility of realizing any proposed quantifiable benefits. Generally, the only time that a status quo does not exist is when a solution is being proposed to address a new requirement or mission. When programs/projects are totally new (new start) to the Army, there are no benefits, financial or otherwise, compared to the status quo. In such a case, the alternative with the lowest nonrecurring investment cost will be used as the basis of comparison with other alternatives. Quick Review The current process and/or situation presents the case and helps establish expectations for what is to follow. The current state must be developed enough to understand the impact the alternatives will have on it. The current state or status quo is the standard that will be used to evaluate alternatives. When a comparison is made between the current state and the future state (where you want to be), the status quo allows for the identification of shortcomings which the CBA should address. U.S. Army Cost Benefit Analysis Guide - V 1.0 18 STEP 4 - Define Alternatives with Cost Estimates This section discusses three areas: Define Alternatives The Cost Analysis Process Data Sources Organizing Cost Data Define Alternatives CBA alternatives should reflect a review of the mission and strategic goals to verify that the alternative's objectives are still valid and have not been overcome by events or changed by legislation or administrative direction. The status quo alternative is always the first alternative. As stated in Step 3, the status quo alternative of the CBA is the \"baseline\" program or system against which the costs and benefits of all feasible alternatives are compared. The number of alternatives can be controlled by avoiding similar but slightly different alternatives (variations on a theme) and by early elimination of non-viable alternatives. The reasons for eliminating potential alternatives should be included in the CBA documentation. Some of the criteria used as a basis for eliminating non-viable alternatives are listed below. Unacceptably high cost Non-compliance with cost and benefit analysis guidance Lack of compliance with established constraints Dependence on assumptions that are considered unrealistic Non-compliance with law, regulations and/or policy (not only acquisition) Unacceptable performance Inability to meet Initial Operation Capability (IOC) or full operational capability (FOC) requirements Political considerations such as environment, world opinion, treaty compliance, etc. Because each project requiring a cost benefit analysis is different, the evaluator will have questions and concerns which impact specific aspects of that particular project. The following list of questions was developed as a tool to assist in the preparation, review, and validation of cost benefit analyses: Have all feasible alternatives been considered? Is the status quo presented as an alternative? If not, this needs to be explained in the documentation. U.S. Army Cost Benefit Analysis Guide - V 1.0 19 Are all alternatives presented feasible? Are the alternatives distinctly different, rather than restructuring a single course of action? Have the alternatives that were eliminated from the analysis been clearly identified and has a rationale been provided for their elimination? If other Government organizations can provide the desired product or service, have they been identified as alternatives? If the project increases productive capacity, has a contracting alternative been examined? Are the alternatives well defined? If the alternatives overlap one another, are there sufficient differences between them to make them distinctly different, or are they just variations on a theme? The Cost Analysis Process Cost analysis is a critical element in the CBA process. The cost estimates support management decisions by translating resource requirements (equipment and personnel) associated with programs, projects, or processes, into dollar values. A systematic approach is necessary to develop accurate and timely cost estimates. The figure below shows the steps in completing a cost estimate. Cost estimating is an iterative process that may require reevaluating previous steps. Cost Analysis Process CES = Cost Element Structure U.S. Army Cost Benefit Analysis Guide - V 1.0 20 Cost Analysis Preparation Preparation includes knowing the purpose of the estimate, understanding the program/system and establishing a plan to complete the estimate. The purpose of the estimate is to evaluate alternative courses of action. Once the purpose is understood, it is important to agree on the end product (deliverable) to the customer. The following are examples of documents that could be used to understand program requirements and their material solution: JCIDS documents/memo's outlining requirements (CDD, ICD, CPD etc) Operational Requirement Document (ORD) Acquisition Decision Memorandum (ADM) System Training Plan (STRAP) Manpower Estimate Report (MER) Test and Evaluation Master Plan (TEMP) Acquisition Strategy (AS) Acquisition Program Baseline (APB) Selected Acquisition Report (SAR) Defense Acquisition Executive Summary (DAES) Technical definition, characteristics, design features, and technologies Cost Analysis Requirements Description (CARD) Risk assessment Schedule Once all available documents are reviewed, analysts should meet with subject matter experts (program office and contractor) to review and clarify any questions they may have. Ground Rules and Assumptions (GR&A) GR&A establish the boundaries of the estimate by clarifying what the estimate includes and what it excludes. It can also be used to highlight issues of importance to decision makers. The cost estimator should work with the technical team to establish GR&A. Listed below are examples that should be established in the GR&A. Whether the financial data is shown in constant or current dollars and, if in constant dollars, what base year was used in the estimate Inflation indices used Scope of the estimate Maintenance concept U.S. Army Cost Benefit Analysis Guide - V 1.0 21 Training strategy Support concept Acquisition strategy Procurement/fielding schedules Sparing concept Long lead items/procurement lead time Quantity of development units or prototype units Fee structure Development and production, O&S start and stop dates Commonality among components and other systems Technology assumptions Hardware refresh cycle Software assumptions Specific items or costs excluded from the cost estimate Government Furnished Equipment (GFE) Sunk costs Funding appropriation(s) for potentially contentious items Data collection and analysis This step includes the process of identifying, collecting, and analyzing data before applying cost estimating within the analysis process. Prepare a formal data collection plan, which may consist of the following data collection tasks: Prepare a data collection schedule/plan Identify the types of data needed (e.g., cost, programmatic, schedule, technical) Determine and locate sources of data Collect cost data and program documentation Determine the sample size of data to be collected for each cost element Determine which estimating methods, tools, and models will be used with which data sets Verify, validate and adjust (normalize) the data Collect data continuously throughout the pre-cost estimating process Adjusting for inflation is the most common form of normalization of data. It is also important for predicting annual budget requirements for funding multi-year activities, analyzing program alternatives (for a cost benefit analysis), and normalizing data for other uses. When adjusting U.S. Army Cost Benefit Analysis Guide - V 1.0 22 for inflation, make certain all dollar/cost data is adjusted in the same way so that it is comparable. In all cases, inflation should be applied by using OMB inflation rates, which are available at: http://asafm.army.mil/Documents/OfficeDocuments/Budget/Guidances/inflate//indices.xls. Work Breakdown Structure (WBS) A work breakdown structure defines in detail the work necessary to accomplish an initiative/proposal's objectives. A typical WBS reflects the requirements, what must be accomplished to develop the initiative/proposal, and provides a basis for identifying resources and tasks for developing a cost estimate. A WBS deconstructs an initiative/proposal's output (deliverables) into successive levels with smaller specific elements (cost elements) that can be analyzed. Cost elements are the lowest level of a cost estimate, and the cost estimate total is the sum of all the cost elements. A well-developed cost element structure helps ensure that no costs are missed and that there is no double counting. For example, a common cost element is personnel costs. This element can be further broken down into military and civilian personnel costs which in-turn can be analyzed as to what grade or rank makeup these two cost subcategories. Cost Estimate Once the GR&A are established, the data has been collected and analyzed, and the WBS structure established; it is time to build the estimate. Keep in mind that this is an iterative process and the GR&A data and WBS need to be continually reviewed to see if changes are needed. Normally, a cost estimate contains all costs from the start through implementation, operation, and disposal for a program or project. Collectively, these costs are the life cycle cost (LCC). The Army LCC is phased by the five appropriation groupings ??? Research, Development, Test and Evaluation (RDT&E), Procurement, Military Construction (MILCON), Military Personnel, and Operations and Maintenance (O&M)-each of which has its own inflation indices. Identify those costs which are recurring and those costs that are one time only. It is important not to confuse the two as it may lead to double counting which in-turn will cause costs to be overstated. Accuracy/Reasonableness Checking the cost estimate for reasonableness will help identify potential errors and highlight cost estimating methodologies that may need to change. For instance, if contractor system engineering and program management (SEPM) cost doubles from one year to the next there may be a valid reason it increases; there could be an error in the estimate; or you may be using a Cost Estimating Relationship (CER) or factor that does not reflect what is going on in the U.S. Army Cost Benefit Analysis Guide - V 1.0 23 Program. The analyst needs to review each WBS element and ask the question, \"Based on everything known about this program/product, does the cost of this WBS element pass the common sense test (i.e. should it be costing this much by year)?\" Data Sources Below is a list of some potential data sources for cost estimates. Regardless of the nature of the data used, identify the source and date of the data in the documentation of the CBA. Army material Command (AMC) Major Subordinate Commands (MSCs) Budget and Program Objective Memorandum (POM) submission Contract performance data Contractors and manufacturing plant visits Cost Analysis Requirements Document (CARD) Cost libraries Historical cost data reports Management Decision Evaluation Package (MDEP) Manpower utilization records/reports ODASA-CE Program Management Offices (PMOs) Program Office Estimate (POE) Research Development and Engineering Centers (RDECs) Subject matter experts Trade Studies In addition to evaluating available data for its utility in cost estimating, the analyst must look for relationships among data. A basic premise is that relationships among data may continue to exist in the absence of known facts and conditions. The presence of these relationships provides the analyst with indicators that can form the basis for assumptions, cost factors, and CERs. Cost factors and CERs may be expressed in dollars, physical quantities, ratios, or percentages. Various methods may be used to develop them; whatever method is chosen should be relevant, valid, verifiable, and reasonable. U.S. Army Cost Benefit Analysis Guide - V 1.0 24 Cost Strategy The expected approach to costing is to ensure that an alternative under consideration is fully costed. The question of cost is separate from and must precede the question of budget. The cost question is: What is the full cost of the alternative? The budget question is: What impact will the alternative have on the budget? For example, an analyst is told to prepare a cost estimate for the establishment of a new maintenance facility to be staffed by 50 Soldiers and 50 Civilians. The analyst is further told that all Soldiers will transfer from other units and of the 50 Civilians, 30 will come from existing allocations. This means that the remaining 20 are brand new hires. The cost of this initiative includes the funding for 50 Soldiers and 50 Civilians. But budget impact is limited to the cost of the 20 Civilian new hires, because the costs of the 50 Soldiers and 30 Civilians are already reflected in the budget. The budgetary impacts will be addressed in Step 7 of this Guide under the heading called \"Define Trade-Offs and Billpayers.\" Organizing Cost Data When an analyst is organizing and evaluating cost data, it may be helpful to build tables for identifying and aggregating costs. Using tables to display costs will also help identify those costs that will require trade-offs (which is discussed in Step 7 of the Guide), particularly costs that appear in the years of execution (current year and budget year before the next POM). These tables may also be used to prepare briefing charts for decision makers (which is discussed in Step 8 of the Guide). The example below is a simple table that displays cost elements along the left side of the table and years which the cost analysis covers across the top. The analyst must determine the specific time period the CBA covers (e.g. the execution and POM years or something longer). The analyst would create a table for each alternative. While the focus of this Step (6) is on costing alternatives, a summary table may be built for use in Step 7 which will include both the costs and quantifiable benefits for each alternative to facilitate their comparison. The analyst can insert formulas that include the effects of inflation as well as discounting on the cash flows. The structure and content of the table are primarily influenced by the CBA itself and the needs of the decision maker and/or analyst. U.S. Army Cost Benefit Analysis Guide - V 1.0 25 Example of a table that aggregates cost by cost element and by year. Alternative A Cost Elements Personnel Facilities Equipment Contracts Training N MILCON o Maintenance t Supplies e Etc. : Etc. Etc. A TOTAL Time Period FY FY FY FY FY FY FY FY 2010 2011 2012 2013 2014 2015 2016 2017 Total The cost elements shown to the left reflect some possible ones/ideas and not what must be used. The analyst should consider using more specific cost elements if possible. For example, if an initiative will be staffed with both military and civilian personnel, then show the break down between them. The cost elements selected will depend on the cost data used in the CBA. \" A \"Best Practice\" would be to organize cost elements in the table by appropriation type (MPA, OMA, OPA, MILCON, and etc.) if possible. Quick Review All reasonable ways of satisfying the objective should be documented and discussed. Alternatives dismissed as infeasible should be noted in the backup CBA documentation. The quality of the solution to the problem statement is no better than the alternatives used in the CBA. Therefore, generating alternatives is an important step in the process of preparing a CBA. Alternatives do not need to be functionally identical as long as they fulfill the objective. Data is the foundation of every CBA. How good the data is affects the CBA's overall credibility. The data plan supports collection of the necessary data. Knowing the things that influence an alternative's costs and benefits will help you in capturing the right data. Data collection can be a lengthy process and continues throughout the development of the cost estimate. Emphasis should be placed on gathering data that demonstrates the costs and benefits of the identified alternatives. The analyst should acquire the most recent data available. Old data may be dated/obsolete. It is common practice to adjust your data through a process called normalization, which is ensuring that the data is consistent (e.g. keeping units the same $/hr vs. $/Day - use one or the other, not both, in the analysis). Differentiate the nonrecurring (one- time costs) and recurring costs. U.S. Army Cost Benefit Analysis Guide - V 1.0 26 STEP 5 - Identify Quantifiable and Non-Quantifiable Benefits This section discusses five areas: Benefits Analysis Overview Types of Benefits Identify, Estimate, and Evaluate Benefits Quantitative and Qualitative Benefit Categories Benefits Analysis Overview Benefits are results expected in return for costs incurred for a chosen alternative. They are the quantitative and qualitative improvements expected or resulting from the implementation of a project/initiative (which may include but are not limited to the following: equipment, facilities, hardware, systems, etc.). The following definitions or measurements describe benefits: effectiveness, physical yield, products, morale, quality of life, and timeliness. Benefits are either quantifiable or nonquantifiable. The purpose of benefit analysis is to identify, measure, and evaluate the benefits of proposed alternatives. Benefits may be expressed in non-dollar terms such as improved performance, operational effectiveness, or more rapid fielding of equipment. When preparing a CBA identify all benefits, whether quantifiable or non-quantifiable. Benefits justify the costs identified in the CBA. Where possible, benefits should be quantified in dollar values. Benefits not assigned a dollar value can sometimes be quantified in other terms. All included benefits must be relevant to the analysis. Each benefit must be clearly and distinctly identifiable, and should not duplicate or overlap any other measure. This evaluation will determine the level of the risk to the Army if the alternative is not implemented, partially implemented, or delayed. Benefits should be framed in the context of how they support the Army in meeting its missions, functions and responsibilities. The benefit analysis should consider the DOTMLPF construct (Doctrine, Organization, Training, Materiel, Leader and Education, Personnel, and Facilities). U.S. Army Cost Benefit Analysis Guide - V 1.0 27 Types of Benefits Quantifiable Benefits Quantifiable benefits are benefits that can be assigned a numeric value such as dollars, physical count of tangible items, or percentage change. Financial benefits are always quantifiable and are measured in dollars. o Cost reduction. A reduction in the number of dollars needed to meet a customerestablished requirement by improving a process or function. o Savings. A cost reduction that permits a manager to remove dollars from the program or budget. o Cost avoidance. Any cost reduction that is not savings. o Revenue generation. An increase in the dollars that flow into the Army, over and above appropriated funds, or over and above the expected amount of customer funding received through a revolving fund. o Productivity improvements. A reduction in personnel time and effort requirements associated with a function or assigned task. In most cases, a productivity improvement will also result in a savings or cost avoidance. Examples of other quantifiable benefits and methods of measurement include but are not limited to: Increased number of commodities or items produced for each alternative (such as the number of meals served, hours flown, or components manufactured). Increased number of items produced per a given period of time (such as flight hours per month, number of items per man-hour, or number of trucks serviced per year). Improved system reliability in terms of reduction to its probable failure ratio (such as mean-time-between-failure, or number of repairs per item per year). Reduced number of errors per operating cycle or period (such as the number of errors per card punched, errors per 100 records, or errors per 100 items produced). Improved maintainability/supportability measures (such as increased mean-time-torepair or reduced average downtime). Improved accuracy, timeliness, and completeness of data produced by a system, resulting in efficient utilization of the Army's resources through more effective decisions made upon more accurate data. Improved performance and operational effectiveness. U.S. Army Cost Benefit Analysis Guide - V 1.0 28 Non-Quantifiable Benefits Some benefits do not lend themselves to direct, quantitative measures. These benefits, though difficult to assess, should be addressed qualitatively. Although subjective in nature, qualitative statements can make a positive contribution to the analysis. Despite being difficult to assess, the CBA preparer should attempt to use best analytical practices in order to include nonquantifiable benefits in the analysis. Some examples of non-quantifiable benefits are improved morale, compatibility, improved quality and security, and improved readiness. Military Benefits Analysis (MBA) Military worth or Military Benefits Analysis (MBA) of new concepts demonstrates value or a technology payoff to the warfighter. Systems or systems analysts traditionally conduct MBA to evaluate the warfighter benefits resulting either from new asset development and implementation or from the establishment of new employment concepts for existing assets. The payoff or benefit must consider a clear interest of the user community in making an informed investment decision; therefore, determining the MBA for particular technologies is vitally important. The scope ranges from the campaign to mission level and thus differs in magnitude, time frame, and level of detail. MBA typically includes parameters such as time to accomplish objectives, number of targets neutralized, amount of collateral damage, and volume of resources consumed (including dollars). It includes facilities, maintenance, resource reduction, and other parameter considerations. If a new concept is similar to an existing in its performance and use, analysts can easily employ existing MBA tools and approaches to establish the concept weapon's warfighter benefit. However, most new concepts are radically different from existing. New concepts call for additional varying parameters and metrics. Evaluating the new concept military benefit is becoming increasingly difficult because existing analytical tools and techniques were not built to address these complex applications. New concepts, constructs, technologies, and other developments cause the need for cooperative systems designs with multiple parameters sets against multiple mobile targets and intercepts while operating autonomously, cooperatively, and synchronously. These require high level analysis using stochastic, non-parametric, inferential, and statistical approaches including propagations of Markovian processes. Effective MBA mission level constructs provide detailed insight to the scaled scenarios generated by compressed time, weapons effects (e.g., expected kills per _____), weapon's performance, and desired warfighter outcomes in a controlled space. MBA performed at the mission level effectively provides analysts with insight not attainable at the campaign level. Performing this dual (campaign and mission level) approach gives clarity for appropriate U.S. Army Cost Benefit Analysis Guide - V 1.0 29 comparisons to refine parameters to the expected benefit range. This process enables interpretation of the benefit by providing a potential performance picture. Force structure, force mix (array), target mix, and performance in designated scenarios (must be operationally relevant) define and shape the warfighter benefits. MBA is crucial for evaluating the warfighter technology benefits and, consequently, for making informed technology investment decisions. As a result, MBA capability to support investments decisions is a necessary and sufficient collaborative tool for future technologies and integrating concept acquisition. Identify, Estimate, and Evaluate Benefits All significant benefits must be included in the benefit analysis portion of the CBA, whether quantifiable or non-quantifiable. Benefits that cannot be quantified should be described in narrative form. Ensure that the benefits are validated by the functional proponent (or the organization responsible for the basic requirement) and coordinated with all appropriate activities. It is strongly recommended that identification and documentation of benefits begin early in the evaluation process. Identifying Benefits The following steps are recommended to identify benefits and establish quantitative measures for benefits where possible. Identify all resources flowing into the project and the resulting benefits flowing out of the project. Determine and list the benefits of each alternative, both quantifiable and nonquantifiable. Define each benefit in relation to the alternatives in the CBA. All benefits included must be relevant to the analysis. Each benefit must be clearly and distinctly identifiable from all other benefits; it should not duplicate or overlap any other measure. Develop a quantitative measure for each benefit where possible. This will allow direct comparison of alternatives for each benefit. Be consistent. Benefits should not be evaluated one way for one alternative, and a different way for another alternative. U.S. Army Cost Benefit Analysis Guide - V 1.0 30 Benefit Categories The following list of categories may help define benefits. This list is not all inclusive, nor is it intended to provide precise definitions of the benefits listed. It is only meant to be illustrative of benefits categories that could be applicable to program objectives. Acceptability - Does the alternative contribute to the operation of parallel or higher level organizations? Does it improve the quality of the process? Accuracy - Does the alternative reduce error rates or improve the accuracy of information? Adaptability - Is the system/project adaptable to existing DoD, industry, national, or international standards? Availability - When can the system/project be delivered or implemented; when is it needed to meet proposed output schedules? What is the mean time between failures? Functionality - How well does the system perform; how quickly can it process data or calculations, or other functions? Compatibility - How will existing operations, facilities, equipment, data requirements be affected? How much initial training will be required? How will work methods and procedures be altered? Maintainability - Is the system difficult to repair? Are parts readily available? How much staff will be required to maintain the software/hardware? What is the anticipated down time for maintenance? Is the maintenance downtime longer for any alternative? Manageability - Will the system/project decrease the involvement/need for supervisors or quality inspections? Will a different type of personnel than currently assigned be required? Are trained personnel available? Morale - will the system/project contribute to a positive employee attitude towards work? Production - Will the number of products produced be increased? Productivity - Will the rate of production increase? Will the system/project decrease the number of staff resources previously needed to produce the same product, or will the system/project allow more items to be produced with existing staff resources? Quality - Will a better product be produced? Will better service be provided? Will quality of products be more consistent? Is customer satisfaction improved? Reliability - how many (how often) system failures will occur over time? Security - Will more or less precautions be needed? Service life - How long will the equipment be able to support the operation? Will the equipment be obsolete before it reaches the end of its useful life? Upgradeability - How compatible will additional equipment, such as memory, terminals, workstations, or other equipment, be with existing equipment or users of the system? Versatility - Will the equipment in any alternative provide additional capacity or capability beyond that required for the system? U.S. Army Cost Benefit Analysis Guide - V 1.0 31 Estimating Quantifiable Benefits Every effort should be made to quantify benefits to the maximum extent possible. Sub-divide quantifiable benefits into those that are dollar quantifiable and those that are quantifiable in other terms. The methods of measurement for quantifiable benefits are as follows, in order of desirability: Dollar quantifiable terms. Physical count of tangible items (for example, units of output). Index or ratio (for example, 40 percent or greater). The benefit estimating process is similar to that for cost estimating discussed in Step 4. Data must be collected from appropriate sources and analyzed; relationships among data must be identified; inflation and discounting must be applied to annual dollar values via standard methods. Cost estimates should apply inflation indices and then benefits should be computed by comparing the status quo (with applied inflation indices) with the cost of the alternative(s). The economic life (the period during which the alternative provides benefits) of the alternatives and the fiscal years (FY) when benefits accrue must be carefully considered. Identify all benefits by the appropriation and the FY in which they are expected to occur. Upon decision approval, savings in the year of execution and budget year shall be retained by the command. Savings in the program years are considered in the Planning, Programming, Budget and Execution (PPBE) process. Savings beyond the Program Objective Memorandum (POM) period, as well as cost avoidances and productivity improvements, are treated differently. Consider the limitatio