Calculating the Cost Benefit of Training Helpful Information
In your text, you will find the explanations for each of the items in the equation and some of the information to use in the calculation on p. 248, and the rest of the information is on 249-250. It it difficult in the text, because the pages are not facing each other, so my suggestion is to take a picture of the page with the explanations on your phone, or write them out, and the go to the page with the scenario, and fill in the actual information from the scenario for each item in the equation. See attached resource for assistance. I think this is a helpful way to accomplish this. I am not worried about everyone hitting the exact calculation - I am more concerned about students being able to put the information together by following each step, and for students to realize the various cost considerations, so just do your best.
Utility Analysis The results of an evaluation study often express the effect of an HRD program in terms of a change in some aspect of the trainee's performance or behavior. For example, if untrained employees average 22.5 units produced (per day or per hour) and trained employees average 26 units produced, the gain due to training is 3.5 units per employee. Utility analysis provides a way to translate these results into dollar terms. For example, Thornton and Potemra estimated that the utility of an assess- ment center procedure to promote police officers to the rank of sergeant was over $400,000.144 One popular approach to utility analysis is the Brogden- Cronbach-Gleser model.145 This model computes the gain to an organization in dollar terms AU, or "change in utility," using the following variables: N = Number of trainees = Length of time the benefits are expected to last di = An effect size, which expresses the true difference of job performance between the trained and untrained groups (expressed in standard devi- ation units) SD, = Dollar value of job performance of untrained employees (expressed in standard deviation units) C = Costs of conducting the training Wayne Cascio combined these elements into a formula to compute the dollar value of improved performance due to training.146 The left side of the equation esti- mates the benefits of training, while the right side presents the cost. The formula is AU = (N)(1)(d)(SD) - T Some terms in the equation can be directly measured, such as N, C, and de, but others, such as T and SD, typically must be estimated. More complicated versions of this formula have been developed to account for other factors that may affect the real monetary value of the benefits accrued, such as attrition and decay in the strength of training effects over time. 17 Cascio suggests a method for incorporating the results of utility analysis into cost-benefit analysis for training and HRD programs. Drawing upon techniques of capital budgeting the three phases of Cascio's approach are as follows: 1. Compute the minimum annual benefits required to break even on the program (e.g, how much of a payback must the program generate in order to cover its costs?) 2. Use break-even analysis to determine the minimum effect size (d) that will yield the minimum required annual benefit (how much of an improvement in job per- formance must the trained employees show for the program to generate the pay- back needed to break even?) 3. Use the results from meta-analytic studies to determine the expected effect size and expected payoff from the program (what is the likely degree of improvement in job perfomance that the HRD program being proposed has shown in previ- ously conducted research on this program or method?) The goal of such cost-benefit analyses is to assist HRD professionals in demon- strating the expected gains of their programs, and compare these gains to either the gains from other programs or other potential investments (like the purchase of a new piece of equipment). Although the computational formulas for this approach are somewhat complex, Cascio points out that they can be computerized, thereby requiring only that the HRD manager or professional determine and input the values that correspond to each of the key parameters (such as costs, benefits, and effect size). Readers are urged to complete the utility exercise at the end of the chapter. While utility analysis can help translate the benefits of training programs into monetary terms, many training professionals have concerns about the practicality of such efforts. Further, some researchers question its value because of the nature of the estimates used to determine some of the factors in the formula." 149 It is also unclear to what extent HRD professionals use utility analysis to communicate the effectiveness of HRD programs. Given that utility analysis is intended to help man- agers see HRD programs as an investment and to make more informed decisions about HRD programs, it is reasonable to ask whether their decisions are influenced by utility estimates. Research on this question has produced mixed results. On one hand, Latham and Whyte found that managers are not influenced in the way HR practitioners would hope.150 They found that including utility analysis information actually reduced man- agers' support for a valid employee selection program. Similarly, Hazer and High- house observed that the degree to which managers accept the notion that the effectiveness of HR programs can be measured in terms of dollars remains an open question" (p. 110). On the other hand, when seeking to replicate the Latham and Whyte study, Carson and colleagues did not find any negative impact for including utility information.162 Additionally, Morrow, Jarret, and Rupinski reported that hav- ing a senior management team that: (1) is interested in a demonstration that HRD programs are a worthwhile investment and (2) pre-approves the utility model and procedures to be used will lead to acceptance of utility information as legitimate 193 Macan and Foster found that managers rank utility information highly for decision- making purposes, and reported a small, positive effect for including utility estimates as part of the information available to managers. 154 A recent study by Chocard and Davoine found strong positive utility values reported across ten different manage- ment training programs.156 Utility analysis (in addition to ROI and cost estimates) presents an opportunity for HRD professionals to provide information to decision-makers in monetary terms. However, simply providing managers with the dollar estimates generated by utility analysis will not by itself be sufficient to gain acceptance or use. 196 As with ROI estimates, gaining management acceptance appears to be a key consider- ation.157 Sturman proposed a number of modifications to the equations that are used to calculate utility estimates. He concluded his article by stating that "for a com- plex decision making tool to be useful, the users of the decision aid must desire the information it provides and be trained in its use" (p. 297). 168 Commenting on the Whyte and Latham results mentioned earlier, Sturman continued: "We should not be surprised that an individual untrained with a use of a decision aid fails to adhere to the results of the aid" (p. 297). 16 Toward that end, the following recommendations offered by various authors should increase the chances that management will accept and use utility information: Involve senior management when determining the utility model and procedures to be used Train HR professionals and managers in the details of utility analysis. Offer an explanation of the components of the utility model; focus on utility information as a communication tool to aid in decision making. Involve management in arriving at estimates. Use credible and conservative estimates. Admit that the results of utility analysis are often based on fallible but reason- able estimates. Use utility analysis to compare alternatives, rather than to justify individual programs. Finally, it is important to remember that not all decision-makers, and not all HRD programs, require justification in monetary terms. HRD professionals should find out from senior managers what they consider important when determining the value of HRD programs, and then provide management with information in those terms. 161 For some organizations, this may include the dollar value, while in others demonstrat- ing positive improvements in nonmonetary terms may be preferred. A lively debate conceming the inclusion of "values" (other than financial return) in HRD evaluation was presented in Human Resource Development Quarterly. 162 This ties back to the earlier discussion of the multiple stakeholders affected by HRD interventions. Inter- ested readers are encouraged to look further into these issues, as they again remind us of the ethical issues involved in all evaluation efforts. Next, an interesting situation is presented where researchers moved away from utility analysis and ROI calcula- tions, and still managed to provide meaningful organizational-level data on the impact of training (see the boxed insert "Measuring Organizational Results without Measur- ing ROI). Finally, the impact of technology on HRD evaluation is discussed 10 Utility Analysis The results of an evaluation study often express the effect of an HRD program in terms of a change in some aspect of the trainee's performance or behavior. For example, if untrained employees average 22.5 units produced (per day or per hour) and trained employees average 26 units produced, the gain due to training is 3.5 units per employee. Utility analysis provides a way to translate these results into dollar terms. For example, Thornton and Potemra estimated that the utility of an assess- ment center procedure to promote police officers to the rank of sergeant was over $400,000.144 One popular approach to utility analysis is the Brogden- Cronbach-Gleser model.145 This model computes the gain to an organization in dollar terms AU, or "change in utility," using the following variables: N = Number of trainees = Length of time the benefits are expected to last di = An effect size, which expresses the true difference of job performance between the trained and untrained groups (expressed in standard devi- ation units) SD, = Dollar value of job performance of untrained employees (expressed in standard deviation units) C = Costs of conducting the training Wayne Cascio combined these elements into a formula to compute the dollar value of improved performance due to training.146 The left side of the equation esti- mates the benefits of training, while the right side presents the cost. The formula is AU = (N)(1)(d)(SD) - T Some terms in the equation can be directly measured, such as N, C, and de, but others, such as T and SD, typically must be estimated. More complicated versions of this formula have been developed to account for other factors that may affect the real monetary value of the benefits accrued, such as attrition and decay in the strength of training effects over time. 17 Cascio suggests a method for incorporating the results of utility analysis into cost-benefit analysis for training and HRD programs. Drawing upon techniques of capital budgeting the three phases of Cascio's approach are as follows: 1. Compute the minimum annual benefits required to break even on the program (e.g, how much of a payback must the program generate in order to cover its costs?) 2. Use break-even analysis to determine the minimum effect size (d) that will yield the minimum required annual benefit (how much of an improvement in job per- formance must the trained employees show for the program to generate the pay- back needed to break even?) 3. Use the results from meta-analytic studies to determine the expected effect size and expected payoff from the program (what is the likely degree of improvement in job perfomance that the HRD program being proposed has shown in previ- ously conducted research on this program or method?) The goal of such cost-benefit analyses is to assist HRD professionals in demon- strating the expected gains of their programs, and compare these gains to either the gains from other programs or other potential investments (like the purchase of a new piece of equipment). Although the computational formulas for this approach are somewhat complex, Cascio points out that they can be computerized, thereby requiring only that the HRD manager or professional determine and input the values that correspond to each of the key parameters (such as costs, benefits, and effect size). Readers are urged to complete the utility exercise at the end of the chapter. While utility analysis can help translate the benefits of training programs into monetary terms, many training professionals have concerns about the practicality of such efforts. Further, some researchers question its value because of the nature of the estimates used to determine some of the factors in the formula." 149 It is also unclear to what extent HRD professionals use utility analysis to communicate the effectiveness of HRD programs. Given that utility analysis is intended to help man- agers see HRD programs as an investment and to make more informed decisions about HRD programs, it is reasonable to ask whether their decisions are influenced by utility estimates. Research on this question has produced mixed results. On one hand, Latham and Whyte found that managers are not influenced in the way HR practitioners would hope.150 They found that including utility analysis information actually reduced man- agers' support for a valid employee selection program. Similarly, Hazer and High- house observed that the degree to which managers accept the notion that the effectiveness of HR programs can be measured in terms of dollars remains an open question" (p. 110). On the other hand, when seeking to replicate the Latham and Whyte study, Carson and colleagues did not find any negative impact for including utility information.162 Additionally, Morrow, Jarret, and Rupinski reported that hav- ing a senior management team that: (1) is interested in a demonstration that HRD programs are a worthwhile investment and (2) pre-approves the utility model and procedures to be used will lead to acceptance of utility information as legitimate 193 Macan and Foster found that managers rank utility information highly for decision- making purposes, and reported a small, positive effect for including utility estimates as part of the information available to managers. 154 A recent study by Chocard and Davoine found strong positive utility values reported across ten different manage- ment training programs.156 Utility analysis (in addition to ROI and cost estimates) presents an opportunity for HRD professionals to provide information to decision-makers in monetary terms. However, simply providing managers with the dollar estimates generated by utility analysis will not by itself be sufficient to gain acceptance or use. 196 As with ROI estimates, gaining management acceptance appears to be a key consider- ation.157 Sturman proposed a number of modifications to the equations that are used to calculate utility estimates. He concluded his article by stating that "for a com- plex decision making tool to be useful, the users of the decision aid must desire the information it provides and be trained in its use" (p. 297). 168 Commenting on the Whyte and Latham results mentioned earlier, Sturman continued: "We should not be surprised that an individual untrained with a use of a decision aid fails to adhere to the results of the aid" (p. 297). 16 Toward that end, the following recommendations offered by various authors should increase the chances that management will accept and use utility information: Involve senior management when determining the utility model and procedures to be used Train HR professionals and managers in the details of utility analysis. Offer an explanation of the components of the utility model; focus on utility information as a communication tool to aid in decision making. Involve management in arriving at estimates. Use credible and conservative estimates. Admit that the results of utility analysis are often based on fallible but reason- able estimates. Use utility analysis to compare alternatives, rather than to justify individual programs. Finally, it is important to remember that not all decision-makers, and not all HRD programs, require justification in monetary terms. HRD professionals should find out from senior managers what they consider important when determining the value of HRD programs, and then provide management with information in those terms. 161 For some organizations, this may include the dollar value, while in others demonstrat- ing positive improvements in nonmonetary terms may be preferred. A lively debate conceming the inclusion of "values" (other than financial return) in HRD evaluation was presented in Human Resource Development Quarterly. 162 This ties back to the earlier discussion of the multiple stakeholders affected by HRD interventions. Inter- ested readers are encouraged to look further into these issues, as they again remind us of the ethical issues involved in all evaluation efforts. Next, an interesting situation is presented where researchers moved away from utility analysis and ROI calcula- tions, and still managed to provide meaningful organizational-level data on the impact of training (see the boxed insert "Measuring Organizational Results without Measur- ing ROI). Finally, the impact of technology on HRD evaluation is discussed 10
