Utility Project Worksheet Lens Model: Interpreting a Lens Model Table with One Cue Sum of absolute differences between outcome and judgment=divide by number of judgments and interpret (then use common sense) Outcome and Judgment Means---Compare judgment mean to outcome mean. If it is too low, increase average judgment. If too high, reduce average judgment Outcome and Judgment Standard Deviations---Compare to outcome standard deviation. If too low, increase spread. If too high, reduce spread Achievement Index---interpret sign (direction) and absolute value (strength) Linear Achievement Index --- compare to cue validity (which in this simple case is equal to linear outcome predictability). If linear achievement index is lower, it can be increased to the cue validity through improved consistency and improved linear cue usage. (In this simple case, linear achievement index is equal to the product of cue validity and cue utilization coefficient.) Nonlinear Achievement Index ---If over 10, determine whether it is due to luck, cheating, use of an external cue, and/or using the cue in a nonlinear fashion which matches how it relates to the outcome. Linear Outcome Predictability (How good are the cues?) --Because there is only one cue, this is equal to the cue validity. If low, improve by adding new, useful cues. Linear Judgmental Predictability (How consistent was the judge?) --- Because there is only one cue, this is equal to the cue utilization coefficient. If low, improve through greater consistency in using the cue. Knowledge Lincar (How well did the judge use the cues?)--In this simple case of one cue, knowledge linear match is always equal to on one Nonlinear - Ignore. Cue Validities and Cue Utilization Coefficients--match cue usage to cue validities in terms of both sign and magnitude arch 1. Enter yourjudgments and complete the table below: Month January Average High Temperature 63 Actual Electric Bill $76 Your Judgment $__ $. $. February 66 $ 62 March 73 $ 62 April 80 $ 54 $ May 86 $114 $ June 91 $165 $168 94 $. 95 $180 July August September October 90 $139 $__ $_ $__ 82 $106 November 73 $ 84 $ December 64 $ 54 2. Complete the graphs below and estimate cue utilization coefficient and achivement indexfrom the graphs Graph of Judgments vs. Temperature Cue 250 200 150 100 50 0 0 10 20 30 40 50 60 70 80 90 100 Estimate of Cue Utilization coefficient (Correlation between Judgments and Temperature Cue) 250 200 150 100 50 0 50 100 150 200 250 Estimate of Achievement Index (Correlation Between Judgments and Actual Bills) 3. Using Excel, compute the values for the table below. Lens Model Table: Utility Bill Estimation Task Number of bills correctly predicted- Sum of absolute differences between outcome and judgment= Outcome Mean= 105.33 Outcome Standard Deviations 47.30 Judgment Mean= Judgment Standard Deviation= Achievement Index (correlation between judgments and actual bills) Linear Achievement index (Cue Validity Cue Utilization coefficient) Nonlinear Achievement Index(Achievement Index - Linear Achievement Index) Cue Validity = .89 Cue Utilization (correlation between Judgment and Temperature) Edit Styles 4. a) Using the cue utilization coefficient and achievement indices that you computed using Excel, evaluate the accuracy of your scatter plot estimates of cue utilization coefficient and achievement index. Cue Utilization Coefficient Estimated from scatter plot Computed using Excel= Achievement index Estimated from scatter plot) Computed using Excel=_ b) if they were inaccurate, how did you misjudge the scatter plot(s)? Cue Utilization coefficient Achievement Index 5. a) Report the values and assess your accuracy on each of the following four dimensions: sum of absolute differences ii) achievement index ill) mean iv) standard deviation D Focus 00 b)if the nonlinear achievement index is greaterthan.1, is it due to luck, cheating, using a cue not included in the analysis, and/or to using the cue in a nonlinear fashion (which matches the cue's relationship to the outcome)? c) For any problem dimension (sum of absolute differences, achievement index, mean, and standard deviation), identify a corresponding way to improve your accuracy. sum of absolute differences II) achievement index ii) mean iii) mean iv) standard deviation d) Compute and report the autocorrelation for actual electric bill? Styles Sample Results for Electric Bill Estimation Task Given the average monthly high temperatures below, the 2017 electric bills for a 2200 square foot house built in 1985 were estimated. The house is air conditioned with two central air conditioning units which are controlled by setback thermostats. The water heater and the central heating units are powered by natural gas. Two adults occupy the house and work full time. The historical average high temperatures are from the period 1980-2010. Historical Actual Average High Electric Judge X's Month Temperature Bill Judgment January 63 $ 76 $100 February 66 $ 62 $115 March 73 $62 $120 April 80 $ 54 $125 May 86 $114 $145 June 91 $165 $150 July 94 $168 $160 August 95 $180 $160 September 90 $139 $140 October 82 $106 $130 November 73 $ 84 $120 December 64 $ 54 $115 Lens Model Table: Utility Bill Estimation Task Number of bills correctly predicted=0 Sum of absolute differences between outcome and judgment=402 Outcome Mean= 105.33 Outcome Standard Deviation=47.30 Judgment Mean=131.67 Judgment Standard Deviation19.23 Achievement index (correlation betweenjudgments and actual bills).90 Linear Achievement Index (Cue Validity XcCue Utilization coefficient)= 86 Nonlinear Achievement index (Achievement Index - Linear Achievement Index)=.04 Cue Validity = .89 Cue Utilization (correlation between Judgment and Temperature)=.96 Utility Project Worksheet Lens Model: Interpreting a Lens Model Table with One Cue Sum of absolute differences between outcome and judgment=divide by number of judgments and interpret (then use common sense) Outcome and Judgment Means---Compare judgment mean to outcome mean. If it is too low, increase average judgment. If too high, reduce average judgment Outcome and Judgment Standard Deviations---Compare to outcome standard deviation. If too low, increase spread. If too high, reduce spread Achievement Index---interpret sign (direction) and absolute value (strength) Linear Achievement Index --- compare to cue validity (which in this simple case is equal to linear outcome predictability). If linear achievement index is lower, it can be increased to the cue validity through improved consistency and improved linear cue usage. (In this simple case, linear achievement index is equal to the product of cue validity and cue utilization coefficient.) Nonlinear Achievement Index ---If over 10, determine whether it is due to luck, cheating, use of an external cue, and/or using the cue in a nonlinear fashion which matches how it relates to the outcome. Linear Outcome Predictability (How good are the cues?) --Because there is only one cue, this is equal to the cue validity. If low, improve by adding new, useful cues. Linear Judgmental Predictability (How consistent was the judge?) --- Because there is only one cue, this is equal to the cue utilization coefficient. If low, improve through greater consistency in using the cue. Knowledge Lincar (How well did the judge use the cues?)--In this simple case of one cue, knowledge linear match is always equal to on one Nonlinear - Ignore. Cue Validities and Cue Utilization Coefficients--match cue usage to cue validities in terms of both sign and magnitude arch 1. Enter yourjudgments and complete the table below: Month January Average High Temperature 63 Actual Electric Bill $76 Your Judgment $__ $. $. February 66 $ 62 March 73 $ 62 April 80 $ 54 $ May 86 $114 $ June 91 $165 $168 94 $. 95 $180 July August September October 90 $139 $__ $_ $__ 82 $106 November 73 $ 84 $ December 64 $ 54 2. Complete the graphs below and estimate cue utilization coefficient and achivement indexfrom the graphs Graph of Judgments vs. Temperature Cue 250 200 150 100 50 0 0 10 20 30 40 50 60 70 80 90 100 Estimate of Cue Utilization coefficient (Correlation between Judgments and Temperature Cue) 250 200 150 100 50 0 50 100 150 200 250 Estimate of Achievement Index (Correlation Between Judgments and Actual Bills) 3. Using Excel, compute the values for the table below. Lens Model Table: Utility Bill Estimation Task Number of bills correctly predicted- Sum of absolute differences between outcome and judgment= Outcome Mean= 105.33 Outcome Standard Deviations 47.30 Judgment Mean= Judgment Standard Deviation= Achievement Index (correlation between judgments and actual bills) Linear Achievement index (Cue Validity Cue Utilization coefficient) Nonlinear Achievement Index(Achievement Index - Linear Achievement Index) Cue Validity = .89 Cue Utilization (correlation between Judgment and Temperature) Edit Styles 4. a) Using the cue utilization coefficient and achievement indices that you computed using Excel, evaluate the accuracy of your scatter plot estimates of cue utilization coefficient and achievement index. Cue Utilization Coefficient Estimated from scatter plot Computed using Excel= Achievement index Estimated from scatter plot) Computed using Excel=_ b) if they were inaccurate, how did you misjudge the scatter plot(s)? Cue Utilization coefficient Achievement Index 5. a) Report the values and assess your accuracy on each of the following four dimensions: sum of absolute differences ii) achievement index ill) mean iv) standard deviation D Focus 00 b)if the nonlinear achievement index is greaterthan.1, is it due to luck, cheating, using a cue not included in the analysis, and/or to using the cue in a nonlinear fashion (which matches the cue's relationship to the outcome)? c) For any problem dimension (sum of absolute differences, achievement index, mean, and standard deviation), identify a corresponding way to improve your accuracy. sum of absolute differences II) achievement index ii) mean iii) mean iv) standard deviation d) Compute and report the autocorrelation for actual electric bill? Styles Sample Results for Electric Bill Estimation Task Given the average monthly high temperatures below, the 2017 electric bills for a 2200 square foot house built in 1985 were estimated. The house is air conditioned with two central air conditioning units which are controlled by setback thermostats. The water heater and the central heating units are powered by natural gas. Two adults occupy the house and work full time. The historical average high temperatures are from the period 1980-2010. Historical Actual Average High Electric Judge X's Month Temperature Bill Judgment January 63 $ 76 $100 February 66 $ 62 $115 March 73 $62 $120 April 80 $ 54 $125 May 86 $114 $145 June 91 $165 $150 July 94 $168 $160 August 95 $180 $160 September 90 $139 $140 October 82 $106 $130 November 73 $ 84 $120 December 64 $ 54 $115 Lens Model Table: Utility Bill Estimation Task Number of bills correctly predicted=0 Sum of absolute differences between outcome and judgment=402 Outcome Mean= 105.33 Outcome Standard Deviation=47.30 Judgment Mean=131.67 Judgment Standard Deviation19.23 Achievement index (correlation betweenjudgments and actual bills).90 Linear Achievement Index (Cue Validity XcCue Utilization coefficient)= 86 Nonlinear Achievement index (Achievement Index - Linear Achievement Index)=.04 Cue Validity = .89 Cue Utilization (correlation between Judgment and Temperature)=.96