Name: Lesson 3 Exercise A college professor is teaching a seminar on the topic of test and measurement. Realizing that knowledge of statistics is important to developing an understanding of the topic, the professor has decided to give a short quiz to assess his students' statistical skill. The quiz is composed of 10 items (questions). To test the reliability of his data, he decides to calculate a \"splithalf\" reliability for the data. The data appears in the Table 1 below. Table 1 Item 1 2 3 4 5 6 7 8 9 10 Total Score Bob 1 1 1 1 1 0 0 1 1 0 7 Susan 1 1 1 1 1 0 1 1 1 1 9 Students Eliot 1 1 1 1 1 1 1 1 1 1 10 Georgia 1 1 1 1 0 0 1 0 1 0 6 Brad 1 1 0 1 0 1 0 0 0 0 4 1. To check the internal consistency of the test, the professor does a splithalf procedure. The table below shows the two halfscores (even and odd). Calculate the mean, variance, and standard deviation for the two halfscores. Please use N in the denominator and not N1. (3 points) Scores for Half Tests Score for Even Half (X) 3 4 5 2 3 17 Name Bob Susan Eliot Georgia Brad Sum Score for Odd Half (Y) 4 5 5 4 1 19 Mean = Variance ( ) = 2 Standard Dev. ( ) = 2. Calculate the Pearson's r between the two half scores (3 points). Covariance of Even and Odd Halves of Test X X Name Bob Susan Eliot Georgia Brad Y Y xy rxy X X Y Y Spearman Brown Correction 3. The correlation coefficient you have just calculated is the estimated reliability for a fiveitem quiz, to find the reliability for the entire tenitem quiz, you must use the SpearmanBrown Formula. Kr org r rev 1 K 1 r org Where K is equal to the number of items in the revised test divided by the number of items on test the original reliability came from, and is the reliability rrevised coefficient for the revised test. In this case the original test is the five item test and the revised test is the tenitem test. Calculate the estimated reliability for the entire 10item quiz. Show your answer below. (2 points) 4. Would the estimate of reliability be the same if the first half of the test were correlated with the second half of the test? Explain your answer. (5 points) 5. Cronbach's Alpha. Cronbach's Alpha (or Coefficient Alpha) is another method for estimating reliability. The formula for alpha is: 2 K i 1 K 1 2 Where K is the number of items, 2 is the variance of total scores and i2 is the item variance for each item. Calculate the Coefficient Alpha for this quiz. (2 points) 6. KR20 Estimate of Reliability. When the items of a test are dichotomous (i.e. score either correct or incorrect) the KR20 formula is an appropriate substitute for the Coefficient Alpha. The KR20 formula for estimating reliability is: K pq KR20 1 K 1 2 Where K is the number of items on the test, p is the proportion of subjects getting an item correct, q is the proportion of subjects getting the same item wrong and 2 is the total variance for the test. Calculate the estimate of reliability using the KR 20 formula. (2 points) 7. KR21 Estimate of Reliability. The KR21 estimate of reliability is a quick and dirty alternative to the KR20 estimate of the reliability of a test. The formula is: K K KR21 1 K 1 K 2 2 Where K is the number of items, is the mean total score across test takers and is the variance. Calculate the KR21 estimate of reliability. (2 points) 8. How do all of these estimates of reliability compare to each other? Explain the similarities and differences. (5 points) The professor (who is really a mad scientist) has discovered a way to administer the same pretest a second time without the students remembering or being aware that they had already taken the test. In addition the professor has created an alternate form (form B) of the pretest. The results of the second administration (retest) and the administration of form B appear in Table 2 below. Table 2 Item Retest Form B Bob 6 5 Susan 10 7 Students Eliot 8 8 Georgia 7 5 Brad 6 6 10. Use this data to calculate the testretest reliability and alternate forms reliability. For alternate forms compare the scores from the first administration with the scores for form B. (2 points) Testretest reliability = Alternate form reliability= 12. Discuss the pattern of reliability among the different reliability coefficients (internal consistency, testretest and alternate forms) as it relates to sources of error. (4 points)