Play the role of a test administrator evaluating the "advertised" applicability of a MOCK TEST MANUAL called the Wilson Critical Thinking Test (WCTT). You will analyze the test properties presented in the mock test manual in order to make decisions on the appropriate use of the WCTT in terms of its reliability and validity.

Wilson then began to construct his measure of critical thinking. First, he wrote a short passage on a specific topic which contained unstated assumptions, various pieces of data/evidence, and several inferences. Because Wilson's goal was to develop a measure of critical thinking skills, not verbal ability, he attempted to write the passage using relatively elementary vocabulary, understandable terms, and simple sentence structure. Wilson generated four sets of eight multiple choice items related to the passage. Each set of items measured one of the four core elements of critical thinking, yielding a total of 32 multiple-choice items. Unlike traditional multiple-choice items in which one response alternative is correct and the others are incorrect, Wilson's multiple choice items included two correct/acceptable response options and two incorrect/unacceptable response options. Items could be scored as 2 (best response), 1 (acceptable response), or 0 (incorrect response). The written passage and the 32 multiple choice items were administered to 236 students (M age=20.1 years, SD=1.52) at a large Midwestern university. An item analysis was performed on the 32 original items, and those with item-total correlations of less than .30 were discarded. Twelve items (three items for each of the four critical thinking elements) remained, forming the final Wilson Critical Thinking Test (WCTT). WCTT Norms Normative data are based on a combined sample of 1524 undergraduate students (863 females and 661 males; M age=19.5 years, SD=2.01) attending five universities in the United States. No significant differences in scores between gender, regional, or ethnic groups were found. Range of WCTT scores= 3-23; Mean score=15.2 (SD=3), Median score=15.

Study #1: The WCTT was administered to 332 undergraduate students enrolled in general education courses. Sixteen days later, the same students took the WCTT again. The correlation between the two sets of scores was .89.

Study #2: A sample of 475 college students completed the WCTT, along with several other measures. Correlations between the WCTT and these measures were as follows: the Watson-Glaser Critical Thinking Appraisal (r=.61, p<.001), the Peak Verbal Ability Test (r=.43, p<.001) and the Need for Achievement Inventory (r=.06, ns).

Study #3: A sample of 300 college-bound students completed the WCTT during their senior year of high school. One year later, these students were contacted and asked to participate in a follow-up study. Of the original 300 students, 282 agreed to provide official transcripts from their first year in college. The correlation between participants' WCTT scores and first year college grade point average was .30 (p<.001).

Study #4: Analysis of WCTT scores obtained from a sample of 640 students at a large Midwestern university yielded a Cronbach's alpha coefficient of .78.

Study #5: The WCTT was administered to all graduating seniors (N=315) of a liberal arts college in the northeast. The mean WCTT score of students graduating with honors was 18.4 and the mean score of the other graduating students was 15.9. This difference was statistically significant (p<.01).

1.What evidence is presented regarding the reliability of the WCTT?

2.What evidence is presented regarding the content-related validity of the WCTT?

3.What evidence is presented regarding the criterion-related validity of the WCTT?

4.What evidence is presented regarding the construct-related validity of the WCTT?