EXERCISE 12.1: DETERMINATION OF TEST COMPOSITION OBJECTIVE: To gain practice developing test specifications for knowledge tests.

In developing knowledge tests, many important decisions must be made to ensure that test objectives are achieved. Although unlimited time and resources might allow for creation of a near-perfect knowledge test, almost all exams must be developed with practical considerations in mind. For each of the tests described below, complete the following:

A. Determine the item format.

Select one or more item formats that will appropriately measure the test objectives.

B. Determine the number of points per format.

Assuming the test will be scored out of 100 points, determine the total number of points that will be used for each item format chosen. For example, if you choose to create a test with multiple-choice and essay questions, assign the total number of points to be assessed by multiplechoice format and the total number of points to be assessed by essay format. ( Note: The total number of points must sum to 100.)

C. Determine the number of items per format and the number of points per item for each format.

For each item format selected, determine the number of items that will be written. The number of points assigned to each individual item (within a particular format) will be determined by dividing the number of points for this item format by the number of items using this format.

D. Provide justification.

Explain and justify why you believe your decisions will lead to the development of a practical, reliable, and content-valid exam.

1. Test Description:

Midterm exam for an entry-level statistics course for the behavioral sciences Number of students: 30 Class period: 60 minutes, plus 40-minute lab Students would be expected to:

• Summarize, display, and interpret sets of data.

• Understand the logic of statistical analysis, probability, and hypothesis testing.

• Conduct descriptive statistical analyses and probability problems with the use of a calculator.

Developing Tests of Maximal Performance 169 2. Test Description:

Final exam for an introductory psychology course Number of students: 200 Class period: 1.5 hours Students would be expected to:

• Define basic psychological terminology.

• Comprehend the role of research in the field of psychology.

• Understand major psychological theories and research findings.

• Identify leading contributors to the field of psychology.

• Critically apply the principles and theories of psychology to contemporary daily life.

3. Test Description:

Final exam in a college-level history course titled History of Western Civilization Number of students: 25 Class period: 3 hours Students would be expected to:

• Demonstrate factual knowledge of the history of Western civilization.

• Understand how various conditions, social structures, and ideas shape the development of society.

• Identify the historical roots of current practices and debates.

4. Test Description:

Final exam for a graduate-level educational measurement course Number of students: 12 Class period: 3 hours Students would be expected to:

• Discuss the purposes, utility, and limitations of various psychometric concepts.

• Compare and contrast classical test theory with item response theory.

• Identify and compute appropriate psychometrics for a given testing situation.