Your advisor wants to develop a new measure of negative affectivity to complement existing measures that are traditional response formats. Your advisor believes that existing measures are limited because they tap only conscious aspects of negative affectivity, whereas she believes that there is a significant aspect of personality that lies beyond conscious understanding. To address her concerns, you have been tasked with developing a projective test that measures the unconscious aspects of negative affectivity. For your initial version of the test, you have found ten pieces of abstract art and you present these images on the computer, asking each respondent to type up to five sentences about their instant and initial reactions to the abstract art. You have randomly sampled 100 undergraduate students from your research subject pool. After you have collected these data, your advisor asks “Okay, how do we score these tests?” (Yes, you should have thought of this before you collected the data!) You propose that you score each of the items based on your interpretation of the number of negative statements made in response to each piece of abstract art. Your advisor says, “Well, we shouldn’t just rely on your judgment and so I will rate each of the responses to each piece of abstract art so we can test each other’s reliability.” 

Finally, your advisor suggests that peoples’ responses to art may be influenced by their mood when they took the test and so she proposes that you follow up with each of your 100 respondents and ask them to complete the test two weeks later. You have collected all of the data and now your advisor says, “How do we analyze all of this?” Given that you have just read this module, you suggest GT analyses, and she says “I have never done one of those; tell me how we do it.” She asks you the following questions for you to ponder. 

Questions  

1. What are the factors of variation that you would compute for your G Study? For each of these factors, would you consider that factor random or fixed? Is your design fully crossed or are some factors nested within others? 

2. What statistics would you report back to your advisor to help her better understand whether the projective test is working well? 

3. When conducting a D study, would you be most interested in absolute or relative decisions for this new measure? 

4. Explain to your advisor how the information gathered from your GT analysis compares to information that could have been found from traditional CTT-based investigations of reliability as well as IRT-based investigations. 

5. Suppose that the GT study shows that there is reasonable consistency over time and raters. Develop a new GT-based study to investigate other sources of variation and error. Think of these other sources of variation and decide how you would measure them.