Researchers have developed a new measure of personality called the “Blockbuster Personality Type Questionnaire” to assess peoples’ personality types. The questionnaire includes images from the Top 10 most popular Hollywood movies in the last 10 years and asks respondents to describe what is happening in the scenes. Responses are scored in terms of how consistent they are with the original movies, and how much respondents showed empathy for the different characters in the scenes. Based on their scores, respondents are classified as either being either “active heroes”, “passive victims”, or “conniving villains”, reflecting the hidden desires and goals that they were born with.
The researchers want the test to be used for selection and recruitment in organizations around the world, so they collected information about the test’s reliability and validity. They asked students from a first-year psychology class to take the test at the start and at the end of semester and compared their scores for a measure of reliability. To test validity, they used scores on the test to predict how well the students do in their university studies.
What are some of the issues (i.e., limitations, at least three aspects) with what the researchers have done? How could they have designed and evaluated the test in a better way? For each issue/limitation that you identify, explain why it is an issue and how it could be improved.