Scenerio:

From small businesses to large corporations, the companies use psychological tests to measure a wide variety of individual attributes.The results help them make decisions on hiring, placement, training, promotion, wages, and more.But are they the right decisions?Without good psychometric properties, that psychological test could end up being a waste of time. In the real world, a test taker's performance won't be perfectly consistent.For example, if I give a spelling test to a group of fourth-graders, students may have unwanted inconsistency in their test scores.Some students may feel better than they did yesterday, and others feel worse than they did yesterday. Or let's say the tests contains the words Baltimore, Milwaukee, and Seattle.Each of those cities has a professional baseball team.So a child who took the test shortly after looking at baseball scores might have temporary advantages.Or if half the class were to take the test in a noisy, poorly-lit room, we might expect their scores to be lower than what they would have gotten under normal conditions. These sources of unwanted inconsistency are referred to as unsystematic errors of measurement.If errors of measurements are responsible for much of the variability of thetest scores, the test scores will be inconsistent.If the test is given again, scores may not remain stable.However, if errors of measurement have little effect on test scores, the test reflects mainly on the consistent aspects of spelling ability we are interested in. Consequently, these errors reduce the reliability, and therefore the generalizability of a person to score from a single measurement.A less reliable score is less valid evidence, which makes for worse decision making.That's why it is important to have a good grasp of psychometric concepts and methods when you develop and make decisions based on psychological assessments. This week, we'll also learn about the exploratory fact analysis, or EFA.EFA is a statistical method used to uncover the underlying structure of a relatively large set of variables.Let's say you have a colleague who wants to use a personality inventory with these six adjectives, talkative, assertive, imaginative, creative, outgoing, intellectual. Participants report the degree to which each adjective describes their personality in general.Your colleague asked for your opinion of this common objective-based assessment.You think about it, and you begin to wonder, what exactly does the inventory measure?Does it measure six separate facets of a personality, with each facet being reflected by a single adjective, or does it measure a single construct? If so, what's the construct?What do these six adjectives have in common?Or are two or three separate dimensions reflected within these six adjectives?With EFA you can answer all of these questions. Here is a scenario.Let's imagine you are a marketing manager looking to make a new hire. Over the last two decades, researchers have found that creativity is one of the most important characteristics for marketing employees.Problem solving, planning, research and communication skills are undoubtedly highly related to creativity.All of these basic talents are required for careers in marketing.With that in mind, you consider using a 20-item test designed to measure creative thinking in terms of risk management, idea generation, fair and supportive evaluation, new ideas, reward and recommendation of creativity, and collaboration. Your team creates the test, but the 20 items seems a bit of base.For example, one item asks testees to agree or disagree with the statement, I try to avoid talking with my colleagues.You do a little bit of a research and find that the inter-item correlations for the test range from -0.07 to 0.12, for an average of 0.08. Should you be concerned about the usefulness of these test scores in this situation?

Questions:

• Should you be concerned about the possibility that this test will produce meaningless scores?
• If you use this test, what would be problems in terms of reliability and validity?
• You really want to improve this test. How would you improve the quality of this test using various psychometric analyses?