Questions and Answers of Measurement Theory In Action

6. Given the data that Cecilia collected, compute each of the following:a. Correction for attenuation in the criterionb. Correction for predictor range restriction ( Note: Use the corrected validity
7. Examining the corrected validity coefficient produced in question 6b, how impressed would you be with this test of mechanical ability for use in selection of new employees? Would you recommend use
1. Why must we examine those individuals who graduated from high school if our real concern is with those students who dropped out of high school?
2. What type of criterion-related validity design did Principal Dickerson employ? Explain.
3. Principal Dickerson developed his hunch after reviewing the files of six recent dropouts. Would the files of six dropouts be sufficient to identify a potentially useful trend that should be
4. What other methods might Principal Dickerson have employed to identify possible correlates of high school dropout rates?5. Principal Dickerson is planning to investigate the relationship between
EXERCISE 8.1: IDENTIFYING POSSIBLE PREDICTORS AND CRITERIA OBJECTIVE: To gain practice identifying relevant predictors and criteria for validation.For each of the criteria presented in items 1–3,
1. Grades in an educational psychology doctoral program
2. A medical student’s “bedside manner” as a doctor
3. Success in a retail sales position For each of the scenarios presented in items 4–6, recommend at least two relevant, practical measures that could serve as criteria.
4. A state in the Southeast would like to determine the usefulness of requiring road tests for drivers older than 70 years of age.
5. A supervisor wishes to determine the job performance of her factory workers.
6. A researcher wishes to determine whether regular consumption of a certain vitamin supplement influences cardiovascular health in men aged 50–75.
EXERCISE 8.2: DETECTING VALID PREDICTORS OBJECTIVE: To gain experience identifying valid predictors in a data set.PROLOGUE: The data set “Bus Driver.sav” (see Appendix B) contains a number of
1. Overall, how highly are the possible predictors intercorrelated?
2. Overall, how highly are the job performance measures intercorrelated?
3. Overall, how useful would the personality measures be in predicting job performance?
4. Overall, how useful would the demographic variables be in predicting job performance?
5. Inspecting only the significant correlations, interpret the findings for tenure across the job performance criteria.
6. If you were examining the validity of a set of variables that you hoped to use for prediction, and you found validity coefficients similar to those in this analysis, what would you do?
EXERCISE 8.3: PREDICTING SALES JOB PERFORMANCE USING ZERO-ORDER CORRELATIONS OBJECTIVE: To practice computation of a validity coefficient using statistical software.PROLOGUE: A sales manager hoping
1. What type of validation study is the manager conducting? Explain.
2. Examining the zero-order correlations, what variables are significantly related to selling?
3. What percentage of variance in selling is accounted for by impression management?
4. If impression management were measured with an = .80, what would be the validity estimate of this variable following correction for attenuation due to unreliability?
5. Cognitive ability has little relationship to performance in these employees.a. Is this finding likely due to poor reliability in the measure of cognitive ability? Explain.
1. Psychological and educational measures assess latent constructs. Examination of validity, therefore, requires a clear specification of the expected relationships between the construct of interest
2. Relationships between variables measured using the same measurement method (e.g., self-reports assessed at a single period in time completed by the same individual using the same response scale
3. Recognize that no single study can ever prove the validity of test scores.Validation is an ongoing process.
7. Why is CMV a concern in construct validation studies that involve correlation matrices?
10. Messick (1995b) identified six aspects of construct validation. Choose any three of these aspects to discuss how Messick’s conceptualization has extended your awareness of the meaning of
1. What role should cost play in determining an appropriate measure for research?
2. Why would Dr. Bachelor be skeptical of the scale of emotional intelligence that Khatera found?
3. What information in the article should Khatera search for to help address some of her advisor’s concerns?
4. What other steps could Khatera take to gather information regarding the validity of the newly found scale of emotional intelligence?
5. What evidence of validity for a measure is most frequently provided by (a) publishers of commercially available tests and (b) authors of research scales?
6. What validity evidence would you consider sufficient to use in an important research study such as a graduate thesis?
1. In what ways does “content validity” provide evidence of the meaningfulness of test scores?
2. In what ways does “criterion-related validity” provide evidence of the meaningfulness of test scores?
3. Is Edgar’s concern over not being able to distinguish among the various validation strategies warranted? Explain.
4. Explain why the trinitarian view of validity (content, criterion-related, construct) indicates an insufficient view of validation.
5. Explain how Messick’s (1995b) aspects of construct validation incorporate each of the following “outdated” terms under the umbrella of construct validity:a. Content validityb.
1. In what ways does “content validity” provide evidence of the meaningfulness of test scores?
2. In what ways does “criterion-related validity” provide evidence of the meaningfulness of test scores?
3. Is Edgar’s concern over not being able to distinguish among the various validation strategies warranted? Explain.
4. Explain why the trinitarian view of validity (content, criterion-related, construct) indicates an insufficient view of validation.
5. Explain how Messick’s (1995b) aspects of construct validation incorporate each of the following “outdated” terms under the umbrella of construct validity:a. Content validityb.
EXERCISE 9.1: IDENTIFYING MEASURES FOR CONSTRUCT VALIDATION OBJECTIVE: To gain experience in identifying relevant measures for examining convergent and discriminant validity.PROLOGUE: Imagine that
1. A newly developed paper-and-pencil intelligence test intended to assess academic giftedness among sixth graders
2. An interview evaluating an adult’s personal integrity
3. A self-report form assessing one’s interpersonal assertiveness
4. A peer rating form measuring an individual’s general optimism
5. A supervisor’s rating form of an employee’s career achievement motivation
EXERCISE 9.2: EXAMINING ELEMENTS OF A NOMOLOGICAL NETWORK OBJECTIVE: To determine whether empirical evidence provides support for expected relationships between measures.An industrial/organizational
1. What is the expected pattern of relationships among these three measures?Explain.
2. Is there evidence of convergent validity for the personality-based integrity measure?
3. Is there evidence of discriminant validity for the personality-based integrity measure?
4. Based on the results obtained, can the psychologist claim that she has established the construct validity of the measure? Explain.
1. When conducting psychometric meta-analysis, realize that garbage in will equal garbage out (i.e., meta-analysis will not turn a series of poor studies into a definitive answer).
2. Be sure to clearly define the population to which you are interested in generalizing your results. This is just one of many judgment calls that will need to be made during the meta-analysis
3. Realize that psychometric meta-analysis is simply another tool to add to your methodological toolbox. It will not be a panacea for a series of illconceived and ill-conducted studies.
4. Document, in detail, all steps completed during your meta-analysis study.
2. Most papers and books on meta-analysis say that one should include both published and unpublished studies on a given topic. How does one go about getting unpublished studies?
6. How do you decide which moderators to examine?
1. What other databases or outlets should Raul have used to obtain a more complete set of studies on supervisor satisfaction?
2. If you were Raul, what criteria would you use to decide which studies to include and which to exclude?
3. How should Raul go about deciding which factors in the studies to code to investigate possible moderators later on?
4. Is it possible for Raul to use the studies that don’t provide correlations?What are his options?
5. How many studies do you think Raul will ultimately need in order to satisfy his thesis committee? In order to obtain “accurate” results?
1. If you were Ming-Yu, would you have coded for any other variables?
2. What factors should Ming-Yu have considered in choosing which software package to use to analyze the data?
3. What do you think could have led to Ming-Yu getting results contradictory to the results Professor Riggs had found in most of his individual studies?
4. Should Ming-Yu have “updated” the previous literature search? If so, how?
5. Because Ming-Yu didn’t find any difference, does she need to conduct moderator analyses?
EXERCISE 10.1: OUTLINING A META-ANALYTIC STUDY OBJECTIVE: To practice outlining a meta-analytic study.Individually or in small groups of three to five, students will select a topic on which to
EXERCISE 10.2: ALBEMARLE SUPREME COURT CASE OBJECTIVE: To calculate meta-analytic estimates by hand.BACKGROUND: In Table 10.3 , you will find data from the Albemarle Paper Company v. Moody (1975)
1. Test bias, unfairness, and equivalence are distinct concepts that should be dealt with in their own right.
2. MMR is still the most common analytic strategy for determining predictive test bias.
3. Recent Monte Carlo simulations indicate that both intercept and slope bias may be more prevalent in pre-employment testing than previous thought.
4. Test accommodations need to be made on a case-by-case basis.
1. Based on the data in Figure 11.1 , what would have happened if we had used a common regression line to predict suicide risk in all three age groups?
6. Which stakeholders in the testing process (see Module 1) are responsible for determining whether test bias actually exists or not?
7. Can a test that is determined to be biased still be a fair test? Alternatively, can a test that is determined to be unfair still be an unbiased test?
8. Describe the process of back translation. Why is back translation insufficient to guarantee equivalence?
11. What factors should be considered when determining whether a requested test accommodation is reasonable?
1. Does there appear to be any test bias in terms of gender? If so, what kind of predictor bias seems to be evident?
2. Does there appear to be any test bias in terms of age? If so, what kind of predictor bias seems to be evident?
3. If you were Larry, what omitted variables would you investigate?Would you look for different potential omitted variables for gender and age? Why or why not?
4. What other factors besides the omitted-variable concern might be impacting Larry’s results?
5. Would drawing a scatterplot (similar to Figures 11.3 and 11.4 ) help in determining what is happening with the data, or is the MMR analysis sufficient?
6. Does the criterion variable that Larry used (i.e., whether a new hire passed probation) make a difference in whether we are likely to find test bias?
1. If you were Joelle, where would you start? What key factors would you want to consider?
2. What information would you want to know from the test publisher or from reviews conducted of the test?
3. Does this appear to be more of a test bias or test fairness issue? Why?
4. If Joelle wanted to examine for test bias, what data would she need?
5. Which of the models of test fairness presented in the module overview would be most applicable here? Why?
6. Assuming Joelle actually found the test to be “unfair,” what could she do to make the test fair?
EXERCISE 11.1: EXAMINING TEST BIAS OBJECTIVE: To practice computing moderated multiple regression (MMR)analyses and drawing scatterplots to examine test bias issues.Using the data set of Mersman and
EXERCISE 11.2: EXAMINING DIFFERENT MODELS OF TEST BIAS/FAIRNESS (EEOC REQUEST FOR INFORMATION AND ANALYSIS—PART 2)OBJECTIVE: To practice running and interpreting data to investigate several forms
Exercise 2.1 first if you have completed that exercise.)1. The development of maximal performance tests relies heavily on test specifications.2. Quality items result from a deliberate and painstaking

Showing 600 - 700 of 1226