In an article published in the Journal of Personality, Marsh (1990) provided evidence that confirmatory factor analysis (CFA) can be very useful in analyzing a multitrait-multimethod (MTMM) matrix—as long as care is taken to specify the correct model. Marsh (1990) examined the construct validity of three commonly used measures of preadolescent self-concept: the 80-item Piers Harris (PH) instrument, the 76-item Self Description Questionnaire I (SDQI), and the 28-item Perceived Competence Scale for Children (PCS). Although each of the scales examined has been found to be multidimensional, the number of previously identified dimensions differs across the scales. Marsh posited that each of the three measures assesses physical, social, and academic aspects of self concept. Two of the three measures also include a general self concept dimension, and two of the three measures assess other aspects of self-concept. A sample of 290 Australian fifth graders was administered each of the scales. Marsh used the three different measures to represent different methods, while the previously identified dimensions of self concept were taken as multiple traits. The resulting MTMM matrix was analyzed using both the Campbell-Fiske (1959) guidelines (see Module 9) and CFA.

Using the Campbell-Fiske approach, Marsh first found that similar dimensions across the measures were indeed substantially correlated with one another, providing evidence of construct validity. Second, Marsh examined whether convergent validities exceeded the correlations between different traits measured using different methods. Evidence indicated that the mean convergent validities were greater than heterotrait-heteromethod correlations for 60 out of 62 comparisons. This provided good evidence for this initial step in the examination of discriminant validity. Third, Marsh compared the convergent validities to the heterotrait-monomethod correlations. The expected pattern was found for two of the three measures of self-concept. However, mean heterotrait-monomethod correlations slightly exceeded the mean convergent validities for the PH measure. Thus, support for this criterion of discriminant validity was found for only the SDQI and PCS measures. Finally, the last Campbell-Fiske criterion was examined. This criterion argues that correlations among traits should be similar whether the methods are the same or different. Although this pattern was found for the SDQI and PCS, evidence was not supportive of the discriminant validity of the PH scales. In reanalyzing the data using CFA, Marsh (1990) constructed four possible models to explain the data, based on Widaman’s (1985) taxonomy of models that vary different characteristics of the trait and method factors. Model 1 is a trait-only model that proposes no effect of method. Model 2, the traits and uncorrelated methods factor, assumes that method effects associated with each of the measures are uncorrelated. Model 3 is a bit more complex, in that it does not assume that method effects are unidimensional across all variables assessed by a particular method. Rather, Model 3 represents method variance as correlated uniquenesses. These are correlations between pairs of variables measured by the same method once the trait effects are removed. Finally, Model 4 proposes that unidimensional method factors are correlated with each other. This model can be referred to as traits and correlated method factors. For each of these four possible models, Marsh (1990) evaluated whether the solution was well defined for both a possible four- and a possible five-trait solution. Models 1 and 4 were found to be poorly defined for both possible solutions. Model 2 was found to be well defined for the four-trait solution, but only marginally defined for the five-trait solution. Model 3, however, was found to be very well defined for both the four- and five-trait solutions. Marsh pointed out that when method trait are considered, Model 3 typically provides solutions that are better defined than competing models. Inspection of the CFA results indicated that the correlated uniquenesses associated with the SDQI were considerably smaller than those associated with the other two measures, indicating the lesser influence of method effects for the SDQI. Although convergent validity was found for all three measures of preadolescent self-concept, evidence for discriminant validity was strongest for the SDQI. 

Questions

1. What concerns about the interpretation of an MTMM matrix are better addressed by CFA rather than the Campbell-Fiske (1959) guidelines? 

2. In what ways are the four CFA models proposed by Widaman (1985) similar? In what ways do these models differ from one another? 

3. What are correlated uniquenesses? 

4. What might cause a CFA model to be poorly defined? 

5. What methods can be used to evaluate alternative CFA models? Are some methods more appropriate than others? 

6. How can researchers who use advanced statistical analyses communicate with those who are less statistically savvy?