INTRODUCTION TO THE CHI SQUARE TEST

Suppose we drew at random a sample of 200 members of a professional association of sociologists and asked them whether they were in favor of a proposed change to their bylaws. The results are shown in Table 1. But do these observed results reflect the true results ¹ that we would have obtained if we had questioned the entire population? Remember that the null hypothesis (see Topic 38) says that the observed difference was created by random sampling errors; that is, in the population, the true difference is zero. Put another way, the observed difference (n = 120 vs. n = 80) is an illusion created by chance errors.

Table 1Members' approval of a change in bylaws

Response

Yes60.0%

(n = 120)

No40.0%

(n = 80

Total100.0%

The usual test of the null hypothesis when we are considering frequencies (that is, number of cases or n) is chi square, whose symbol is:X2

It turns out that after doing some computations, which are beyond the scope of this book, for the data in Table 1, the results are:

X2 =4.00,df= 1,p < .05

What does this mean for a consumer of research who sees this in a report? The values of chi square and degrees of freedom (df) were calculated solely to obtain the probability that the null hypothesis is correct. That is, chi square and degrees of freedom are not descriptive statistics that you should attempt to interpret. Rather, think of them as substeps in the mathematical procedure for obtaining the value of p. Thus, the consumer of research should concentrate on the fact that p is less than .05. As you probably recall from Topic 38, when the probability (P) that the null hypothesis is correct is .05 or less, we reject the null hypothesis. (Remember, when the probability that something is true is less than 5 in 100 - a low probability -conventional wisdom suggests that we should reject it as being true.) Thus, the difference we observe in Table 1 was probably not created by random sampling errors; therefore, we can say that the difference is statistically significant at the .05 level.

So far, we have concluded that the difference we observed in the sample was probably not created by sampling errors. So where did the difference come from? Two possibilities remain:

1. Perhaps there was a bias in procedures such as the person asking the question in the survey leading the respondents by talking enthusiastically about the proposed change in the bylaws. If we are convinced that adequate measures were taken to prevent procedural bias, we are left with only the next possibility as a viable explanation.

2. Perhaps the population of sociologists is, in fact, in favor of the proposed change, and this fact is correctly identified by studying the random sample.

Now let's consider some results from a survey in which the null hypothesis was not rejected. Table 2 shows the numbers and percentages of subjects in a random samplefrom a population of

teachers who prefer each of three methods for teaching reading. In the table, there are three differences (30 for A versus 27 for B, 30 for A versus 22 for C, and 27 for B versus 22 for C).

Table 2 Teachers' preferences for methods

Method AMethod BMethod C

n = 30 (37.97%) n = 27 (34.18%) n = 22 (27.85%)

The null hypothesis says that this set of differences was created by random sampling errors; in other words, it says that there is no true difference in the population; we have observed a difference only because of sampling errors. The results of the chi square test for the data in Table 2 are:

X2 = 1.214, df = 2,p > .05

Using the decision rule that p must be equal to or less than .05 to reject the null hypothesis, we fail to reject the null hypothesis, which is called a statistically insignificant result. In other words, the null hypothesis must remain on our list as a viable explanation for the set of differences we observed by studying a sample.

In this topic, we have considered the use of chi square in a univariate analysis in which we classify each subject in only one way (such as which candidate each prefers). In the next topic, we'll consider its use in bivariate analysis in which we classify each subject in two ways (such as which candidate each prefers and the gender of each) in order to examine a relationship between the two.

EXERCISE ON TOPIC

1. When we study a sample, are the results called the true results or the observed results?

2. According to the null hypothesis, what created the difference in Table I in this topic?

3. What is the name of the test of the null hypothesis used when we are analyzing frequencies?

4. As a consumer of research, should you try to interpret the value of df?

5. What is the symbol for probability?

6. If you read that a chi square test of a difference yielded a p of less than 5 in 100, what should you conclude about the null hypothesis on the basis of conventional wisdom?

7. Does p < .05 or p > .05 usually lead a researcher to declare a difference to be statistically significant?

8. If we fail to reject a null hypothesis, is the difference in question statistically significant?

9. If we have a statistically insignificant result, does the null hypothesis remain on our list of viable hypotheses?

^l We are using the term true results here to stand for the results of a census of the entire population. The results of a census are true in the sense that they are free of sampling errors. Of course, there may also be measurement errors, which we are not considering here.

THE MEAN, MEDIAN, AND MODE

The most frequently used average is the mean, which is the balance point in a distribution. Its computation is simple-just sum (add up) the scores and divide by the number of scores. The most common symbol for the mean in academic journals is M (for the mean of a population) or m (for the mean of a sample). The symbol preferred by statisticians is:

X which is pronounced "X-bar."

Because the mean is very frequently used as the average, let's consider its formal definition, which is the value around which the deviations sum to zero. You can see what this means by considering the scores in Table 1. When we subtract the mean of the scores (which is 4.0) from each of the other scores, we get the deviations (whose symbol is x). If we sum the deviations, we get zero, as shown in Table 1.

Table 1 Scores and deviation scores

X minus M equals x

1 - 4.0 = -3.0

1 - 4.0 = -3.0

1 - 4.0 = -3.0

2 - 4.0 = -2.0

2 - 4.0 = -2.0

4 - 4.0 = 0.0

6 - 4.0 = 2.0

7 - 4.0 = 3.0

8 - 4.0 = 4.0

8 - 4.0 = 4.0

Sum of the deviations (x)=0.0

Note that if you take any set of scores, compute their mean, and follow the steps in Table 1, the sum of the deviations will always equal zero. ^I

Considering the formal definition, you can see why we also informally define the mean as the balance point in a distribution. The positive and negative deviations balance each other out.

A major drawback of the mean is that it is drawn in the direction of extreme scores. Consider the following two sets of scores and their means.

Scores for Group A: 1, 1, 1, 2,3,6, 7, 8, 8M = 4.11

Scores for Group B: 1,2,2,3,4, 7, 9,25,32M= 9.44

Notice that in both sets there are nine scores ang the two distributions are very similar except for the scores of 25 and 32 in Group B, which are much higher than the others and, thus, make skewed distribution. (To review skewed distributions, see Topic 43.) Notice that the two very high scores have greatly pulled up the mean for Group B; in fact, the mean for Group B is more than twice as high as the mean for Group A because of the two high scores.

When a distribution is highly skewed, we use a different average, the median, which is defined as the middle score. To get an approximate median, put the scores in order from low to high as they are for Groups A and B above, and then count to the middle. Since there are nine scores in Group A, the median (middle score) is 3 (five scores up from the bottom). For Group B, the median (middle score) is 4 (five scores up from the bottom), which is more representative of the center of this skewed distribution than the mean, which we noted was 9.44. Thus, one use of the median is to describe the average of skewed distributions. Another use is to describe the average of ordinal data, which we'll explore in Topic 46.

A third average, the mode, is simply the mostfrequently occurring score. For Group B, there are more scores of 2 than any other score; thus, 2 is the mode. The mode is sometimes used in informal reporting but is very seldom used in formal reports of research.

Because there is more than one type of average, it is vague to make a statement such as, "The average is 4.11." Rather, we should indicate the specific type of average being reported with statements such as, "The mean is 4.11."

^l It might be slightly off from zero if you use a rounded mean such as using 20.33 as the mean when its precise value is 20.3333333333.

Note that a synonym for the term averages is measures of central tendency. Although the latter is seldom used in reports of scientific research, you may encounter it in other research and statistics textbooks.

EXERCISE ON TOPIC

1. Which average is defined as the most frequently occurring score?

2. Which average is defined as the balance point in a distribution?

3. Which average is defined as the middle score?

4. What is the formal definition of the mean?

5. How is the mean calculated?

6. Should the mean be used for highly skewed distributions?

7. Should the median be used for highly skewed distributions?

8. What is a synonym for the term averages?

9. Suppose a fellow student gave a report in class and said, "The average was 25.88." For what additional information should you ask? Why?