Evaluating health care research reports. When evaluating research reports in health care, a popular tool is the Assessment of Multiple Systematic Reviews (AMSTAR).

AMSTAR, which incorporates 11 items (questions), has been widely accepted by professional health associations. A LO4 group of dental researchers have revised the assessment tool and named it R-AMSTAR (The Open Dentistry Journal, Vol. 4, 2010). The revised assessment tool was validated on five systematic reviews (named R1, R2, R3, R4, and R5)

on rheumatoid arthritis. For each review, scores on the 11 R-AMSTAR items (all measured on a 4-point scale) were obtained. The data, are shown in the accompanying table.

Review Item 1 Item 2 Item 3 Item 4 Item 5 Item 6 Item 7 Item 8 Item 9 Item 10 Item 11 R1 4.0 1.0 4.0 2.0 3.5 3.5 3.5 3.5 1.0 1.0 1.0 R2 3.5 2.5 4.0 4.0 3.5 4.0 3.5 2.5 3.5 1.5 1.0 R3 4.0 4.0 3.5 4.0 1.5 2.5 3.5 3.5 2.5 1.5 1.0 R4 3.5 2.0 4.0 4.0 2.0 4.0 3.5 3.0 3.5 1.0 1.0 R5 3.5 4.0 4.0 3.0 2.5 4.0 4.0 4.0 2.5 1.0 2.5

a. One goal of the study was to compare the mean item scores of the five reviews. Set up the null and alternative hypotheses for this test.

b. Examine the data in the table and explain why a randomized block ANOVA is appropriate to apply.

c. The SAS output for a randomized block ANOVA of the data (with Review as treatments and Item as blocks)
appears above. Interpret the p-values of the tests shown.

d. The SAS printout also reports the results of a Tukey multiple-comparison analysis of the five Review means. Which pairs of means are significantly different?
Do these results agree with your conclusion in part c?

e. The experimentwise error rate used in the analysis in part d is .05. Interpret this value.

Transcribed Image Text:

Dependent Variable: SCORE The ANOVA Procedure Source Model Error Corrected Total DF Sum of Squares Mean Square F Value Pr > F 14 46.01818182 3.28701299 6.36 F REVIEW 4 2.51818182 ITEM 10 43.50000000 0.62954545 4.35000000 1.22 0.3186 8.41