1. Calculate the sample size needed given these factors: one-tailed t-test with two independent groups of equal size small effect size (see Piasta, S.B., & Justice, L.M., 2010) alpha =.05 beta = .2 Assume that the result is a sample size beyond what you can obtain. Use the compromise function to compute alpha and beta for a sample half the size. Indicate the resulting alpha and beta. Present an argument that your study is worth doing with the smaller sample (include peer-reviewed journal articles as needed to support your response). 2. Calculate the sample size needed given these factors: ANOVA (fixed effects, omnibus, one-way) small effect size alpha =.05 beta = .2 3 groups Assume that the result is a sample size beyond what you can obtain. Use the compromise function to compute alpha and beta for a sample approximately half the size. Give your rationale for your selected beta/alpha ratio. Indicate the resulting alpha and beta. Give an argument that your study is worth doing with the smaller sample. 3. In a few sentences, describe two designs that can address your research question. The designs must involve two different statistical analyses. For each design, specify and justify each of the four factors and calculate the estimated sample size you'll need. Give reasons for any parameters you need to specify for G*Power. Support your paper with a minimum of 5 resources. In addition to these specified resources, other appropriate scholarly resources, including older articles, may be included. Length: 5-7 pages not including title and reference pages References: Minimum of 5 scholarly resources. Reference on G *power http://www.psycho.uni-duesseldorf.de/abteilungen/aap/gpower3/ G * Power 3.1 manual January 31, 2014 This manual is not yet complete. We will be adding help on more tests in the future. If you cannot find help for your test in this version of the manual, then please check the G*Power website to see if a more up-to-date version of the manual has been made available. Contents 1 Introduction 2 2 The G * Power calculator 7 3 Exact: Correlation - Difference from constant (one sample case) 9 4 Exact: Proportion - difference from constant (one sample case) 11 5 Exact: Proportion - inequality, two dependent groups (McNemar) 14 6 Exact: Proportions - inequality of two independent groups (Fisher's exact-test) 17 7 Exact test: Multiple Regression - random model 18 8 Exact: Proportion - sign test 22 9 Exact: Generic binomial test 23 10 F test: Fixed effects ANOVA - one way 21 Wilcoxon signed-rank test: Means - difference from constant (one sample case) 51 22 Wilcoxon-Mann-Whitney test of a difference between two independent means 54 23 t test: Generic case 24 c2 test: Variance - difference from constant (one sample case) 58 25 z test: Correlation - inequality of two independent Pearson r's 59 26 z test: Correlation - inequality of two dependent Pearson r's 60 24 11 F test: Fixed effects ANOVA - special, main effects and interactions 26 12 t test: Linear Regression (size of slope, one group) 31 13 F test: Multiple Regression - omnibus (deviation of R2 form zero), fixed model 34 14 F test: Multiple Regression - special (increase of R2 ), fixed model 37 15 F test: Inequality of two Variances 40 16 t test: Correlation - point biserial model 41 17 t test: Linear Regression (two groups) 43 57 18 t test: Means - difference between two dependent means (matched pairs) 46 19 t test: Means - difference from constant (one sample case) 48 20 t test: Means - difference between two independent means (two groups) 50 1 27 Z test: Multiple Logistic Regression 64 28 Z test: Poisson Regression 69 29 Z test: Tetrachoric Correlation 74 References 78 1 Introduction Distribution-based approach to test selection First select the family of the test statistic (i.e., exact, F , t , c2 , or ztest) using the Test family menu in the main window. The Statistical test menu adapts accordingly, showing a list of all tests available for the test family. G * Power (Fig. 1 shows the main window of the program) covers statistical power analyses for many different statistical tests of the F test, Example: For the two groups t-test, first select the test family based on the t distribution. t test, c2 -test and z test families and some exact tests. G * Power provides effect size calculators and graphics options. G * Power supports both a distribution-based and a design-based input mode. It contains also a calculator that supports many central and noncentral probability distributions. G * Power is free software and available for Mac OS X and Windows XP/Vista/7/8. 1.1 Then select Means: Difference between two independent means (two groups) option in the Statictical test menu. Types of analysis G * Power offers five different types of statistical power analysis: 1. A priori (sample size N is computed as a function of power level 1 b, significance level a, and the to-bedetected population effect size) 2. Compromise (both a and 1 b are computed as functions of effect size, N, and an error probability ratio q = b/a) Design-based approach to the test selection Alternatively, one might use the design-based approach. With the Tests pull-down menu in the top row it is possible to select 3. Criterion (a and the associated decision criterion are computed as a function of 1 b, the effect size, and N) 4. Post-hoc (1 b is computed as a function of a, the population effect size, and N) the parameter class the statistical test refers to (i.e., correlations and regression coefficients, means, proportions, or variances), and 5. Sensitivity (population effect size is computed as a function of a, 1 b, and N) 1.2 the design of the study (e.g., number of groups, independent vs. dependent samples, etc.). Program handling The design-based approach has the advantage that test options referring to the same parameter class (e.g., means) are located in close proximity, whereas they may be scattered across different distribution families in the distributionbased approach. Perform a Power Analysis Using G * Power typically involves the following three steps: 1. Select the statistical test appropriate for your problem. 2. Choose one of the five types of power analysis available Example: In the Tests menu, select Means, then select Two independent groups" to specify the two-groups t test. 3. Provide the input parameters required for the analysis and click "Calculate". Plot parameters In order to help you explore the parameter space relevant to your power analysis, one parameter (a, power (1 2 ), effect size, or sample size) can be plotted as a function of another parameter. 1.2.1 Select the statistical test appropriate for your problem In Step 1, the statistical test is chosen using the distributionbased or the design-based approach. 2 Figure 1: The main window of G * Power 1.2.2 the required power level (1 Choose one of the five types of power analysis available b ), the pre-specified significance level a, and In Step 2, the Type of power analysis menu in the center of the main window is used to choose the appropriate analysis type and the input and output parameters in the window change accordingly. the population effect size to be detected with probability (1 b). In a criterion power analysis, a (and the associated decision criterion) is computed as a function of Example: If you choose the first item from the Type of power analysis menu the main window will display input and output parameters appropriate for an a priori power analysis (for t tests for independent groups if you followed the example provided in Step 1). 1-b, the effect size, and a given sample size. In a compromise power analysis both a and 1 computed as functions of b are the effect size, N, and an error probability ratio q = b/a. In an a priori power analysis, sample size N is computed as a function of In a post-hoc power analysis the power (1 puted as a function of 3 b) is com- a, Because Cohen's book on power analysis Cohen (1988) appears to be well known in the social and behavioral sciences, we made use of his effect size measures whenever possible. In addition, wherever available G * Power provides his definitions of "'small"', "'medium"', and "'large"' effects as "'Tool tips"'. The tool tips may be optained by moving the cursor over the "'effect size"' input parameter field (see below). However, note that these conventions may have different meanings for different tests. the population effect size parameter, and the sample size(s) used in a study. In a sensitivity power analysis the critical population effect size is computed as a function of a, 1 b, and Example: The tooltip showing Cohen's measures for the effect size d used in the two groups t test N. 1.2.3 Provide the input parameters required for the analysis In Step 3, you specify the power analysis input parameters in the lower left of the main window. Example: An a priori power analysis for a two groups t test would require a decision between a one-tailed and a two-tailed test, a specification of Cohen's (1988) effect size measure d under H1 , the significance level a, the required power (1 b) of the test, and the preferred group size allocation ratio n2 /n1 . Let us specify input parameters for a one-tailed t test, If you are not familiar with Cohen's measures, if you think they are inadequate for your test problem, or if you have more detailed information about the size of the to-beexpected effect (e.g., the results of similar prior studies), then you may want to compute Cohen's measures from more basic parameters. In this case, click on the Determine button to the left the effect size input field. A drawer will open next to the main window and provide access to an effect size calculator tailored to the selected test. a medium effect size of d = .5, a = .05, (1 b) = .95, and an allocation ratio of n2 /n1 = 1 Example: For the two-group t-test users can, for instance, specify the means 1 , 2 and the common standard deviation (s = s1 = s2 ) in the populations underlying the groups to calculate Cohen's d = |1 2 |/s. Clicking the Calculate and transfer to main window button copies the computed effect size to the appropriate field in the main window This would result in a total sample size of N = 176 (i.e., 88 observation units in each group). The noncentrality parameter d defining the t distribution under H1 , the decision criterion to be used (i.e., the critical value of the t statistic), the degrees of freedom of the t test and the actual power value are also displayed. In addition to the numerical output, G * Power displays the central (H0 ) and the noncentral (H1 ) test statistic distributions along with the decision criterion and the associated error probabilities in the upper part of the main window. This supports understanding the effects of the input parameters and is likely to be a useful visualization tool in the teaching of, or the learning about, inferential statistics. Note that the actual power will often be slightly larger than the pre-specified power in a priori power analyses. The reason is that non-integer sample sizes are always rounded up by G * Power to obtain integer values consistent with a power level not less than the pre-specified one. 4 The distributions plot may be copied, saved, or printed by clicking the right mouse button inside the plot area. The button X-Y plot for a range of values at to bottom of the main window opens the plot window. Example: The menu appearing in the distribution plot for the t-test after right clicking into the plot. By selecting the appropriate parameters for the y and the x axis, one parameter (a, power (1 b), effect size, or sample size) can be plotted as a function of another parameter. Of the remaining two parameters, one can be chosen to draw a family of graphs, while the fourth parameter is kept constant. For instance, power (1 b) can be drawn as a function of the sample size for several different population effects sizes, keeping a at a particular value. The plot may be printed, saved, or copied by clicking the right mouse button inside the plot area. Selecting the Table tab reveals the data underlying the plot (see Fig. 3); they may be copied to other applications by selecting, cut and paste. The input and output of each power calculation in a G*Power session are automatically written to a protocol that can be displayed by selecting the "Protocol of power analyses" tab in the main window. You can clear the protocol, or to save, print, and copy the protocol in the same way as the distributions plot. Note: The Power Plot window inherits all input parameters of the analysis that is active when the X-Y plot for a range of values button is pressed. Only some of these parameters can be directly manipulated in the Power Plot window. For instance, switching from a plot of a two-tailed test to that of a one-tailed test requires choosing the Tail(s): one option in the main window, followed by pressing the X-Y plot for range of values button. (Part of) the protocol window. 1.2.4 Plotting of parameters G * Power provides to possibility to generate plots of one of the parameters a, effectsize, power and sample size, depending on a range of values of the remaining parameters. The Power Plot window (see Fig. 2) is opened by clicking the X-Y plot for a range of values button located in the lower right corner of the main window. To ensure that all relevant parameters have valid values, this button is only enabled if an analysis has successfully been computed (by clicking on calculate). The main output parameter of the type of analysis selected in the main window is by default selected as the dependent variable y. In an a prior analysis, for instance, this is the sample size. 5 Figure 2: The plot window of G * Power Figure 3: The table view of the data for the graphs shown in Fig. 2 6 2 The G * Power calculator sign(x) - Sign of x: x < 0 ! 1. G * Power contains a simple but powerful calculator that can be opened by selecting the menu label "Calculator" in the main window. Figure 4 shows an example session. This small example script calculates the power for the one-tailed t test for matched pairs and demonstrates most of the available features: 1, x = 0 ! 0, x > 0 ! lngamma(x) Natural logarithm of the gamma function ln(G( x )) frac(x) - Fractional part of floating point x: frac(1.56) is 0.56. int(x) - Integer part of float point x: int(1.56) is 1. There can be any number of expressions min(x,y) - Minimum of x and y The result is set to the value of the last expression in the script max(x,y) - Maximum of x and y Several expression on a line are separated by a semicolon uround(x,m) - round x up to a multiple of m uround(2.3, 1) is 3, uround(2.3, 2) = 4. Expressions can be assigned to variables that can be used in following expressions Supported distribution functions (CDF = cumulative distribution function, PDF = probability density function, Quantile = inverse of the CDF). For information about the properties of these distributions check http://mathworld.wolfram.com/. The character # starts a comment. The rest of the line following # is ignored Many standard mathematical functions like square root, sin, cos etc are supported (for a list, see below) zcdf(x) - CDF zpdf(x) - PDF zinv(p) - Quantile of the standard normal distribution. Many important statistical distributions are supported (see list below) normcdf(x,m,s) - CDF normpdf(x,m,s) - PDF norminv(p,m,s) - Quantile of the normal distribution with mean m and standard deviation s. The script can be easily saved and loaded. In this way a number of useful helper scripts can be created. The calculator supports the following arithmetic operations (shown in descending precedence): Power: ^ Multiply: Divide: / Plus: + Minus: - chi2cdf(x,df) - CDF chi2pdf(x,df) - PDF chi2inv(p,df) - Quantile of the chi square distribution with d f degrees of freedom: c2d f ( x ). (2^3 = 8) (2 2 = 4) (6/2 = 3) (2 + 3 = 5) (3 fcdf(x,df1,df2) - CDF fpdf(x,df1,df2) - PDF finv(p,df1,df2) - Quantile of the F distribution with d f 1 numerator and d f 2 denominator degrees of freedom Fd f1 ,d f2 ( x ). 2 = 1) Supported general functions abs(x) - Absolute value | x | tcdf(x,df) - CDF tpdf(x,df) - PDF tinv(p,df) - Quantile of the Student t distribution with d f degrees of freedom td f ( x ). sin(x) - Sinus of x asin(x) - Arcus sinus of x cos(x) - Cosinus of x acos(x) - Arcus cosinus of x ncx2cdf(x,df,nc) - CDF ncx2pdf(x,df,nc) - PDF ncx2inv(p,df,nc) - Quantile of noncentral chi square distribution with d f degrees of freedom and noncentrality parameter nc. tan(x) - Tangens of x atan(x) - Arcus tangens of x atan2(x,y) - Arcus tangens of y/x ncfcdf(x,df1,df2,nc) - CDF ncfpdf(x,df1,df2,nc) - PDF ncfinv(p,df1,df2,nc) - Quantile of noncentral F distribution with d f 1 numerator and d f 2 denominator degrees of freedom and noncentrality parameter nc. exp(x) - Exponential e x log(x) - Natural logarithm ln( x ) p sqrt(x) - Square root x sqr(x) - Square x2 7 Figure 4: The G * Power calculator mr2cdf(R2 , r2 ,k,N) - CDF mr2pdf(R2 , r2 ,k,N) - PDF mr2inv(p,r2 ,k,N) - Quantile of the distribution of the sample squared multiple correlation coefficient R2 for population squared multiple correlation coefficient r2 , k 1 predictors, and samples of size N. nctcdf(x,df,nc) - CDF nctpdf(x,df,nc) - PDF nctinv(p,df,nc) - Quantile of noncentral Student t distribution with d f degrees of freedom and noncentrality parameter nc. betacdf(x,a,b) - CDF betapdf(x,a,b) - PDF betainv(p,a,b) - Quantile of the beta distribution with shape parameters a and b. logncdf(x,m,s) - CDF lognpdf(x,m,s) - PDF logninv(p,m,s) - Quantile of the log-normal distribution, where m, s denote mean and standard deviation of the associated normal distribution. poisscdf(x,l) - CDF poisspdf(x,l) - PDF poissinv(p,l) - Quantile poissmean(x,l) - Mean of the poisson distribution with mean l. laplcdf(x,m,s) - CDF laplpdf(x,m,s) - PDF laplinv(p,m,s) - Quantile of the Laplace distribution, where m, s denote location and scale parameter. binocdf(x,N,p) - CDF binopdf(x,N,p) - PDF binoinv(p,N,p) - Quantile of the binomial distribution for sample size N and success probability p. expcdf(x,l - CDF exppdf(x,l) - PDF expinv(p,l - Quantile of the exponential distribution with parameter l. hygecdf(x,N,ns,nt) - CDF hygepdf(x,N,ns,nt) - PDF hygeinv(p,N,ns,nt) - Quantile of the hypergeometric distribution for samples of size N from a population of total size nt with ns successes. unicdf(x,a,b) - CDF unipdf(x,a,b) - PDF uniinv(p,a,b) - Quantile of the uniform distribution in the intervall [ a, b]. corrcdf(r,r,N) - CDF corrpdf(r,r,N) - PDF corrinv(p,r,N) - Quantile of the distribution of the sample correlation coefficient r for population correlation r and samples of size N. 8 3 Exact: Correlation - Difference from constant (one sample case) asymptotically identical, that is, they produce essentially the same results if N is large. Therefore, a threshold value x for N can be specified that determines the transition between both procedures. The exact procedure is used if N < x, the approximation otherwise. The null hypothesis is that in the population the true correlation r between two bivariate normally distributed random variables has the fixed value r0 . The (two-sided) alternative hypothesis is that the correlation coefficient has a different value: r 6= r0 : H0 : H1 : r r 2. Use large sample approximation (Fisher Z). With this option you select always to use the approximation. There are two properties of the output that can be used to discern which of the procedures was actually used: The option field of the output in the protocol, and the naming of the critical values in the main window, in the distribution plot, and in the protocol (r is used for the exact distribution and z for the approximation). r0 = 0 r0 6= 0. A common special case is r0 = 0 ?see e.g.>[Chap. 3]Cohen69. The two-sided test (\"two tails\") should be used if there is no restriction on the direction of the deviation of the sample r from r0 . Otherwise use the one-sided test (\"one tail\"). 3.1 3.3 In the null hypothesis we assume r0 = 0.60 to be the correlation coefficient in the population. We further assume that our treatment increases the correlation to r = 0.65. If we require a = b = 0.05, how many subjects do we need in a two-sided test? Effect size index To specify the effect size, the conjectured alternative correlation coefficient r should be given. r must conform to the following restrictions: 1 + # < r < 1 #, with # = 10 6 . The proper effect size is the difference between r and r0 : r r0 . Zero effect sizes are not allowed in a priori analyses. G * Power therefore imposes the additional restriction that |r r0 | > # in this case. For the special case r0 = 0, Cohen (1969, p.76) defines the following effect size conventions: Select Type of power analysis: A priori Options Use exact distribution if N <: 10000 Input Tail(s): Two Correlation r H1: 0.65 a err prob: 0.05 Power (1-b err prob): 0.95 Correlation r H0: 0.60 small r = 0.1 medium r = 0.3 large r = 0.5 Pressing the Determine button on the left side of the effect size label opens the effect size drawer (see Fig. 5). You can use it to calculate |r| from the coefficient of determination r2 . Output Lower critical r: 0.570748 Upper critical r: 0.627920 Total sample size: 1928 Actual power: 0.950028 In this case we would reject the null hypothesis if we observed a sample correlation coefficient outside the interval [0.571, 0.627]. The total sample size required to ensure a power (1 b) > 0.95 is 1928; the actual power for this N is 0.950028. In the example just discussed, using the large sample approximation leads to almost the same sample size N = 1929. Actually, the approximation is very good in most cases. We now consider a small sample case, where the deviation is more pronounced: In a post hoc analysis of a two-sided test with r0 = 0.8, r = 0.3, sample size 8, and a = 0.05 the exact power is 0.482927. The approximation gives the slightly lower value 0.422599. Figure 5: Effect size dialog to determine the coefficient of determination from the correlation coefficient r. 3.2 Examples Options The procedure uses either the exact distribution of the correlation coefficient or a large sample approximation based on the z distribution. The options dialog offers the following choices: 3.4 Related tests Similar tests in G * Power 3.0: Correlation: Point biserial model 1. Use exact distribution if N < x. The computation time of the exact distribution increases with N, whereas that of the approximation does not. Both procedures are Correlations: Two independent Pearson r's (two samples) 9 3.5 Implementation notes Exact distribution. The H0 -distribution is the sample correlation coefficient distribution sr (r0 , N ), the H1 distribution is sr (r, N ), where N denotes the total sample size, r0 denotes the value of the baseline correlation assumed in the null hypothesis, and r denotes the 'alternative correlation'. The (implicit) effect size is r r0 . The algorithm described in Barabesi and Greco (2002) is used to calculate the CDF of the sample coefficient distribution. Large sample approximation. The H0 -distribution is the standard normal distribution N (0, 1), the H1-distribution is N ( Fz (r) Fz (r0 ))/s, 1), with Fz (r )p = ln((1 + r )/(1 r ))/2 (Fisher z transformation) and s = 1/( N 3). 3.6 Validation The results in the special case of r0 = 0 were compared with the tabulated values published in Cohen (1969). The results in the general case were checked against the values produced by PASS (Hintze, 2006). 10 4 Exact: Proportion - difference from constant (one sample case) The relational value given in the input field on the left side and the two proportions given in the two input fields on the right side are automatically synchronized if you leave one of the input fields. You may also press the Sync values button to synchronize manually. Press the Calculate button to preview the effect size g resulting from your input values. Press the Transfer to main window button to (1) to calculate the effect size g = p p0 = P2 P1 and (2) to change, in the main window, the Constant proportion field to P1 and the Effect size g field to g as calculated. The problem considered in this case is whether the probability p of an event in a given population has the constant value p0 (null hypothesis). The null and the alternative hypothesis can be stated as: H0 : H1 : p p p0 = 0 p0 6= 0. A two-tailed binomial tests should be performed to test this undirected hypothesis. If it is possible to predict a priori the direction of the deviation of sample proportions p from p0 , e.g. p p0 < 0, then a one-tailed binomial test should be chosen. 4.1 4.2 Options The binomial distribution is discrete. It is thus not normally possible to arrive exactly at the nominal a-level. For twosided tests this leads to the problem how to \"distribute\" a to the two sides. G * Power offers the three options listed here, the first option being selected by default: Effect size index The effect size g is defined as the deviation from the constant probability p0 , that is, g = p p0 . The definition of g implies the following restriction: # (p0 + g) 1 #. In an a priori analysis we need to respect the additional restriction | g| > # (this is in accordance with the general rule that zero effect hypotheses are undefined in a priori analyses). With respect to these constraints, G * Power sets # = 10 6 . Pressing the Determine button on the left side of the effect size label opens the effect size drawer: 1. Assign a/2 to both sides: Both sides are handled independently in exactly the same way as in a one-sided test. The only difference is that a/2 is used instead of a. Of the three options offered by G * Power , this one leads to the greatest deviation from the actual a (in post hoc analyses). 2. Assign to minor tail a/2, then rest to major tail (a2 = a/2, a1 = a a2 ): First a/2 is applied to the side of the central distribution that is farther away from the noncentral distribution (minor tail). The criterion used for the other side is then a a1 , where a1 is the actual a found on the minor side. Since a1 a/2 one can conclude that (in post hoc analyses) the sum of the actual values a1 + a2 is in general closer to the nominal a-level than it would be if a/2 were assigned to both side (see Option 1). 3. Assign a/2 to both sides, then increase to minimize the difference of a1 + a2 to a: The first step is exactly the same as in Option 1. Then, in the second step, the critical values on both sides of the distribution are increased (using the lower of the two potential incremental avalues) until the sum of both actual a values is as close as possible to the nominal a. You can use this dialog to calculate the effect size g from p0 (called P1 in the dialog above) and p (called P2 in the dialog above) or from several relations between them. If you open the effect dialog, the value of P1 is set to the value in the constant proportion input field in the main window. There are four different ways to specify P2: Press the Options button in the main window to select one of these options. 4.3 Examples We assume a constant proportion p0 = 0.65 in the population and an effect size g = 0.15, i.e. p = 0.65 + 0.15 = 0.8. We want to know the power of a one-sided test given a = .05 and a total sample size of N = 20. 1. Direct input: Specify P2 in the corresponding input field below P1 2. Difference: Choose difference P2-P1 and insert the difference into the text field on the left side (the difference is identical to g). Select Type of power analysis: Post hoc 3. Ratio: Choose ratio P2/P1 and insert the ratio value into the text field on the left side Options Alpha balancing in two-sided tests: Assign a/2 on both sides 4. Odds ratio: Choose odds ratio and insert the odds ratio ( P2/(1 P2))/( P1/(1 P1)) between P1 and P2 into the text field on the left side. 11 Figure 6: Distribution plot for the example (see text) Input Tail(s): One Effect size g: 0.15 a err prob: 0.05 Total sample size: 20 Constant proportion: 0.65 would choose N = 16 as the result of a search for the sample size that leads to a power of at least 0.3. All types of power analyses except post hoc are confronted with similar problems. To ensure that the intended result has been found, we recommend to check the results from these types of power analysis by a power vs. sample size plot. Output Lower critical N: 17 Upper critical N: 17 Power (1-b err prob): 0.411449 Actual a: 0.044376 4.4 Related tests Similar tests in G * Power 3.0: Proportions: Sign test. The results show that we should reject the null hypothesis of p = 0.65 if in 17 out of the 20 possible cases the relevant event is observed. Using this criterion, the actual a is 0.044, that is, it is slightly lower than the requested a of 5%. The power is 0.41. Figure 6 shows the distribution plots for the example. The red and blue curves show the binomial distribution under H0 and H1 , respectively. The vertical line is positioned at the critical value N = 17. The horizontal portions of the graph should be interpreted as the top of bars ranging from N 0.5 to N + 0.5 around an integer N, where the height of the bars correspond to p( N ). We now use the graphics window to plot power values for a range of sample sizes. Press the X-Y plot for a range of values button at the bottom of the main window to open the Power Plot window. We select to plot the power as a function of total sample size. We choose a range of samples sizes from 10 in steps of 1 through to 50. Next, we select to plot just one graph with a = 0.05 and effect size g = 0.15. Pressing the Draw Plot button produces the plot shown in Fig. 7. It can be seen that the power does not increase monotonically but in a zig-zag fashion. This behavior is due to the discrete nature of the binomial distribution that prevents that arbitrary a value can be realized. Thus, the curve should not be interpreted to show that the power for a fixed a sometimes decreases with increasing sample size. The real reason for the non-monotonic behaviour is that the actual a level that can be realized deviates more or less from the nominal a level for different sample sizes. This non-monotonic behavior of the power curve poses a problem if we want to determine, in an a priori analysis, the minimal sample size needed to achieve a certain power. In these cases G * Power always tries to find the lowest sample size for which the power is not less than the specified value. In the case depicted in Fig. 7, for instance, G * Power 4.5 Implementation notes The H0 -distribution is the Binomial distribution B( N, p0 ), the H1 -distribution the Binomial distribution B( N, g + p0 ). N denotes the total sample size, p0 the constant proportion assumed in the null hypothesis, and g the effect size index as defined above. 4.6 Validation The results of G * Power for the special case of the sign test, that is p0 = 0.5, were checked against the tabulated values given in Cohen (1969, chapter 5). Cohen always chose from the realizable a values the one that is closest to the nominal value even if it is larger then the nominal value. G * Power , in contrast, always requires the actual a to be lower then the nominal value. In cases where the a value chosen by Cohen happens to be lower then the nominal a, the results computed with G * Power were very similar to the tabulated values. In the other cases, the power values computed by G * Power were lower then the tabulated ones. In the general case (p0 6= 0.5) the results of post hoc analyses for a number of parameters were checked against the results produced by PASS (Hintze, 2006). No differences were found in one-sided tests. The results for two-sided tests were also identical if the alpha balancing method \"Assign a/2 to both sides\" was chosen in G * Power . 12 Figure 7: Plot of power vs. sample size in the binomial test (see text) 13 5 Exact: Proportion - inequality, two dependent groups (McNemar) the central distribution that is farther away from the noncentral distribution (minor tail). The criterion used on the other side is then a a1 , where a1 is the actual a found on the minor side. Since a1 a/2 one can conclude that (in post hoc analyses) the sum of the actual values a1 + a2 is in general closer to the nominal a-level than it would be if a/2 were assigned to both sides (see Option 1). This procedure relates to tests of paired binary responses. Such data can be represented in a 2 2 table: Treatment Yes No Standard Yes No p11 p12 p21 p22 ps 1 ps pt 1 pt 1 3. Assign a/2 to both sides, then increase to minimize the difference of a1 + a2 to a: The first step is exactly the same as in Option 1. Then, in the second step, the critical values on both sides of the distribution are increased (using the lower of the two potential incremental avalues) until the sum of both actual a values is as close as possible to the nominal a. where pij denotes the probability of the respective response. The probability p D of discordant pairs, that is, the probability of yes/no-response pairs, is given by p D = p12 + p21 . The hypothesis of interest is that ps = pt , which is formally identical to the statement p12 = p21 . Using this fact, the null hypothesis states (in a ratio notation) that p12 is identical to p21 , and the alternative hypothesis states that p12 and p21 are different: H0 : H1 : 5.2.2 You may choose between an exact procedure and a faster approximation (see implementation notes for details): 1. Exact (unconditional) power if N < x. The computation time of the exact procedure increases much faster with sample size N than that of the approximation. Given that both procedures usually produce very similar results for large sample sizes, a threshold value x for N can be specified which determines the transition between both procedures. The exact procedure is used if N < x; the approximation is used otherwise. p12 /p21 = 1 p12 /p21 6= 1. In the context of the McNemar test the term odds ratio (OR) denotes the ratio p12 /p21 that is used in the formulation of H0 and H1 . 5.1 Effect size index Note: G * Power does not show distribution plots for exact computations. The Odds ratio p12 /p21 is used to specify the effect size. The odds ratio must lie inside the interval [10 6 , 106 ]. An odds ratio of 1 corresponds to a null effect. Therefore this value must not be used in a priori analyses. In addition to the odds ratio, the proportion of discordant pairs, i.e. p D , must be given in the input parameter field called Prop discordant pairs. The values for this proportion must lie inside the interval [#, 1 #], with # = 10 6 . If p D and d = p12 p21 are given, then the odds ratio may be calculated as: OR = (d + p D )/(d p D ). 5.2 2. Faster approximation (assumes number of discordant pairs to be constant). Choosing this option instructs G * Power to always use the approximation. 5.3 Examples As an example we replicate the computations in O'Brien (2002, p. 161-163). The assumed table is: Treatment Yes No Options Press the Options button in the main window to select one of the following options. 5.2.1 Computation Standard Yes No .54 .08 .32 .06 .86 .14 .62 .38 1 In this table the proportion of discordant pairs is p D = .32 + .08 = 0.4 and the Odds Ratio OR = p12 /p21 = 0.08/.32 = 0.25. We want to compute the exact power for a one-sided test. The sample size N, that is, the number of pairs, is 50 and a = 0.05. Alpha balancing in two-sided tests The binomial distribution is discrete. It is therefore not normally possible to arrive at the exact nominal a-level. For two-sided tests this leads to the problem how to \"distribute\" a to the two sides. G * Power offers the three options listed here, the first option being selected by default: Select Type of power analysis: Post hoc 1. Assign a/2 to both sides: Both sides are handled independently in exactly the same way as in a one-sided test. The only difference is that a/2 is used instead of a. Of the three options offered by G * Power , this one leads to the greatest deviation from the actual a (in post hoc analyses). Options Computation: Exact Input Tail(s): One Odds ratio: 0.25 a err prob: 0.05 Total sample size: 50 Prop discordant pairs: 0.4 2. Assign to minor tail a/2, then rest to major tail (a2 = a/2, a1 = a a2 ): First a/2 is applied to the side of 14 Output Power (1-b err prob): 0.839343 Actual a: 0.032578 Proportion p12: 0.08 Proportion p21: 0.32 also in the two-tailed case when the alpha balancing Option 2 (\"Assign to minor tail a/2, then rest to major tail\Tutorials!in!Quantitative!Methods!for!Psychology! 2007,!Vol.!3(2),!p.!51"59.! A!short!tutorial!of!GPower! ! ! ! ! ! Susanne!Mayr! Heinrich"Heine"Universitt,!Dsseldorf,!Germany! Edgar!Erdfelder! Universitt!Mannheim,!Mannheim,!Germany! Axel!Buchner! Heinrich"Heine"Universitt,!Dsseldorf,!Germany! ! Franz!Faul! Christian"Albrechts"Universitt,!Kiel,!Germany! The!purpose!of!this!paper!is!to!promote!statistical!power!analysis!in!the!behavioral!sciences!by!introducing! the! easy! to! use! GPower! software.! GPower! is! a! free! general! power! analysis! program! available! in! two! essentially! equivalent! versions,! one! designed! for! Macintosh! OS/OS! X! and! the! other! for! MS"DOS/Windows! platforms.! Psychological! research! examples! are! presented! to! illustrate! the! various! features! of! the! GPower! software.! In! particular,! a! priori,! post"hoc,! and! compromise! power! analyses! for! t"tests,! F"tests,! and! !2"tests! will!be!demonstrated.!For!all!examples,!the!underlying!statistical!concepts!as!well!as!the!implementation!in! GPower!will!be!described.! ! " In!the!behavioral!sciences,!we!routinely!apply!statistical! tests,! but! control! of! statistical! power! cannot! be! taken! for! granted.! However,! neglecting! statistical! powerthe! probability! of! rejecting! false! null! hypothesescan! have! severe! consequences.! For! example,! without! control! of! statistical! power! it! is! very! difficult! to! interpret! nonsignificant! results.! Statistical! tests! can! produce! nonsignificant! results! because! (a)! the! null! hypothesis! (H0)! holds! and! is! retained! correctly! or! (b)! the! alternative! hypothesis! (H1)! holds! but! the! test! has! not! been! powerful! enough!to!detect!the!deviations!from!H0.!Obviously,!there!is! no!reasonable!way!to!decide!between!interpretations!(a)!and! (b)! when! the! power! of! the! test! is! unknown.! As! a! result! of! neglecting! statistical! power! analyses,! null! results! are! published! only! rarely.! Thus,! the! publication! of! research! findings! is! biased! in! favor! of! H1! hypotheses! (Bredenkamp,! 1972,!1980).!! The!omission!of!power!control!is!frequently!justified!by! the! argument! that! power! analyses! are! too! complex! to! perform.!The!GPower!software! 1 !(Erdfelder,!Faul,!&!Buchner,! 1996)1 " ! presented! in! this! article! should! largely! render! this! argument! obsolete.! GPower! is! an! easy! to! use! program! for! performing!various!types!of!power!analysis.!This!paper!tries! to! familiarize! readers! with! the! concept! of! statistical! power! analysis!in!general!and!with!GPower!in!particular.! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!! " !Susanne!Mayr,!Department!of!Experimental!Psychology,!Heinrich" Types!of!power!analyses! Heine"University,! Different! types! of! power! analysis! can! be! distinguished! with!respect!to!their!intended!purposes.!We!want!to!present! the! two! most! common! typesa! priori! and! post"hoc! power! analysisas! well! as! a! third! variant,! compromise! power! Dsseldorf,! Germany;! Edgar! Erdfelder,! Department! of! Psychology,! Mannheim! University,! Mannheim,! Germany;! Axel! Buchner,! Department! of! Experimental! Psychology,! Heinrich"Heine"University,! Dsseldorf,! Germany;! Franz! Faul,! Department! of! Psychology,! Christian"Albrechts"University,! Kiel,! Germany.! Correspondence! concerning! this! article! should! be! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!! addressed!to!Susanne!Mayr,!Institut!fr!Experimentelle!Psychologie,! 1 Heinrich"Heine"Universitt,! Germany.! http://www.psycho.uni"duesseldorf.de/aap/projects/gpower.! Note! Electronic! mail! may! be! sent! to! susanne.mayr@uni"duesseldorf.de.! that!this!tutorial!refers!to!GPower!Version!2.!By!now,!Version!3!(Faul,! D"40225! Dsseldorf,! ! GPower! is! free! and! may! be! downloaded! from! This! work! is! based! on! the! German! language! tutorials! by! Buchner,! Erdfelder,!Lang,!&!Buchner,!2007)!is!already!available!via!the!same! Erdfelder,!and!Faul!(1996)!and!Erdfelder,!Buchner,!Faul,!and!Brandt! weblink.! Version! 3! comprises! an! extended! functionality! which! (2004).! might!be!worthwhile!for!the!interested!reader.! ! ! ! 51! ! analysis.! All! three! types! can! be! accomplished! with! the! GPower!software.!! An!a!priori!analysis!is!done!before!a!study!takes!place.!It! is!the!ideal!type!of!power!analysis!because!it!provides!users! with!a!method!to!control!both!the!type"1!error!probability! #! (i.e.,!the!probability!of!incorrectly!rejecting!H0!when!is!in!fact! true)!and!the!type"2!error!probability! $!(i.e.,!the!probability! of! incorrectly! retaining! H0! when! it! is! in! fact! false).! By! implication,!it!also!controls!the!power!of!the!test,!that!is,!the! complement! of! the! type"2! error! probability! (1! "! $)! (i.e.,! the! probability!of!correctly!rejecting!H0!when!it!is!in!fact!false).! An! a! priori! analysis! is! used! to! determine! the! necessary! sample! size! N! of! a! test! given! a! desired! #! level,! a! desired! power! level! (1! "! #),! and!the!size!of!the!effect!to!be!detected! (i.e.,!a!measure!of!the!difference!between!the!H0!and!the!H1).! In! contrast,! a! post"hoc! analysis! is! typically! performed! after!a!study!has!been!conducted!so!that!the!sample!size!N!is! already! a! matter! of! fact.! Given! N,! !,! and! a! specified! effect! size,!this!type!of!analysis!returns!the!power!(1!-!!),!or!the!!! error! probability! of! the! test.! Obviously,! post"hoc! analyses! are! less! ideal! than! a"priori! analyses! because! only! #! is! controlled,! not! $.! Both! $! and! its! complement! (1! "! $)! are! assessed! but! not! controlled! in! post"hoc! analyses.! Thus,! post" hoc! power! analyses! can! be! characterized! as! instruments! providing!for!a!critical!evaluation!of!the!(often!surprisingly! large)!error!probability!!!associated!with!a!false!decision!in! favor!of!the!H0.!! The! third! type! of! power! analysis! provided! by! GPower,! compromise! power! analysis,! provides! a! pragmatic! solution! to!the!frequently!encountered!problem!that!the!ideal!sample! size!N!calculated!by!an!a"priori!power!analysis!exceeds!the! available! resources! (Erdfelder,! 1984).! For! example,! clinical! investigators! are! sometimes! interested! in! diseases! or! disorders!of!a!very!low!prevalence!for!which!the!number!of! available! participants! is! small.! In! spite! of! these! suboptimal! circumstances,!a!fair!decision!between!H0!and!H1!is!possible.! For! this! situation,! a! reasonable! compromise! between! a! preferably!small!#!and!a!preferably!large!power!(1!-!#)!has! to!be!found.!To!this!end,!a!decision!has!to!be!made!of!how! important!#!should!be!in!comparison!to!#.!This!weighting!is! expressed!by!the!factor!q!(q!=!#!/!#).!Based!on!N,!q,!and!the! specified! effect! size,! the! compromise! power! analysis! then! determines! #! and! #,! and! the!associated!critical!value!of!the! relevant! test! statistic.! In! other! words,! compromise! power! analyses! control! the! error! probability! ratio! q! =! $/#.! Both! #! and! $! are! assessed! given! a! fixed! error! probability! ratio! q.! Note! that! compromise! analyses! can! also! be! very! useful! when! the! available! N! is! \"too! large\".! For! example,! in! goodness"of"fit! tests,! very! large! sample! sizes! are! not! unusual.!Under!these!conditions,!even!negligible!deviations! of! the! empirical! data! structure! from! the! data! structure! implied! by! the! model! (H0)! may! lead! to! model! rejections! if! ! ! 52! conventional!significance!levels!like!#!=!.05!are!used.!In!such! situations,!compromise!power!analyses!provide!users!with!a! method!to!find!more!reasonable,!strict!decision!criteria!such! that! effect! sizes! of! interest! are! detected! with! balanced! probabilities! #!and! $!consistent!with!the!user"defined!error! probability!ratio!q!=!$/#.! Examples!of!statistical!power!analyses!with!GPower! We! will! present! examples! of! statistical! power! analysis! for! the! three! most! often! applied! statistical! tests! in! psychological! research,! that! is! t",! F",! and! !2"tests.! We! will! describe!how!to!obtain!calculations!of!sample!size!(in!case!of! a! priori! analyses),! statistical! power! (in! case! of! post"hoc! analyses),! and! !! and! !! values! (in! case! of! compromise! analyses)! using! the! GPower! program.! GPower! exists! in! two! versions! that! are! equivalent! in! their! numerical! implementation.! One! version! is! MS"DOS! compatible! and! may! be! run! under! Windows;! the! other! version! has! been! designed! for! Mac! OS! 7! to! 9! and! may! be! run! in! the! classic! mode!of!Mac!OS!X.!All!explanations!and!figures!refer!to!the! Macintosh!version;!however,!given!that!the!user!interface!of! the! two! versions! is! very! similar,! no! difficulties! should! emerge! in! following! the! descriptions! for! users! of! the! MS" 2 DOS!version " .! Power!analyses!for!t"tests! Independent!samples!t"test! A!frequently!cited!study!by!Warrington!and!Weiskrantz! (1970,!Experiment!2)!compared!the!memory!performance!of! amnesic!patients!with!that!of!control!subjects.!In!addition!to! commonly! used! direct! memory! tests,! such! as! a! recall! test,! indirect!memory!measures,!such!as!a!word"stem!completion! test,! were! used.! Indirect! tests! are! thought! to! measure!after" effects!of!experiences!without!giving!the!explicit!instruction! to! remember.! Whereas! the! amnesic! patients! performed! worse! than! controls! in! the! recall! test! (means! of! 8! vs.! 13),! there! was! no! significant! difference! between! the! groups! in! the!word"stem!completion!test!(means!of!14.5!vs.!16).!! Do! these! results! prove! that! amnesics! are! as! good! as! controls!in!indirect!test!performance,!at!least!with!respect!to! word"stem! completion?! Looking! at! the! sample! means,! we! note! a! difference! between! amnesics! and! controls! in! the! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!! 2 ! Program!users!can!select!between!an!accuracy!mode!and!a!speed! mode! (see! the! I! prefer...! option! in! Figure! 1).! Whereas! accuracy! mode!calculations!are!based!on!the!actual!noncentral!distribution!of! the!relevant!test!statistic,!speed!mode!calculations!approximate!this! distribution! by! other! types! of! distributions.! The! computational! capability! of! present"day! computers! allows! using! the! accuracy! mode!without!hesitation.! ! ! Figure 1: GPower display of a post-hoc power analysis for a t-test (means) situation. For details see text. word"stem! completion! task.! Relative! to! controls,! amnesic! patients!completed!fewer!word!stems!with!words!they!had! seen! before.! Taking! into! account! that! the! sample! included! only!4!amnesics!and!8!controls,!the!statistical!power!of!the!t" test! for! independent! samples! must! have! been! rather! small.! Additionally,! the! unequal! sample! sizes! in! the! two! groups! tend! to! reduce! statistical! power.! This! is! evident! when! we! have! a! look! at! the! noncentrality! parameter! $! which! defines! the! noncentral! t"distribution! under! H1! and! reflects! the! degree!to!which!H0!is!false!(Johnson!&!Kotz,!1970,!Chap.!31):! n &n ! % ' d& 1 2 ! (1)! N with! n1! and! n2! as! the! sample! sizes! of! the! two! groups! (amnesics! and! controls),! N! =! n1! +! n2,! and! d! =! ($1! "! $2 ) /! %.! The!symbol!d!(commonly!called!Cohen's!d)!is!the!effect!size! index!for!independent!samples!t"tests!used!by!Cohen!(1988).! $1! and! $2! are! the! population! means! of! the! two! groups.! For! standardization! purposes,! the! difference! of! population! means!is!divided!by!the!common!standard!deviation!of!the! two!populations,!%.!H0!of!the!one"tailed!t"test!assumes!$2!"!$1! %!0,!H1!assumes!$2!"!$1!>!0.!For!a!specified!total!sample!size! and!a!given!d,!Equation!(1)!shows!that!the!more!unequal!the! group! sizes,! the! smaller! $! will! be,! and! with! it,! the! smaller! 3" will!be!the!statistical!power. ! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!! "3 ! Note! that! the! relationship! between! the! difference! of! the! sample! sizes! n1! and! n2! and! power! is! modulated! by! the! size! and! the! magnitude!of!disparity!of!the!standard!deviations!in!the!two!groups! that!enter!into!the!calculation!of!Cohen's!d.!When!the!two!standard! deviations! are! different! in! size,! power! will! vary! depending! on! which! group! (the! larger! or! the! smaller)! has! the! larger! standard! deviation!and!on!the!magnitude!of!this!disparity!(see!e.g.!Myers!&! Well,!1995).!For!the!example!chosen!here,!this!complication!of!affairs! is! not! of! any! relevance! because! we! assume! equality! of! standard! deviations!for!the!two!groups!(see!next!paragraph).! ! ! 53! Figure!2:!GPower display!of!a!compromise!power!analysis!for! a!t"test!(means)!situation.!For!details!see!text.! But!how!large!was!the!statistical!power!for!the!reported! results! of! Warrington! and! Weiskrantz! (1970)! in! the! word" stem! completion! task,! if! we! assume! that! the! underlying! population! means! equalled! 14.5! for! the! amnesic! patient! group!and!16!for!the!control!group?!Let!us!assume!that!the! standard! deviation! of! test! performance! equalled! 3! in! the! underlying! populations! of! each! group! (unfortunately,! neither! the! standard! deviation! of! the! samples! nor! the! empirical! t"values! have! been! reported).! In! GPower! we! have! to! choose! Post"hoc! as! type! of! power! analysis! and! t"Test! (means)! as! type! of! test! (see! Figure! 1).! Because! the! hypothesis! is! directionalwe! want! to! know! whether! controls! are! better! than! amnesicsa! one"tailed! test! is! selected.! Next,! we! determine! with! Calc! \"d\"! d! =! (16! -! 14.5)/3!=!0.5!as!the!size!of!the!effect!to!be!detected.!An!effect! of! this! size! equals! \"medium\"! effects! in! terms! of! Cohen's! (1988)! conventions.! What! was! the! probability! to! find! this! effect!given!a!level!of!!!=!.05?!We!specify!!!=!.05,!n1!=!4,!and! n2! =! 8.! The! result! is! disillusioning.! The! statistical! power! of! this! test! amounts! to! only! .1887.! GPower! also! returns! the! critical! t"value! associated! with! the! chosen! !! level,! that! is,! t(10)! =! 1.8125,! and! the! noncentrality! parameter! %! =! 0.8165! determined! by! sample! size! and! specified! effect! size! d! (see! Equation!1).! Conclusion:! There! was! hardly! any! chance! to! detect! a! medium! sized! deficit! of! amnesics! in! Warrington! and! Weiskrantz'! (1970)! word"stem! completion! task.! We! can! use! the!Post"hoc!type!of!power!analysis!to!determine!of!what! size! the! performance! difference! between! groups! in! the! word"stem!completion!task!necessarily!would!have!been!to! find! this! difference! with! a! statistical! power! of! .95.! To! this! end,!we!have!to!keep!the!program!inputs!as!specified!above! (!,!n1,!n2),!but!increase!the!effect!size!\"d\"!until!the!calculated! statistical! power! reaches! .95.! This! happens! with! an! effect! size! of! d! =! 2.1694.! This! standardized! effect! size! value! of! 2.1694! can! be! recalculated! into! a! performance! difference! in! terms!of!the!word"stem!completion!task!(by!transforming!d! ! ! =!(!1!"!!2) /! ("!into!(!1!"!!2) = d!"!and!by!inserting!the!values! of!the!example,!2.1694!!3!=!6.5082).!This!result!implies!that!a! population! mean! difference! not! less! than! 6.5082! words! in! favor! of! the! control! group! would! have! been! necessary! to! achieve!a!power!of!.95.!! Alternatively,!if!we!want!to!detect!an!effect!of!size!d!=!0.5! with! n1! =! 4,! n2! =! 8,! and! equally! large! #! and! !! error! probabilities! (q! =! 1),! the! Compromise! option! has! to! be! chosen! as! type! of! power! analysis! (see! Figure! 2).! Here,! we! specify! the! Beta/alpha! ratio! as! \"1\"! if! we! consider! both! 4 types!of!error!as!equally!serious. " !Then!GPower!returns!!!=!!! =! .3422! (associated! with! a! critical! value! of! t(10)! =! 0.4186).! Under! the! prevailing! circumstances,! choosing! this! significance!level!is!the!best!possible!decision.!However,!this! statistical! test! is! hardly! any! better! than! tossing! a! coin! to! decide!whether!to!accept!or!reject!H0.!! Paired!samples!t"test! In! succession! of! Gesell! and! Thompson's! (1929)! work,! a! number! of! experiments! with! monozygotic! pairs! of! twins! have! been! conducted! giving! one! randomly! chosen! twin! training!of!specific!motor!skills!while!the!other!one!did!not! obtain! any! training! program.! This! allowed! for! a! controlled! investigation! of! whether! certain! abilities! (e.g.! learning! to! walk,! control! of! the! bladder)! develop! in! a! process! of! maturation! or! whether! environmental! influences! can! promote!or!impair!this!development.! Imagine! that! we! want! to! replicate! such! a! twin! study! which!shall!be!analyzed!with!a!paired!samples!t"test.!Let!us! further!assume!that!there!is!only!a!pool!of!20!pairs!of!twins! available.!What!are!reasonable!error!probabilities!we!have!to! accept!for!our!statistical!test?! X! and! Y! denote! the! age! at! which! the! trained! and! the! untrained! twin,! respectively,! will! master! a! special! motor! ability.! H0! of! the! one"tailed! paired! samples! t"test! is! characterized! by! "x"y! =! "x "! "y! %! 0,! with! "x"y! denoting! the! population! mean! of! the! age! differences! of! each! twin! pair.! The!effect!size!dz!is!defined!as:! ! dz ' )x*y ( x* y ' )x*y ( x2 + ( y2 * 2covxy ' )x*y ( x2 + ( y2 * 2,( x( y ,! (2)! with! #x"y! being! the! standard! deviation! of! the! (X! -! Y)! differences,! covxy! being! the! covariance,! and! "! being! the! (positive)! correlation! between! the! X! and! Y! values! in! the! population! given! H1! is! true.! Other! things! being! equal,! the! larger!the!correlation! $,!the!smaller!the!denominator!will!be,! and! the! larger! will! be! the! effect! size! index! dz.! If! H1! is! true,! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! " 4 ! ! Alternatively,! values! of! q! >! 1! could! be! used! if! a! type"2! error! is! 54! the! distribution! of! our! test! statistic! is! the! noncentral! t" distribution! with! N! -! 1! degress! of! freedom! (N! denotes! the! number! of! twin! pairs,! i.e.! the! measurement! pairs)! and! a! noncentrality!parameter! ! %' )x * y ( x* y & N ' dz & N .! Let!us!assume!that!on!average!the!developmental!difference! in!a!specific!motor!skill!amounts!to!2!months.!For!a!specific! motor!skill,!the!standard!deviation!of!the!age!difference!may! amount! to! 4! months.! Hence,! following! Equation! (2),! the! effect!size!to!be!detected!with!this!replication!study!equals!dz! =!2/4!=!0.5.!Because!we!want!to!decide!upon!the!size!of!the!!! and! !! error! probabilities! given! N! and! dz! are! fixed,! we! need! the! Compromise! analysis! in! GPower.! The! option! t"Test! (means)!we!have!used!in!the!previous!example!is!based!on! independent! samples! and! calculates! the! degrees! of! freedom! as! N!-!2.!This!is!no!longer!adequate!for!the!current!situation! because!the!twin!data!are! dependent.!For!a!paired!samples! t" test!there!are! N!-!1!degrees!of!freedom.!Therefore,!we!have! to! choose! the! option! Other! t"Tests! for! which! the! degrees! of! freedom! can! be! determined! independently! of! N.! The! hypothesis! is! directional! againwe! want! to! know! whether! trained! twins! are! beyond! their! untrained! siblings! in! their! motor! skill! developmentso! that! we! choose! the! one" tailed! option.! In! Other! t"Tests,! the! to"be"specified! effect! 5 size!is!labelled! f!instead!of! d. " !The!noncentrality!parameter! is!calculated!as!follows:! % ' f & N .! ! (4)! Comparing!Equations!(3)!and!(4)!we!see!that!the!calculated! dz!value!(i.e.!0.5)!can!be!inserted!for!the!effect!size!f!to!obtain! the!correct!noncentrality!parameter!for!matched"pairs! t"tests! using!the!Other!t"Tests!option.!N!has!to!be!set!to!20!(20! pairs! of! twins! were! available).! If! !! and! !! shall! be! of! same! size,!the!Beta/alpha!ratio!option!again!has!to!be!set!to!\"1\".! The!test!has! N!-!1!=!19!degrees!of!freedom!(DF!for!t"Test).! GPower! returns! a! noncentrality! parameter! of! %!=!2.2361!and! recommends! to! choose! !! =! !! =! .1357.! For! this! situation! the! power! is! 1! "! !! =! .8643.! In! order! to! reject! H0!(i.e.,!in!order!to! reject! the! hypothesis! of! no! differences! between! the! twins,! which! implies! rejecting! the! maturation! hypothesis)! and! to! accept!the! H1!(i.e.,!to!accept!the!\"environmental!influences\"! hypothesis),! the! empirical! t"value! has! to! exceed! the! critical! value! t(19)! =! 1.1328.! Even! though! this! result! is! less! devastating! than! that! of! the! previous! example,! there! is! nevertheless! a! large! error! probability! associated! with! each! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! "5 !The!reason!for!using!the!symbol!f!rather!than!d!is!that!the!Other!t" Tests! option! of! GPower! has! been! designed! to! provide! power! considered!less!serious!than!a!type"1!error.! analyses!for!any!type!of!t"test,!not!just!t"tests!for!means.! ! ! ! (3)! ! ! decision! possible.! In! order! to! reduce! this! error! probability,! the!sample!size!would!have!to!be!increased.!To!what!extend! we! would! have! to! increase! the! sample! size! can! be! incrementally! determined! with! the! Post"hoc! analysis! option.!A!Post"hoc!analysis!again!with!the!options!Other! t"Tests!and!one"tailed!as!well!as!the!specifications!!!=!.05,! f! =! 0.5,! N! =! 45,! and! df! =! 44! returns! a! power! value! of! 1! "! !! =! .9512.! A! sample! of! this! size! is! necessary! to! obtain! a! power! level!above!.95.! In! the! above! example,! pairs! of! twins! provided! the! dependent! data.! As! a! matter! of! course,! we! would! have! to! proceed!analogously!for!other!kinds!of!dependent!data,!for! example,!for!repeated!measurements!of!the!same!subjects.! 6" t"test!for!correlations ! Berry! and! Broadbent! (1984)! investigated! the! relation! between! task! performance! in! controlling! a! computer! simulation!and!verbalizable!knowledge!about!the!simulated! system.! Experiment! 1! found! a! negative! correlation! (in! the! range!of!".25!and!".30)!between!both!variables.!The!better!the! participants! controlled! the! simulation,! the! worse! they! were! able!to!provide!information!about!the!simulated!system.!! However,! this! negative! correlation! was! not! statistically! significant.!The!authors!attributed!the!lack!of!significance!to! the! small! sample! size! (N! =! 12).! But! how! large! was! the! probability! to! find! a! correlation! of! a! specified! size! in! this! study?!! Have! a! look! at! the! definition! of! the! noncentrality! parameter! % for! t"tests! for! correlations! between! two! variables:! %' ! ,2 & N ,! 1* ,2 (5)! 55! Figure 3: GPower display of an a priori power analysis for a t-test (correlations) situation. For details see text. hoc!as!type!of!power!analysis!and!t"Test!(correlations)!as! type!of!test.!The!test!is!one"tailed!because!we!want!to!test! H0:!,!-!0!versus!H1:!,!