The following example shows that the power of a test can sometimes be increased by selecting a random rather than a fixed sample size even when the randomization does not depend on the observations. Let X1,..., Xn be independently distributed as N(θ, 1), and consider the problem of testing H : θ = 0 against K : θ = θ1 > 0.

(i) The power of the most powerful test as a function of the sample size n is not necessarily concave.

(ii) In particular for α = 0.005, θ1 = 1 2 , better power is obtained by taking 2 or 16 observations with probability 1 2 each than by taking a fixed sample of 9 observations.

(iii) The power can be increased further if the test is permitted to have different significance levels α1 and α2 for the two sample sizes and it is required only that the expected significance level be equal to α = 0.005. Examples are: (a)

with probability 1 2 take n1 = 2 observations and perform the test of significance at level α1 = 0.001, or take n2 = 16 observations and perform the test at level

α2 = 0.009;

(b) with probability 1 2 take n1 = 0 or n2 = 18 observations and let the respective significance levels be α1 = 0, α2 = 0.01.

Note. This and related examples were discussed by Kruskal in a seminar held at Columbia University in 1954. A more detailed investigation of the phenomenon has been undertaken by Cohen (1958).