Sample Size and Hypothesis Testing

Hypothesis testing determines if there is sufficient evidence to support a claim (the statistical hypothesis) about a population parameter based on a sample of data. Right-sizing experiments involve trade-offs involving the probabilities of different kinds of false claims, precision of estimates, and operational and ethical constraints on sample size. Power is the probability of obtaining a true positive (correctly rejecting the null hypothesis when the alternate hypothesis is true). Significance is the probability of obtaining a false positive. The non-centrality parameter measures the difference between population means under the alternative hypothesis and is a method of evaluating the power of the test. Simulations based on the non-centrality parameter are a useful method for determining sample size and power when conventional sample size formulae are not applicable. The role of sample size balance and allocation ratio is discussed.