Null Hypothesis Significance Testing

Abstract Null hypothesis significance testing (NHST) is an inferential statistical method for deciding whether a well‐specified hypothesis, identified as the null hypothesis , is to be regarded as true for a population from which a given set of data has been obtained by random sampling. In the usual procedure the data from a particular dependent (i.e., measured) variable are first summarized by a single number called a test statistic. Usually, some assumptions must then be made in order to find the relative likelihoods of all possible values of that test statistic when the null hypothesis is true, and thus to find the null hypothesis distribution (NHD). The next step is to calculate the probability of obtaining one's actual test statistic from the NHD, or one that is even further from the mean of the NHD. From what is called the “frequentist” point of view, that probability, called a p value , tells us the proportion of times that the NHD would yield a test statistic at least as inconsistent with the null hypothesis as the one you obtained, over many exact replications of your study. In the accept‐support (AS) form of NHST, researchers actually want to obtain a p value that is close to its maximum of 1.0, because the null hypothesis being tested is consistent with the theory that motivated the study.