Effect size and rehabilitation research

Research in the area of rehabilitation has long relied on the test of statistical significance as the primary means for evaluating the meaningfulness of empirical research results. The concept of statistical significance is deeply rooted in the null hypothesis inference testing model and perpetuates the myth that it is not possible to draw any meaningful conclusion without a significant test result. Under the null hypothesis inference testing model, most tests of statistical significance are the product of the sample size utilized in the study and size of the effect (significance = effect size x sample size) (Rosenthal & Rosnow, 1991). Reliance on the null hypothesis significance testing model has had significant implications for rehabilitation research and the Journal because most rehabilitation researchers are faced with samples that come from two extremes. One extreme are samples that come from large data sets such as RSA-911 database. With these samples researchers often find significant test results but differences that are of little practical significance. If only statistical significance is reported it may lead to an inappropriate interpretation of the findings. At the other extreme are small samples that are commonly drawn from surveys or agency participants. With these samples researchers very often find insignificant results but differences that are of practical significance for the rehabilitation practitioner. Again if only statistical significance is reported it may lead to an inappropriate interpretation of the findings. To address the problems associated with the null hypothesis inference testing model, psychology and related fields are starting to move in the direction of placing more emphasis on practical significance, the degree of relationship between variables, or the magnitude of the effect, instead of tests of statistical significance. For example, the 5th edition of the American Psychological Association Manual (APA, 2001) emphasizes the necessity to include some index of effect size or strength of relationship in psychological research further stating that the failure to report indicators of effect or strength of relationship to be a defect (p.5). In addition, 23 professional research journals have also identified the importance of reporting effect sizes by requiring authors and researchers to report indicators of effect in their papers submitted for publication (Harris, 2003; Snyder, 2000; Trusty, Thompson, & Petrocelli, 2004). In simplistic terms, effect size refers to any statistic that describes the degree of difference or relationship between the variables of interest. According to Vacha-Haase and Thompson (2004) the following three major types of effect size indicators can be utilized in psychological research: standardized differences, variance accounted for effects, and corrected effect sizes. As editors of the Journal we also feel that these indicators of effect size have direct application to rehabilitation research and therefore will require that indicators of effect be incorporated into empirically-based research articles that are submitted for publication to the Journal. The utilization of effect size indicators for research published in the journal will allow the readers and rehabilitation researchers to better utilize the research published in the journal to make programmatic decisions or further research in the area of rehabilitation. As a result of our requiring that indicators of effect be included in empirical research submitted to the journal, we are offering the following recommendations: 1) It will be important for researchers seeking publication in the journal to get familiar with indicators of effect size as they apply to rehabilitation research. There are numerous articles and statistical textbooks that provide a very good and easy to understand explanation of indicators of effect. Many of these articles and texts are included in the reference list of this editorial. …