Judicious Use of Multiple Hypothesis Tests |
| |
Authors: | PAUL J ROBACK ROBERT A ASKINS†‡ |
| |
Institution: | Department of Mathematics, Statistics, and Computer Science, Saint Olaf College, 1520 Saint Olaf Avenue, Northfield, MN 55057, U.S.A.;Department of Biology, Connecticut College, 270 Mohegan Avenue, New London, CT 06320–4196, U.S.A., email |
| |
Abstract: | Abstract: When analyzing a table of statistical results, one must first decide whether adjustment of significance levels is appropriate. If the main goal is hypothesis generation or initial screening for potential conservation problems, then it may be appropriate to use the standard comparisonwise significance level to avoid Type II errors (not detecting real differences or trends). If the main goal is rigorous testing of a hypothesis, however, then an adjustment for multiple tests is needed. To control the familywise Type I error rate (the probability of rejecting at least one true null hypothesis), sequential modifications of the standard Bonferroni method, such as Holm's method, will provide more statistical power than the standard Bonferroni method. Additional power may be achieved through procedures that control the false discovery rate (FDR) (the expected proportion of false positives among tests found to be significant). Holm's sequential Bonferroni method and two FDR-controlling procedures were applied to the results of multiple-regression analyses of the relationship between habitat variables and the abundance of 25 species of forest birds in Japan, and the FDR-controlling procedures provided considerably greater statistical power. |
| |
Keywords: | Bonferroni method false discovery rate Holm's method hypothesis generation multiple hypothesis testing q value |
|
|