Biological Importance and Statistical Significance
Journal of Agricultural and Food Chemistry
January 1, 2013
Task Force #12
Journal of Agricultural and Food Chemistry. 2013;61(35):8340-8348
Abstract: Statistical ideas behind the analysis of experiments related to crop composition and the genetic factors underlying composition are discussed. The emphasis is on concepts rather than statistical formulations. Statistical analysis and biological considerations are shown to be complementary rather than contradictory, in that the statistical analysis of a data set depends on the experimental design, that no amount of statistical sophistication can rescue a badly designed study, and that good experimental design is crucial. The traditional null hypothesis significance testing approach has severe limitations, but p values and statistical significance still often seem to be the primary objective of an analysis. Emphasis instead should be on identifying the size of effects that are biologically important and, with the involvement of the “domain” scientist, using these to help design experiments with appropriate sample sizes and statistical power. The issues discussed here are also directly applicable to other areas of research.
To download this article, click here.
- Box, J. F. R. A. Fisher: The Life of a Scientist; Wiley: New York, 1978. LINK
- Edwards, A. W. R. A. Fisher. Twice professor of genetics: London and Cambridge or “a fairly well-known geneticist” Biometrics 1990, 46, 897– 904 LINK
- Smith, A. F. M. Mad cows and ecstasy: chance and choice in an evidence-based society J. R. Stat. Soc. Ser. A: Stat. Soc. 1996, 159, 367– 384 LINK
- EFSA Scientific Committee. Statistical significance and biological relevance. EFSA J. 2011, 9, 2372 (17 pp). LINK
- EFSA Panel on Genetically Modified Organisms (GMO). Guidance on the environmental risk assessment of genetically modified plants. EFSA J. 2010, 8, 1879 (111 pp). LINK
- EFSA Panel on Genetically Modified Organisms (GMO). Statistical considerations for the safety evaluation of GMOs. EFSA J. 2010, 8, 1250 (59 pp). LINK
- EFSA Scientific Committee. Guidance on conducting repeated-dose 90-day oral toxicity study in rodents on whole food/feed. EFSA J. 2011, 9, 2438 (21 pp). LINK
- Fisher, R. Presidential address to the First Indian Statistical Congress Sankhya 1938, 4, 14– 17
- Price, W. D.; Underhill, L. Application of laws, policies, and guidance from the United States and Canada to the regulation of food and feed derived from genetically modified crops: interpretation of composition data.. J. Agric. Food Chem. 2013, DOI: 10.1021/jf401178d. LINK
- Fisher, R. A. Statistical methods in genetics Heredity 1952, 6, 1–12 LINK
- Fisher, R. A. Statistical Methods for Research Workers; Oliver and Boyd: Edinburgh, 1925.
- Box, G. E. P.; Hunter, W. G.; Hunter, J. S. Statistics for Experimenters; Wiley: New York, 1978.
- Cox, D. R. The role of statistical significance tests Scand. J. Stat. 1977, 4, 49–70
- Neyman, J.; Pearson, E. S. On the use and interpretation of certain test criteria for purposes of statistical inference: part I Biometrika 1928, 175– 240 LINK
- Cohen, J. The earth is round (p < 0.05) Am. Psychol. 1994, 49, 997–1003 LINK
- Moran, J. L.; Solomon, P. J. A farewell to P-values Crit. Care Resuscitation: J. Australasian Acad. Crit. Care Med.e 2004, 6, 130 LINK
- Gigerenzer, G. Mindless statistics J. Socioeconomics 2004, 33, 587– 606 LINK
- Dallal, J. Why P=0.05?; 2012; LINK
- Altman, D. G. Practical Statistics for Medical Research; Chapman and Hall: London, UK, 1991.
- Fisher, R. A.; Yates, F. Statistical Tables for Biological, Agricultural, and Medical Research; Oliver and Boyd: London, UK, 1938.
- Yates, F. The Design and Analysis of Factorial Experiments; Technical Communication 35 of the Commonwealth Bureau of Soils; Commonwealth Agriculture Bureau: Farnham Royal, UK, 1937.
- Yates, F. The influence of statistical methods for research workers on the development of the science of statistics J. Am. Stat. Assoc. 1951, 46, 19– 34
- Brumfield, G. Language: disputed definitions Nature 2008, 455, 1023 (see comment). LINK
- Reese, R. A. Significant confusion in scientists’ grasp of statistics Nature 2008, 456, 315 LINK
- Salsburg, D. The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century; W. H. Freeman: New York, 2001.
- Ziliak, S. T.; McCloskey, D. N. The Cult of Statistical Significance: How the Standard Error Costs Us Jobs, Justice, and Lives; University of Michigan Press: Ann Arbor, MI, 2008. LINK
- Nester, M. Quotes criticizing significance testing, 1997. LINK
- Nester, M.; Anderson, D. R. A few notes regarding hypothesis testing, 1997. LINK
- Thompson, B. 402 citations questioning the indiscriminate use of null hypothesis significance tests in observational studies, 2001. LINK
- Hubbard, R.; Lindsay, R. M. Why P values are not a useful measure of evidence in statistical significance testing Theory Psychol. 2008, 18, 69– 88 LINK
- Ioannidis, J. P. Why most published research findings are false PLoS Med. 2005, 2, e124 LINK
- Chow, S. L. Précis of statistical significance: rationale, validity, and utility Behav. Brain Sci. 1998, 21, 169– 194 LINK
- Fidler, F. From Statistical Significance to Effect Estimation: Statistical Reform in Psychology, Medicine, and Ecology. University of Melbourne, Melbourne, Australia, 2005. LINK
- Senn, S. Two cheers for P-values? J. Epidemiol. Biostat. 2001, 6, 193– 204 LINK
- Carlin, J. B.; Doyle, L. W. Sample size. Continuing education: statistics for clinicians J. Paediatr. Child Health 2002, 38, 300– 304 LINK
- Turner, J. R. New Drug Development: Design, Methodology, and Analysis; Wiley: Hoboken, NJ, 2007. LINK
- Cohen, J. Statistical Power Analysis for the Behavioral Sciences, 2nd ed.; Lawrence Erlbaum Associates: Hillsdale, NJ, 1988.
- Lenth, R. V.Some practical guidelines for effective sample size determination Am. Stat. 2001, 55, 187– 193 LINK
- Pocock, S. J.Clinical trials with multiple outcomes: a statistical perspective on their design, analysis, and interpretation Control Clin. Trials 1997, 18, 530– 545 LINK
- Organisation for Economic Co-operation and Development. OECD Guidelines for the Testing of Chemicals, Section 4: Health Effects Test No. 443: Extended One-Generation Reproductive Toxicity Study; OECD Publishing: Paris, France, 2011.
- Organisation for Economic Co-operation and Development. OECD Guidelines for the Testing of Chemicals, Section 4: Health Effects Test No. 487: Genetic Toxicology: Rodent Dominant Lethal Test; OECD Publishing: Paris, France, 1984.
- Martin, M. J.; Judson, R. S.; Reif, D. M.; Kavlock, R. J.; Dix, D. J.Profiling chemicals based on chronic toxicity results from the U.S. EPA ToxRef Database Environ. Health Perspect. 2009, 117, 393– 399
- Cochran, W. G.; Cox, G. M. Experimental Designs, 2nd ed.; Wiley: New York, 1957.
- Altman, D. G. Statistics in medical journals Stat. Med. 1982, 1, 59– 71 LINK
- Gardner, M. J.; Altman, D. G. Confidence intervals rather than P values: estimation rather than hypothesis testing Br. Med. J. (Clin. Res. Ed.) 1986, 292, 746– 750 LINK
- Weller, A. C. Editorial Peer Review: Its Strengths and Weaknesses; Information Today: Medford, NJ, 2001.
- Cox, D. R.; Snell, E. J. Applied Statistics: Principles and Examples; Chapman and Hall: London, UK, 1981.
- Organisation for Economic Co-operation and Development. Safety Evaluation of Foods Derived by Modern Biotechnology: Concept and Principles; OECD Publishing: Paris, France, 1993. LINK
- Kuiper, H. A.; Kleter, G. A.; Noteborn, H. P. J. M.; Kok, E. J. Substantial equivalence – an appropriate paradigm for the safety assessment of genetically modified foods Toxicology 2002, 181–182, 427– 431 LINK
- U.S. Food and Drug Administration, Center for Drug Evaluation and Research. Guidance for Industry: Statistical Approaches to Establishing Bioequivalence, ( 2001). LINK
- Committee for Proprietary Medicinal Products. Points to consider on switching between superiority and non-inferiority. Br. J. Clin. Pharmacol. 2001, 52, 223– 228. LINK
- Altman, D. G.; Bland, J. M. Absence of evidence is not evidence of absence Br. Med. J. 1995, 311, 485 LINK
- Altman, D. G.; Bland, J. M. Confidence intervals illuminate absence of evidence Br. Med. J. 2004, 328, 1016– 1017 LINK
- Benjamini, Y.; Hochberg, Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing J. R. Stat. Soc. Ser. B: Stat. Soc. 1995, 57, 289– 300 LINK
- Payne, R. A Guide to Regression, Nonlinear and Generalized Linear Models in GenStat, 15th ed.; VSN International: Hemel Hempstead, UK, 2012. LINK
- Nelder, J. A. Discussion on the papers by Wynn and Bloomfield, and O’Neill and Wetherill J. R. Stat. Soc. Ser. B: Stat. Soc. 1971, 33, 244– 246
- Bryan-Jones, J.; Finney, D. J. On an error in ‘Instructions to authors’ Hortic. Sci. 1983, 18, 279– 281
- Little, T. M. If Galileo published in HortScience multiple range tests . Hortic. Sci. 1978, 13.
- Preece, D. A. The design and analysis of experiments: what has gone wrong Utilitas Math. A 1982, 21, 201– 244
- Perry, J. N. Multiple-comparison procedures: a dissenting view J. Econ. Entomol. 1986, 79, 1149– 1155 LINK
- Nelder, J. A.; Wedderburn, R. W. M. Generalized linear models J. R. Stat. Soc. Ser. A: Stat. Soc. 1972, 370– 384 LINK
- Box, G. E. P.; Draper, N. R. Empirical Model-Building and Response Surfaces; Wiley: New York, 1987.
- Tukey, J. W. Exploratory Data Analysis; Pearson: Reading, MA, 1977.
- Johnson, R. A.; Wichern, D. W. Applied Multivariate Statistical Analysis, 6th ed.; Prentice Hall: New York, 2007.
- Shewry, P. Natural variability in grain composition in wheat and related cereals. J. Agric. Food Chem. 2013, DOI: 10.1021/jf3043092. LINK
- Harrison, J. M.; Breeze, M. L.; Harrigan, G. G.Introduction to Bayesian statistical approaches to compositional analyses of transgenic crops 1. Model validation and setting the stage Regul. Toxicol. Pharmacol. 2011, 60, 381– 388 LINK
- U.S. Food and Drug Administration, Center for Devices and Radiological Health. Guidance for the Use of Bayesian Statistics in MedicalDevice Clinical Trials, ( 2010). LINK
- Kruschke, J. K. An open letter, 2010 LINK