site-logo

Biological Importance and Statistical Significance

Lovell DP
Journal of Agricultural and Food Chemistry
January 1, 2013

Download

Task Force #12

Journal of Agricultural and Food Chemistry. 2013;61(35):8340-8348

Abstract: Statistical ideas behind the analysis of experiments related to crop composition and the genetic factors underlying composition are discussed. The emphasis is on concepts rather than statistical formulations. Statistical analysis and biological considerations are shown to be complementary rather than contradictory, in that the statistical analysis of a data set depends on the experimental design, that no amount of statistical sophistication can rescue a badly designed study, and that good experimental design is crucial. The traditional null hypothesis significance testing approach has severe limitations, but p values and statistical significance still often seem to be the primary objective of an analysis. Emphasis instead should be on identifying the size of effects that are biologically important and, with the involvement of the “domain” scientist, using these to help design experiments with appropriate sample sizes and statistical power. The issues discussed here are also directly applicable to other areas of research.

To download this article, click here.

References

  1. Box, J. F. R. A. Fisher: The Life of a Scientist; Wiley: New York, 1978. LINK
  2. Edwards, A. W. R. A. Fisher. Twice professor of genetics: London and Cambridge or “a fairly well-known geneticist” Biometrics 1990, 46, 897– 904 LINK
  3. Smith, A. F. M. Mad cows and ecstasy: chance and choice in an evidence-based society J. R. Stat. Soc. Ser. A: Stat. Soc. 1996, 159, 367– 384 LINK
  4. EFSA Scientific Committee. Statistical significance and biological relevance. EFSA J. 2011, 9, 2372 (17 pp). LINK
  5. EFSA Panel on Genetically Modified Organisms (GMO). Guidance on the environmental risk assessment of genetically modified plants. EFSA J. 2010, 8, 1879 (111 pp). LINK
  6. EFSA Panel on Genetically Modified Organisms (GMO). Statistical considerations for the safety evaluation of GMOs. EFSA J. 2010, 8, 1250 (59 pp). LINK
  7. EFSA Scientific Committee. Guidance on conducting repeated-dose 90-day oral toxicity study in rodents on whole food/feed. EFSA J. 2011, 9, 2438 (21 pp). LINK
  8. Fisher, R. Presidential address to the First Indian Statistical Congress Sankhya 1938, 4, 14– 17
  9. Price, W. D.; Underhill, L. Application of laws, policies, and guidance from the United States and Canada to the regulation of food and feed derived from genetically modified crops: interpretation of composition data.. J. Agric. Food Chem. 2013, DOI: 10.1021/jf401178d. LINK 
  10. Fisher, R. A. Statistical methods in genetics Heredity 1952, 6, 1–12 LINK
  11. Fisher, R. A. Statistical Methods for Research Workers; Oliver and Boyd: Edinburgh, 1925.
  12. Box, G. E. P.; Hunter, W. G.; Hunter, J. S. Statistics for Experimenters; Wiley: New York, 1978.
  13. Cox, D. R. The role of statistical significance tests Scand. J. Stat. 1977, 4, 49–70
  14. Neyman, J.; Pearson, E. S. On the use and interpretation of certain test criteria for purposes of statistical inference: part I Biometrika 1928, 175– 240 LINK
  15. Cohen, J. The earth is round (p < 0.05) Am. Psychol. 1994, 49, 997–1003 LINK
  16. Moran, J. L.; Solomon, P. J. A farewell to P-values Crit. Care Resuscitation: J. Australasian Acad. Crit. Care Med.e 2004, 6, 130 LINK
  17. Gigerenzer, G. Mindless statistics J. Socioeconomics 2004, 33, 587– 606 LINK
  18. Dallal, J. Why P=0.05?; 2012; LINK
  19. Altman, D. G. Practical Statistics for Medical Research; Chapman and Hall: London, UK, 1991.
  20. Fisher, R. A.; Yates, F. Statistical Tables for Biological, Agricultural, and Medical Research; Oliver and Boyd: London, UK, 1938.
  21. Yates, F. The Design and Analysis of Factorial Experiments; Technical Communication 35 of the Commonwealth Bureau of Soils; Commonwealth Agriculture Bureau: Farnham Royal, UK, 1937.
  22. Yates, F. The influence of statistical methods for research workers on the development of the science of statistics J. Am. Stat. Assoc. 1951, 46, 19– 34
  23. Brumfield, G. Language: disputed definitions Nature 2008, 455, 1023 (see comment). LINK
  24. Reese, R. A. Significant confusion in scientists’ grasp of statistics Nature 2008, 456, 315 LINK
  25. Salsburg, D. The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century; W. H. Freeman: New York, 2001.
  26. Ziliak, S. T.; McCloskey, D. N. The Cult of Statistical Significance: How the Standard Error Costs Us Jobs, Justice, and Lives; University of Michigan Press: Ann Arbor, MI, 2008. LINK
  27. Nester, M. Quotes criticizing significance testing, 1997. LINK
  28. Nester, M.; Anderson, D. R. A few notes regarding hypothesis testing, 1997. LINK
  29. Thompson, B. 402 citations questioning the indiscriminate use of null hypothesis significance tests in observational studies, 2001. LINK
  30. Hubbard, R.; Lindsay, R. M. Why P values are not a useful measure of evidence in statistical significance testing Theory Psychol. 2008, 18, 69– 88 LINK
  31. Ioannidis, J. P. Why most published research findings are false PLoS Med. 2005, 2, e124 LINK
  32. Chow, S. L. Précis of statistical significance: rationale, validity, and utility Behav. Brain Sci. 1998, 21, 169– 194 LINK
  33. Fidler, F. From Statistical Significance to Effect Estimation: Statistical Reform in Psychology, Medicine, and Ecology. University of Melbourne, Melbourne, Australia, 2005. LINK
  34. Senn, S. Two cheers for P-values? J. Epidemiol. Biostat. 2001, 6, 193– 204 LINK
  35. Carlin, J. B.; Doyle, L. W. Sample size. Continuing education: statistics for clinicians J. Paediatr. Child Health 2002, 38, 300– 304 LINK
  36. Turner, J. R. New Drug Development: Design, Methodology, and Analysis; Wiley: Hoboken, NJ, 2007. LINK
  37. Cohen, J. Statistical Power Analysis for the Behavioral Sciences, 2nd ed.; Lawrence Erlbaum Associates: Hillsdale, NJ, 1988.
  38. Lenth, R. V.Some practical guidelines for effective sample size determination Am. Stat. 2001, 55, 187– 193 LINK
  39. Pocock, S. J.Clinical trials with multiple outcomes: a statistical perspective on their design, analysis, and interpretation Control Clin. Trials 1997, 18, 530– 545 LINK
  40. Organisation for Economic Co-operation and Development. OECD Guidelines for the Testing of Chemicals, Section 4: Health Effects Test No. 443: Extended One-Generation Reproductive Toxicity Study; OECD Publishing: Paris, France, 2011.
  41. Organisation for Economic Co-operation and Development. OECD Guidelines for the Testing of Chemicals, Section 4: Health Effects Test No. 487: Genetic Toxicology: Rodent Dominant Lethal Test; OECD Publishing: Paris, France, 1984.
  42. Martin, M. J.; Judson, R. S.; Reif, D. M.; Kavlock, R. J.; Dix, D. J.Profiling chemicals based on chronic toxicity results from the U.S. EPA ToxRef Database Environ. Health Perspect. 2009, 117, 393– 399
  43. Cochran, W. G.; Cox, G. M. Experimental Designs, 2nd ed.; Wiley: New York, 1957.
  44. Altman, D. G. Statistics in medical journals Stat. Med. 1982, 1, 59– 71 LINK
  45. Gardner, M. J.; Altman, D. G. Confidence intervals rather than P values: estimation rather than hypothesis testing Br. Med. J. (Clin. Res. Ed.) 1986, 292, 746– 750 LINK
  46. Weller, A. C. Editorial Peer Review: Its Strengths and Weaknesses; Information Today: Medford, NJ, 2001.
  47. Cox, D. R.; Snell, E. J. Applied Statistics: Principles and Examples; Chapman and Hall: London, UK, 1981.
  48. Organisation for Economic Co-operation and Development. Safety Evaluation of Foods Derived by Modern Biotechnology: Concept and Principles; OECD Publishing: Paris, France, 1993. LINK
  49. Kuiper, H. A.; Kleter, G. A.; Noteborn, H. P. J. M.; Kok, E. J. Substantial equivalence – an appropriate paradigm for the safety assessment of genetically modified foods Toxicology 2002, 181–182, 427– 431 LINK
  50. U.S. Food and Drug Administration, Center for Drug Evaluation and Research. Guidance for Industry: Statistical Approaches to Establishing Bioequivalence, ( 2001). LINK
  51. Committee for Proprietary Medicinal Products. Points to consider on switching between superiority and non-inferiority. Br. J. Clin. Pharmacol. 2001, 52, 223– 228. LINK
  52. Altman, D. G.; Bland, J. M. Absence of evidence is not evidence of absence Br. Med. J. 1995, 311, 485 LINK
  53. Altman, D. G.; Bland, J. M. Confidence intervals illuminate absence of evidence Br. Med. J. 2004, 328, 1016– 1017 LINK
  54. Benjamini, Y.; Hochberg, Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing J. R. Stat. Soc. Ser. B: Stat. Soc. 1995, 57, 289– 300 LINK
  55. Payne, R. A Guide to Regression, Nonlinear and Generalized Linear Models in GenStat, 15th ed.; VSN International: Hemel Hempstead, UK, 2012. LINK
  56. Nelder, J. A. Discussion on the papers by Wynn and Bloomfield, and O’Neill and Wetherill J. R. Stat. Soc. Ser. B: Stat. Soc. 1971, 33, 244– 246
  57. Bryan-Jones, J.; Finney, D. J. On an error in ‘Instructions to authors’ Hortic. Sci. 1983, 18, 279– 281
  58. Little, T. M. If Galileo published in HortScience multiple range tests . Hortic. Sci. 1978, 13.
  59. Preece, D. A. The design and analysis of experiments: what has gone wrong Utilitas Math. A 1982, 21, 201– 244
  60. Perry, J. N. Multiple-comparison procedures: a dissenting view J. Econ. Entomol. 1986, 79, 1149– 1155 LINK
  61. Nelder, J. A.; Wedderburn, R. W. M. Generalized linear models J. R. Stat. Soc. Ser. A: Stat. Soc. 1972, 370– 384 LINK
  62. Box, G. E. P.; Draper, N. R. Empirical Model-Building and Response Surfaces; Wiley: New York, 1987.
  63. Tukey, J. W. Exploratory Data Analysis; Pearson: Reading, MA, 1977.
  64. Johnson, R. A.; Wichern, D. W. Applied Multivariate Statistical Analysis, 6th ed.; Prentice Hall: New York, 2007.
  65. Shewry, P. Natural variability in grain composition in wheat and related cereals. J. Agric. Food Chem. 2013, DOI: 10.1021/jf3043092. LINK
  66. Harrison, J. M.; Breeze, M. L.; Harrigan, G. G.Introduction to Bayesian statistical approaches to compositional analyses of transgenic crops 1. Model validation and setting the stage Regul. Toxicol. Pharmacol. 2011, 60, 381– 388 LINK
  67. U.S. Food and Drug Administration, Center for Devices and Radiological Health. Guidance for the Use of Bayesian Statistics in MedicalDevice Clinical Trials, ( 2010). LINK
  68. Kruschke, J. K. An open letter, 2010 LINK