Monday, March 17, 2008

Problems with Quantitative Research

“a research finding is less likely to be true when the studies conducted in a field are smaller; when effect sizes are smaller; when there is a greater number and lesser preselection of tested relationships; where there is greater flexibility in designs, definitions, outcomes, and analytical modes; when there is greater financial and other interest and prejudice; and when more teams are involved in a scientific field in chase of statistical significance.”

John P. A. Ioannidis
“Why Most Published Research Findings Are False”
Public Library of Science (PLOS) Medical Journal, Vol 2, #8, August 2005
medicine.plosjournals.org/perlserv/?request=get-document&doi=10.1371%2Fjournal.pmed.0020124

In summary, problems can arise with:

- Small sample size
- No other studies/stats to build on
- Different methods of gathering stats
- Prejudice on part of researchers/funding
- Competition from other researchers

No comments: