Don’t believe the latest study you read in the headlines, chances are, it could be wrong, according to a new report by the National Association of Scholars that delves into what it calls the “use and abuse of statistics in the sciences.”
The report broke down the issue of irreproducibility, or the problem that a lot of scientific research cannot be reproduced. The report took aim at unverifiable climate science, but also critiqued medical studies, behavioral research and other fields.
The 72-page report took the matter a step further in calling the issue a politicization of science.
“Not all irreproducible research is progressive advocacy; not all progressive advocacy is irreproducible; but the intersection between the two is very large. The intersection between the two is a map of much that is wrong with modern science,” the report states.
Co-authored by David Randall and Christopher Wesler, “The Irreproducibility Crisis of Modern Science: Causes, Consequences, and the Road to Reform” focused on the irreproducibility of recent scientific studies.
It references a study performed by researchers at Amgen in 2012. For this study, researchers tried to reproduce the results of “53 landmark studies in oncology and hematology.” Researchers were only able to replicate the results of six studies.
“People have found similar results in psychology and economics. Different fields are affected different amounts,” Randall told The College Fix. “As a rule of thumb, fields that use statistics intensively are more likely to have troubles than fields that don’t.”
NAS Director of Research David Randall speaks about the role that the federal government can play in the #reproducibilitycrisis – only granting funding for preregistered experiments/procedures is one example. #openscience #scientificresearch #transparency pic.twitter.com/ztpa8uaf18
— NAS Scholars (@NASorg) April 17, 2018
The report hypothesized that there are a number of different reasons for irreproducibility that include such things as “flawed statistics, faulty data, deliberate exclusion of data, and political groupthink,” among other reasons. “Actual fraud on the part of researchers appears to be a growing problem,” the report also states.
“‘Stereotype threat’ as an explanation for poor academic performance? Didn’t reproduce. ‘Social priming,’ which argues that unnoticed stimuli can significantly change behavior? Didn’t reproduce that well … Tests of implicit bias as predictors of discriminatory behavior? The methodology turned out to be dubious, and the test of implicit bias may have been biased itself,” the report states.
The report also alludes multiple times to the notion that climate science is on shaky ground.
“Climate science has significant work to do to make its data and its statistical procedures properly reproducible,” Randall said.
Randall cited Judith Curry, a world-renowned climatologist, who has warned that the climate science field is heavily affected by groupthink, a collective way of thinking that has been known to stop individuals from questioning widely accepted theories.
Randall said he believes that climate change data needs to be reproducible because it is “more than usually intrusive into the lives of Americans.”
#Irreproducibility co-author Christopher Welser lists a number of examples of irreproducible research & results that were found later to be false that are mentioned in our report. https://t.co/0qllk7FST4 #openscience #science #transparencyinscience pic.twitter.com/BwzlTdAny8
— NAS Scholars (@NASorg) April 17, 2018
To provide the public with accurate statistical information, the report endorses the expansion of the Secret Science Reform Act of 2015 to cut down on irreproducible data used to back public policy.
When asked what the average person could do in order to make sure that the information that is backing public policy is credible, Randall recommended: “Always ask ‘has this study been reproduced? Did this study have pre-registered research protocols? Does it support an unpopular belief?’ If the answer to any of these is no, suspend judgment. Don’t disbelieve blindly, but don’t believe blindly either.”
MORE: Fed up with the BS, prominent climate change skeptic professor retires
MORE: Climate scientists rebut oft-repeated 2016 ‘warmest year on record’ claim
IMAGE: Shutterstock
Like The College Fix on Facebook / Follow us on Twitter
Please join the conversation about our stories on Facebook, Twitter, Instagram, Reddit, MeWe, Rumble, Gab, Minds and Gettr.