J Scott Armstrong: Fewer Than 1 Percent Of Papers in Scientific Journals Follow Scientific Method

[jmyrlefuller]

http://www.breitbart.com/tech/2017/03/29/j-scott-armstrong-fraction-1-papers-scientific-journals-follow-scientific-method/

by Allum Bokhari
March 29, 2017

Fewer than 1 percent of papers published in scientific journals follow the scientific method, according to research by Wharton School professor and forecasting expert J. Scott Armstrong.

Professor Armstrong, who co-founded the peer-reviewed Journal of Forecasting in 1982 and the International Journal of Forecasting in 1985, made the claim in a presentation about what he considers to be “alarmism” from forecasters over man-made climate change.

(excerpt)

[Smokin Joe]

Interesting. Not unexpected except that the number seems even lower than I had suspected overall.

[Suppressed]

First of all, the mental midgets at Breitbart are either intentionally or unintentionally misrepresenting the quote, but at least they provide the original quote for people to see.  J. Scott Armstrong wrote of papers "in [his] field".

Also, "The Scientific Method" as taught in primary schools is a broad term that isn't encompassing of the full methods that scientists employ.  For example, geologists quite standardly employ T. C. Chamberlin's "Method of Multiple Working Hypotheses" (see http://www.gly.uga.edu/railsback/railsback_chamberlin.html for a quick overview.  [Chamberlin's paper is too long, too high-blown, and too sexist for modern students . . .]  ^-^)

A Quaternary geologist doesn't have the luxury of calling down another ice sheet to experiment.  An astronomer can't generate a supernova to make an observation.  Historical science is still science. 

But Armstrong's criteria must be cautiously applied to understand that.


His point might be valid, but Breitbart doesn't do a good job with this story, IMO.

[Smokin Joe]

Also, "The Scientific Method" as taught in primary schools is a broad term that isn't encompassing of the full methods that scientists employ. 

No, but for many fields, those methods and experimental repeatability are within grasp.

For example, geologists quite standardly employ T. C. Chamberlin's "Method of Multiple Working Hypotheses" (see http://www.gly.uga.edu/railsback/railsback_chamberlin.html for a quick overview.  [Chamberlin's paper is too long, too high-blown, and too sexist for modern students . . .]

We do have a different approach. We have to. We can't put the plates back and have a tectonic do-over. It can be modeled, but the real thing won't fit in the lab. We brainstorm, come up with as many possible ways to emplace sediments, ore bodies, petroleum reserves, or create geologic structures and landforms and then eliminate the ones which don't fit. Sometimes, there isn't enough data to eliminate them all and there is more than one possibility left over (the multiple hypotheses), and sometimes you run out of ideas before you run out of disqualifications (in which case, it's time to revisit the evidence and do some more thinking). It is fun, challenging, and it drives the engineers nuts. 

A Quaternary geologist doesn't have the luxury of calling down another ice sheet to experiment. 

No, but you can go where there are ice sheets and study the processes in operation there. The assumption is that the same laws of physics, fluid dynamics, etc. prevail regardless of when or where it is, and that certain sedimentary deposits which form in specific environments today formed in very similar environments eons ago, in the same flow regimes, subject to the same physical properties and laws observed today.

An astronomer can't generate a supernova to make an observation.

Same principle, find one where the light is just coming in and study it, again in the belief that the laws of physics don't change.

Historical science is still science. 

But Armstrong's criteria must be cautiously applied to understand that.


His point might be valid, but Breitbart doesn't do a good job with this story, IMO.