Reproducibility - Revision history

imported>AnomieBOT: Substing templates: {{Format ISBN}}. See User:AnomieBOT/docs/TemplateSubster for info.

2025-10-04T19:13:51Z

Substing templates: {{Format ISBN}}. See User:AnomieBOT/docs/TemplateSubster for info.

imported>OAbot: Open access bot: url-access=subscription updated in citation with #oabot.

2025-06-24T01:23:25Z

Open access bot: url-access=subscription updated in citation with #oabot.

← Previous revision		Revision as of 01:23, 24 June 2025
Line 2:		Line 2:
	{{About\|the reproducibility of scientific research results\|reproductive capacity of organisms\|fertility\|and\|fecundity\|reproducibility in the context of computer software\|Reproducible builds}}		{{About\|the reproducibility of scientific research results\|reproductive capacity of organisms\|fertility\|and\|fecundity\|reproducibility in the context of computer software\|Reproducible builds}}

	'''Reproducibility''', closely related to '''replicability''' and '''repeatability''', is a major principle underpinning the [[scientific method]]. For the findings of a study to be reproducible means that results obtained by an [[experiment]] or an [[observational study]] or in a [[statistical analysis]] of a [[data set]] should be achieved again with a high degree of reliability when the study is replicated. There are different kinds of replication<ref>{{Cite journal\|last1=Tsang\|first1=Eric W. K.\|last2=Kwan\|first2=Kai-man\|date=1999\|title=Replication and Theory Development in Organizational Science: A Critical Realist Perspective\|url=http://dx.doi.org/10.5465/amr.1999.2553252\|journal=Academy of Management Review\|volume=24\|issue=4\|pages=759–780\|doi=10.5465/amr.1999.2553252\|issn=0363-7425}}</ref> but typically replication studies involve different researchers using the same methodology. Only after one or several such successful replications should a result be recognized as scientific knowledge.		'''Reproducibility''', closely related to '''replicability''' and '''repeatability''', is a major principle underpinning the [[scientific method]]. For the findings of a study to be reproducible means that results obtained by an [[experiment]] or an [[observational study]] or in a [[statistical analysis]] of a [[data set]] should be achieved again with a high degree of reliability when the study is replicated. There are different kinds of replication<ref>{{Cite journal\|last1=Tsang\|first1=Eric W. K.\|last2=Kwan\|first2=Kai-man\|date=1999\|title=Replication and Theory Development in Organizational Science: A Critical Realist Perspective\|url=http://dx.doi.org/10.5465/amr.1999.2553252\|journal=Academy of Management Review\|volume=24\|issue=4\|pages=759–780\|doi=10.5465/amr.1999.2553252\|issn=0363-7425\|url-access=subscription}}</ref> but typically replication studies involve different researchers using the same methodology. Only after one or several such successful replications should a result be recognized as scientific knowledge.

	==History==		==History==
Line 20:		Line 20:

	==Measures of reproducibility and repeatability==		==Measures of reproducibility and repeatability==
	In chemistry, the terms reproducibility and repeatability are used with a specific quantitative meaning.<ref>{{Cite journal \|last= \|first= \|title=IUPAC - reproducibility (R05305) \|url=https://goldbook.iupac.org/terms/view/R05305 \|access-date=2022-03-04 \|website=[[International Union of Pure and Applied Chemistry]]\|doi= 10.1351/goldbook.R05305\|doi-access=free}}</ref> In inter-laboratory experiments, a concentration or other quantity of a chemical substance is measured repeatedly in different laboratories to assess the variability of the measurements. Then, the standard deviation of the difference between two values obtained within the same laboratory is called repeatability. The standard deviation for the difference between two measurement from different laboratories is called ''reproducibility''.<ref name="ASTM E177">{{cite web\|url=https://www.astm.org/Standards/E177.htm \|title=Standard Practice for Use of the Terms Precision and Bias in ASTM Test Methods \|year=2014 \|author=Subcommittee E11.20 on Test Method Evaluation and Quality Control \|publisher=ASTM International \|id=ASTM E177}}{{Subscription required}}</ref>		In chemistry, the terms reproducibility and repeatability are used with a specific quantitative meaning.<ref>{{Cite journal \|last= \|first= \|title=IUPAC - reproducibility (R05305) \|url=https://goldbook.iupac.org/terms/view/R05305 \|access-date=2022-03-04 \|website=[[International Union of Pure and Applied Chemistry]]\|doi= 10.1351/goldbook.R05305\|doi-access=free\|url-access=subscription}}</ref> In inter-laboratory experiments, a concentration or other quantity of a chemical substance is measured repeatedly in different laboratories to assess the variability of the measurements. Then, the standard deviation of the difference between two values obtained within the same laboratory is called repeatability. The standard deviation for the difference between two measurement from different laboratories is called ''reproducibility''.<ref name="ASTM E177">{{cite web\|url=https://www.astm.org/Standards/E177.htm \|title=Standard Practice for Use of the Terms Precision and Bias in ASTM Test Methods \|year=2014 \|author=Subcommittee E11.20 on Test Method Evaluation and Quality Control \|publisher=ASTM International \|id=ASTM E177}}{{Subscription required}}</ref>
	These measures are related to the more general concept of [[variance component]]s in [[metrology]].		These measures are related to the more general concept of [[variance component]]s in [[metrology]].

imported>Natureium: Unsourced fragmented list removed

2025-06-08T23:08:04Z

Unsourced fragmented list removed

← Previous revision		Revision as of 23:08, 8 June 2025
Line 3:		Line 3:

	'''Reproducibility''', closely related to '''replicability''' and '''repeatability''', is a major principle underpinning the [[scientific method]]. For the findings of a study to be reproducible means that results obtained by an [[experiment]] or an [[observational study]] or in a [[statistical analysis]] of a [[data set]] should be achieved again with a high degree of reliability when the study is replicated. There are different kinds of replication<ref>{{Cite journal\|last1=Tsang\|first1=Eric W. K.\|last2=Kwan\|first2=Kai-man\|date=1999\|title=Replication and Theory Development in Organizational Science: A Critical Realist Perspective\|url=http://dx.doi.org/10.5465/amr.1999.2553252\|journal=Academy of Management Review\|volume=24\|issue=4\|pages=759–780\|doi=10.5465/amr.1999.2553252\|issn=0363-7425}}</ref> but typically replication studies involve different researchers using the same methodology. Only after one or several such successful replications should a result be recognized as scientific knowledge.		'''Reproducibility''', closely related to '''replicability''' and '''repeatability''', is a major principle underpinning the [[scientific method]]. For the findings of a study to be reproducible means that results obtained by an [[experiment]] or an [[observational study]] or in a [[statistical analysis]] of a [[data set]] should be achieved again with a high degree of reliability when the study is replicated. There are different kinds of replication<ref>{{Cite journal\|last1=Tsang\|first1=Eric W. K.\|last2=Kwan\|first2=Kai-man\|date=1999\|title=Replication and Theory Development in Organizational Science: A Critical Realist Perspective\|url=http://dx.doi.org/10.5465/amr.1999.2553252\|journal=Academy of Management Review\|volume=24\|issue=4\|pages=759–780\|doi=10.5465/amr.1999.2553252\|issn=0363-7425}}</ref> but typically replication studies involve different researchers using the same methodology. Only after one or several such successful replications should a result be recognized as scientific knowledge.

	~~== Types of reproducibility ==~~
	~~There are different kinds of replication studies, each serving a unique role in scientific validation:~~

	~~Direct replication – the exact experiment or study is repeated under the same conditions to verify the original findings.~~

	~~Conceptual replication – a study tests the same hypothesis but uses a different methodology, materials, or population to see if the results hold in different contexts.~~

	Computational reproducibility – in data science and computational research, reproducibility requires making all datasets, code, and algorithms openly available so others can replicate the analysis and obtain the same results.

	~~== Importance of reproducibility ==~~
	~~Reproducibility serves several critical purposes in science:~~

	~~Verification of results – confirms that findings are not due to random chance or errors.~~

	~~Building trust in research – scientists, policymakers, and the public rely on reproducible studies to make informed decisions.~~

	~~Advancing knowledge – establishes a strong foundation for future research by validating existing theories.~~

	~~Avoiding bias and fraud – helps detect false positives, publication bias, and data manipulation that could mislead the scientific community.~~

	~~Many studies fail reproducibility tests, leading to what is known as the replication crisis in fields like psychology, medicine, and social sciences.~~

	~~Some key challenges include:~~

	~~Insufficient data sharing – many researchers do not make raw data, code, or methodology openly available, making replication difficult.~~

	~~Small sample sizes – studies with limited sample sizes may show results that do not generalize to larger populations.~~

	~~Publication bias – journals tend to publish positive findings rather than null or negative results, leading to an incomplete scientific record.~~

	~~Complex experimental conditions – in some cases, small variations in laboratory settings, equipment, or researcher expertise can affect outcomes, making exact replication difficult.~~

	~~== Real-world applications of reproducibility ==~~
	~~Medical research – reproducibility ensures that clinical trials and drug effectiveness studies produce reliable results before treatments reach the public.~~

	~~AI and machine learning – scientists emphasize reproducibility in AI by requiring open-source models and datasets to validate algorithm performance.~~

	~~Climate science – climate models must be reproducible across different datasets and simulations to ensure accurate predictions of global warming.~~

	~~Pharmaceutical development – drug discovery relies on reproducing experiments across multiple labs to ensure safety and efficacy.~~

	~~== Improving reproducibility in science ==~~
	~~To enhance reproducibility, researchers and institutions can adopt several best practices:~~

	~~Open data and code – making datasets and computational methods publicly available ensures that others can verify results.~~

	~~Registered reports – some scientific journals now accept studies based on pre-registered research plans, reducing bias.~~

	~~Standardized methods – using well-documented, standardized experimental protocols helps ensure consistent results.~~

	~~Independent replication studies – funding agencies and journals should prioritize replication studies to strengthen scientific integrity.~~

	With a narrower scope, ''reproducibility'' has been defined in [[computational science]]s as having the following quality: the results should be documented by making all data and code available in such a way that the computations can be executed again with identical results.

	~~In recent decades, there has been a rising concern that many published scientific results fail the test of reproducibility, evoking a reproducibility or [[replication crisis]].~~

	==History==		==History==

imported>Nyq at 11:44, 28 May 2025

2025-05-28T11:44:26Z

Show changes

← Previous revision		Revision as of 19:13, 4 October 2025
Line 2:		Line 2:
	{{About\|the reproducibility of scientific research results\|reproductive capacity of organisms\|fertility\|and\|fecundity\|reproducibility in the context of computer software\|Reproducible builds}}		{{About\|the reproducibility of scientific research results\|reproductive capacity of organisms\|fertility\|and\|fecundity\|reproducibility in the context of computer software\|Reproducible builds}}

	'''Reproducibility''', closely related to '''replicability''' and '''repeatability''', is a major principle underpinning the [[scientific method]]. For the findings of a study to be reproducible means that results obtained by an [[experiment]] or an [[observational study]] or in a [[statistical analysis]] of a [[data set]] should be achieved again with a high degree of reliability when the study is replicated. There are different kinds of replication<ref>{{Cite journal\|last1=Tsang\|first1=Eric W. K.\|last2=Kwan\|first2=Kai-man\|date=1999\|title=Replication and Theory Development in Organizational Science: A Critical Realist Perspective~~\|url=http://dx.doi.org/10.5465/amr.1999.2553252~~\|journal=Academy of Management Review\|volume=24\|issue=4\|pages=759–780\|doi=10.5465/amr.1999.2553252\|issn=0363-7425~~\|url-access=subscription~~}}</ref> but typically replication studies involve different researchers using the same methodology. Only after one or several such successful replications should a result be recognized as scientific knowledge.		'''Reproducibility''', closely related to '''replicability''' and '''repeatability''', is a major principle underpinning the [[scientific method]]. For the findings of a study to be reproducible means that results obtained by an [[experiment]] or an [[observational study]] or in a [[statistical analysis]] of a [[data set]] should be achieved again with a high degree of reliability when the study is replicated. There are different kinds of replication<ref>{{Cite journal\|last1=Tsang\|first1=Eric W. K.\|last2=Kwan\|first2=Kai-man\|date=1999\|title=Replication and Theory Development in Organizational Science: A Critical Realist Perspective\|journal=Academy of Management Review\|volume=24\|issue=4\|pages=759–780\|doi=10.5465/amr.1999.2553252\|issn=0363-7425}}</ref> but typically replication studies involve different researchers using the same methodology. Only after one or several such successful replications should a result be recognized as scientific knowledge.

	==History==		==History==
Line 32:		Line 32:
	To make any research project computationally reproducible, general practice involves all data and files being clearly separated, labelled, and documented. All operations should be fully documented and automated as much as practicable, avoiding manual intervention where feasible. The workflow should be designed as a sequence of smaller steps that are combined so that the intermediate outputs from one step directly feed as inputs into the next step. Version control should be used as it lets the history of the project be easily reviewed and allows for the documenting and tracking of changes in a transparent manner.		To make any research project computationally reproducible, general practice involves all data and files being clearly separated, labelled, and documented. All operations should be fully documented and automated as much as practicable, avoiding manual intervention where feasible. The workflow should be designed as a sequence of smaller steps that are combined so that the intermediate outputs from one step directly feed as inputs into the next step. Version control should be used as it lets the history of the project be easily reviewed and allows for the documenting and tracking of changes in a transparent manner.

	A basic workflow for reproducible research involves data acquisition, data processing and data analysis. Data acquisition primarily consists of obtaining primary data from a primary source such as surveys, field observations, experimental research, or obtaining data from an existing source. Data processing involves the processing and review of the raw data collected in the first stage, and includes data entry, data manipulation and filtering and may be done using software. The data should be digitized and prepared for data analysis. Data may be analysed with the use of software to interpret or visualise statistics or data to produce the desired results of the research such as quantitative results including figures and tables. The use of software and automation enhances the reproducibility of research methods.<ref>{{cite book \|last1=Kitzes \|first1=Justin \|last2=Turek \|first2=Daniel \|last3=Deniz \|first3=Fatma \|title=The practice of reproducible research case studies and lessons from the data-intensive sciences \|date=2018 \|publisher=University of California Press \|location=Oakland, California \|isbn=~~9780520294745~~ \|pages=19–30 \|jstor=~~10.1525/j.ctv1wxsc7 \|url=http://www.jstor.org/stable/~~10.1525/j.ctv1wxsc7}}</ref>		A basic workflow for reproducible research involves data acquisition, data processing and data analysis. Data acquisition primarily consists of obtaining primary data from a primary source such as surveys, field observations, experimental research, or obtaining data from an existing source. Data processing involves the processing and review of the raw data collected in the first stage, and includes data entry, data manipulation and filtering and may be done using software. The data should be digitized and prepared for data analysis. Data may be analysed with the use of software to interpret or visualise statistics or data to produce the desired results of the research such as quantitative results including figures and tables. The use of software and automation enhances the reproducibility of research methods.<ref>{{cite book \|last1=Kitzes \|first1=Justin \|last2=Turek \|first2=Daniel \|last3=Deniz \|first3=Fatma \|title=The practice of reproducible research case studies and lessons from the data-intensive sciences \|date=2018 \|publisher=University of California Press \|location=Oakland, California \|isbn=978-0-520-29474-5 \|pages=19–30 \|jstor=10.1525/j.ctv1wxsc7 }}</ref>

	There are systems that facilitate such documentation, like the [[R (programming language)\|R]] [[Markdown]] language<ref>{{cite journal\|last1=Marwick\|first1=Ben\|last2=Boettiger\|first2=Carl\|last3=Mullen\|first3=Lincoln\|title=Packaging data analytical work reproducibly using R (and friends)\|journal=The American Statistician\|volume=72\|date=29 September 2017\|pages=80–88\|doi=10.1080/00031305.2017.1375986\|s2cid=125412832\|url=http://ro.uow.edu.au/cgi/viewcontent.cgi?article=6445&context=smhpapers}}</ref>		There are systems that facilitate such documentation, like the [[R (programming language)\|R]] [[Markdown]] language<ref>{{cite journal\|last1=Marwick\|first1=Ben\|last2=Boettiger\|first2=Carl\|last3=Mullen\|first3=Lincoln\|title=Packaging data analytical work reproducibly using R (and friends)\|journal=The American Statistician\|volume=72\|date=29 September 2017\|pages=80–88\|doi=10.1080/00031305.2017.1375986\|s2cid=125412832\|url=http://ro.uow.edu.au/cgi/viewcontent.cgi?article=6445&context=smhpapers}}</ref>
Line 41:		Line 41:

	===Reproducible research in practice===		===Reproducible research in practice===
	Psychology has seen a renewal of internal concerns about irreproducible results (see the entry on [[replicability crisis]] for empirical results on success rates of replications). Researchers showed in a 2006 study that, of 141 authors of a publication from the American Psychological Association (APA) empirical articles, 103 (73%) did not respond with their data over a six-month period.<ref>{{Cite journal\|last1=Wicherts \|first1=J. M. \|last2=Borsboom \|first2=D. \|last3=Kats \|first3=J. \|last4=Molenaar \|first4=D. \|title=The poor availability of psychological research data for reanalysis \|doi=10.1037/0003-066X.61.7.726 \|journal=American Psychologist \|volume=61 \|issue=7 \|pages=726–728 \|year=2006 \|pmid=17032082}}</ref> In a follow-up study published in 2015, it was found that 246 out of 394 contacted authors of papers in APA journals did not share their data upon request (62%).<ref>{{Cite journal\|last1=Vanpaemel \|first1=W. \|last2=Vermorgen \|first2=M. \|last3=Deriemaecker \|first3=L. \|last4=Storms \|first4=G. \|title=Are we wasting a good crisis? The availability of psychological research data after the storm \|doi=10.1525/collabra.13 \|journal=Collabra \|volume=1 \|issue=1 \|pages=1–5 \|year=2015 \|doi-access=free}}</ref> In a 2012 paper, it was suggested that researchers should publish data along with their works, and a dataset was released alongside as a demonstration.<ref>{{Cite journal\|last1=Wicherts \|first1=J. M. \|last2=Bakker \|first2=M. \|doi=10.1016/j.intell.2012.01.004 \|title=Publish (your data) or (let the data) perish! Why not publish your data too? \|journal=Intelligence \|volume=40 \|issue=2 \|pages=73–76 \|year=2012}}</ref> In 2017, an article published in ''[[Scientific Data (journal)\|Scientific Data]]'' suggested that this may not be sufficient and that the whole analysis context should be disclosed.<ref>{{cite journal\|last1=Pasquier\|first1=Thomas\|last2=Lau\|first2=Matthew K.\|last3=Trisovic\|first3=Ana\|last4=Boose\|first4=Emery R.\|last5=Couturier\|first5=Ben\|last6=Crosas\|first6=Mercè\|last7=Ellison\|first7=Aaron M.\|last8=Gibson\|first8=Valerie\|last9=Jones\|first9=Chris R.\|last10=Seltzer\|first10=Margo\|title=If these data could talk\|journal=Scientific Data\|date=5 September 2017\|volume=4\|issue=1 \|~~pages~~=170114\|doi=10.1038/sdata.2017.114\|pmid=28872630\|pmc=5584398\|bibcode=2017NatSD...470114P}}</ref>		Psychology has seen a renewal of internal concerns about irreproducible results (see the entry on [[replicability crisis]] for empirical results on success rates of replications). Researchers showed in a 2006 study that, of 141 authors of a publication from the American Psychological Association (APA) empirical articles, 103 (73%) did not respond with their data over a six-month period.<ref>{{Cite journal\|last1=Wicherts \|first1=J. M. \|last2=Borsboom \|first2=D. \|last3=Kats \|first3=J. \|last4=Molenaar \|first4=D. \|title=The poor availability of psychological research data for reanalysis \|doi=10.1037/0003-066X.61.7.726 \|journal=American Psychologist \|volume=61 \|issue=7 \|pages=726–728 \|year=2006 \|pmid=17032082}}</ref> In a follow-up study published in 2015, it was found that 246 out of 394 contacted authors of papers in APA journals did not share their data upon request (62%).<ref>{{Cite journal\|last1=Vanpaemel \|first1=W. \|last2=Vermorgen \|first2=M. \|last3=Deriemaecker \|first3=L. \|last4=Storms \|first4=G. \|title=Are we wasting a good crisis? The availability of psychological research data after the storm \|doi=10.1525/collabra.13 \|journal=Collabra \|volume=1 \|issue=1 \|pages=1–5 \|year=2015 \|doi-access=free}}</ref> In a 2012 paper, it was suggested that researchers should publish data along with their works, and a dataset was released alongside as a demonstration.<ref>{{Cite journal\|last1=Wicherts \|first1=J. M. \|last2=Bakker \|first2=M. \|doi=10.1016/j.intell.2012.01.004 \|title=Publish (your data) or (let the data) perish! Why not publish your data too? \|journal=Intelligence \|volume=40 \|issue=2 \|pages=73–76 \|year=2012}}</ref> In 2017, an article published in ''[[Scientific Data (journal)\|Scientific Data]]'' suggested that this may not be sufficient and that the whole analysis context should be disclosed.<ref>{{cite journal\|last1=Pasquier\|first1=Thomas\|last2=Lau\|first2=Matthew K.\|last3=Trisovic\|first3=Ana\|last4=Boose\|first4=Emery R.\|last5=Couturier\|first5=Ben\|last6=Crosas\|first6=Mercè\|last7=Ellison\|first7=Aaron M.\|last8=Gibson\|first8=Valerie\|last9=Jones\|first9=Chris R.\|last10=Seltzer\|first10=Margo\|title=If these data could talk\|journal=Scientific Data\|date=5 September 2017\|volume=4\|issue=1 \|page=170114\|doi=10.1038/sdata.2017.114\|pmid=28872630\|pmc=5584398\|bibcode=2017NatSD...470114P}}</ref>

	In economics, concerns have been raised in relation to the credibility and reliability of published research. In other sciences, reproducibility is regarded as fundamental and is often a prerequisite to research being published, however in economic sciences it is not seen as a priority of the greatest importance. Most peer-reviewed economic journals do not take any substantive measures to ensure that published results are reproducible, however, the top economics journals have been moving to adopt mandatory data and code archives.<ref>{{cite journal \|last1=McCullough \|first1=Bruce \|title=Open Access Economics Journals and the Market for Reproducible Economic Research \|journal=Economic Analysis and Policy \|date=March 2009 \|volume=39 \|issue=1 \|pages=117–126 \|doi=10.1016/S0313-5926(09)50047-1\|doi-access= }}</ref> There is low or no incentives for researchers to share their data, and authors would have to bear the costs of compiling data into reusable forms. Economic research is often not reproducible as only a portion of journals have adequate disclosure policies for datasets and program code, and even if they do, authors frequently do not comply with them or they are not enforced by the publisher. A Study of 599 articles published in 37 peer-reviewed journals revealed that while some journals have achieved significant compliance rates, significant portion have only partially complied, or not complied at all. On an article level, the average compliance rate was 47.5%; and on a journal level, the average compliance rate was 38%, ranging from 13% to 99%.<ref>{{cite journal \|last1=Vlaeminck \|first1=Sven \|last2=Podkrajac \|first2=Felix \|title=Journals in Economic Sciences: Paying Lip Service to Reproducible Research? \|journal=IASSIST Quarterly \|date=2017-12-10 \|volume=41 \|issue=1–4 \|page=16 \|doi=10.29173/iq6 \|url=https://iassistquarterly.com/index.php/iassist/article/view/6/905\|hdl=11108/359 \|s2cid=96499437 \|hdl-access=free }}</ref>		In economics, concerns have been raised in relation to the credibility and reliability of published research. In other sciences, reproducibility is regarded as fundamental and is often a prerequisite to research being published, however in economic sciences it is not seen as a priority of the greatest importance. Most peer-reviewed economic journals do not take any substantive measures to ensure that published results are reproducible, however, the top economics journals have been moving to adopt mandatory data and code archives.<ref>{{cite journal \|last1=McCullough \|first1=Bruce \|title=Open Access Economics Journals and the Market for Reproducible Economic Research \|journal=Economic Analysis and Policy \|date=March 2009 \|volume=39 \|issue=1 \|pages=117–126 \|doi=10.1016/S0313-5926(09)50047-1\|doi-access= }}</ref> There is low or no incentives for researchers to share their data, and authors would have to bear the costs of compiling data into reusable forms. Economic research is often not reproducible as only a portion of journals have adequate disclosure policies for datasets and program code, and even if they do, authors frequently do not comply with them or they are not enforced by the publisher. A Study of 599 articles published in 37 peer-reviewed journals revealed that while some journals have achieved significant compliance rates, significant portion have only partially complied, or not complied at all. On an article level, the average compliance rate was 47.5%; and on a journal level, the average compliance rate was 38%, ranging from 13% to 99%.<ref>{{cite journal \|last1=Vlaeminck \|first1=Sven \|last2=Podkrajac \|first2=Felix \|title=Journals in Economic Sciences: Paying Lip Service to Reproducible Research? \|journal=IASSIST Quarterly \|date=2017-12-10 \|volume=41 \|issue=1–4 \|page=16 \|doi=10.29173/iq6 \|url=https://iassistquarterly.com/index.php/iassist/article/view/6/905\|hdl=11108/359 \|s2cid=96499437 \|hdl-access=free }}</ref>

	A 2018 study published in the journal ''[[PLOS ONE]]'' found that 14.4% of a sample of public health statistics researchers had shared their data or code or both.<ref>{{Cite journal\|date=2018\|title=Use of reproducible research practices in public health: A survey of public health analysts.\|journal=PLOS ONE\|volume=13\|issue=9\|~~pages~~=e0202447\|issn=1932-6203\|oclc=7891624396\|bibcode=2018PLoSO..1302447H\|last1=Harris\|first1=Jenine K.\|last2=Johnson\|first2=Kimberly J.\|last3=Carothers\|first3=Bobbi J.\|last4=Combs\|first4=Todd B.\|last5=Luke\|first5=Douglas A.\|last6=Wang\|first6=Xiaoyan\|doi=10.1371/journal.pone.0202447\|pmid=30208041\|pmc=6135378\|doi-access=free}}</ref>		A 2018 study published in the journal ''[[PLOS ONE]]'' found that 14.4% of a sample of public health statistics researchers had shared their data or code or both.<ref>{{Cite journal\|date=2018\|title=Use of reproducible research practices in public health: A survey of public health analysts.\|journal=PLOS ONE\|volume=13\|issue=9\|article-number=e0202447\|issn=1932-6203\|oclc=7891624396\|bibcode=2018PLoSO..1302447H\|last1=Harris\|first1=Jenine K.\|last2=Johnson\|first2=Kimberly J.\|last3=Carothers\|first3=Bobbi J.\|last4=Combs\|first4=Todd B.\|last5=Luke\|first5=Douglas A.\|last6=Wang\|first6=Xiaoyan\|doi=10.1371/journal.pone.0202447\|pmid=30208041\|pmc=6135378\|doi-access=free}}</ref>

	There have been initiatives to improve reporting and hence reproducibility in the medical literature for many years, beginning with the [[Consolidated Standards of Reporting Trials\|CONSORT]] initiative, which is now part of a wider initiative, the [[EQUATOR Network]].		There have been initiatives to improve reporting and hence reproducibility in the medical literature for many years, beginning with the [[Consolidated Standards of Reporting Trials\|CONSORT]] initiative, which is now part of a wider initiative, the [[EQUATOR Network]].
	This group has recently turned its attention to how better reporting might reduce waste in research,<ref>{{Cite web\|title=Research Waste/EQUATOR Conference {{!}} Research Waste \|url=http://researchwaste.net/research-wasteequator-conference/ \|website=researchwaste.net ~~\|url-status=dead~~ \|archive-url=https://web.archive.org/web/20161029015313/http://researchwaste.net~~:80~~/research-wasteequator-conference/ \|archive-date=29 October 2016}}</ref> especially biomedical research.		This group has recently turned its attention to how better reporting might reduce waste in research,<ref>{{Cite web \|title=Research Waste/EQUATOR Conference {{!}} Research Waste \|url=http://researchwaste.net/research-wasteequator-conference/ \|website=researchwaste.net \|archive-url=https://web.archive.org/web/20161029015313/http://researchwaste.net/research-wasteequator-conference/ \|archive-date=29 October 2016 \|access-date=18 October 2015 }}</ref> especially biomedical research.

	Reproducible research is key to new discoveries in [[pharmacology]]. A Phase I discovery will be followed by Phase II reproductions as a drug develops towards commercial production. In recent decades Phase II success has fallen from 28% to 18%. A 2011 study found that 65% of medical studies were inconsistent when re-tested, and only 6% were completely reproducible.<ref>{{Cite journal\|last1=Prinz \|first1=F. \|last2=Schlange \|first2=T. \|last3=Asadullah \|first3=K. \|doi=10.1038/nrd3439-c1 \|title=Believe it or not: How much can we rely on published data on potential drug targets? \|journal=Nature Reviews Drug Discovery \|volume=10 \|issue=9 \|page=712 \|year=2011 \|pmid=21892149 \|doi-access=free}}</ref>		Reproducible research is key to new discoveries in [[pharmacology]]. A Phase I discovery will be followed by Phase II reproductions as a drug develops towards commercial production. In recent decades Phase II success has fallen from 28% to 18%. A 2011 study found that 65% of medical studies were inconsistent when re-tested, and only 6% were completely reproducible.<ref>{{Cite journal\|last1=Prinz \|first1=F. \|last2=Schlange \|first2=T. \|last3=Asadullah \|first3=K. \|doi=10.1038/nrd3439-c1 \|title=Believe it or not: How much can we rely on published data on potential drug targets? \|journal=Nature Reviews Drug Discovery \|volume=10 \|issue=9 \|page=712 \|year=2011 \|pmid=21892149 \|doi-access=free}}</ref>
Line 59:		Line 59:
	In March 1989, [[University of Utah]] chemists Stanley Pons and Martin Fleischmann reported the production of excess heat that could only be explained by a nuclear process ("[[cold fusion]]"). The report was astounding given the simplicity of the equipment: it was essentially an [[electrolysis]] cell containing [[heavy water]] and a [[palladium]] [[cathode]] which rapidly absorbed the [[deuterium]] produced during electrolysis. The news media reported on the experiments widely, and it was a front-page item on many newspapers around the world (see [[science by press conference]]). Over the next several months others tried to replicate the experiment, but were unsuccessful.<ref>{{cite journal\|title=Physicists Debunk Claim Of a New Kind of Fusion\|newspaper=New York Times\|last=Browne\|first=Malcolm\|url=http://partners.nytimes.com/library/national/science/050399sci-cold-fusion.html\|date=3 May 1989\|access-date=3 February 2017}}</ref>		In March 1989, [[University of Utah]] chemists Stanley Pons and Martin Fleischmann reported the production of excess heat that could only be explained by a nuclear process ("[[cold fusion]]"). The report was astounding given the simplicity of the equipment: it was essentially an [[electrolysis]] cell containing [[heavy water]] and a [[palladium]] [[cathode]] which rapidly absorbed the [[deuterium]] produced during electrolysis. The news media reported on the experiments widely, and it was a front-page item on many newspapers around the world (see [[science by press conference]]). Over the next several months others tried to replicate the experiment, but were unsuccessful.<ref>{{cite journal\|title=Physicists Debunk Claim Of a New Kind of Fusion\|newspaper=New York Times\|last=Browne\|first=Malcolm\|url=http://partners.nytimes.com/library/national/science/050399sci-cold-fusion.html\|date=3 May 1989\|access-date=3 February 2017}}</ref>

	[[Nikola Tesla]] claimed as early as 1899 to have used a high frequency current to light gas-filled lamps from over {{convert\|25\|mi\|km}} away [[Wireless energy transfer\|without using wires]]. In 1904 he built [[Wardenclyffe Tower]] on [[Shoreham, New York\|Long Island]] to demonstrate means to send and receive power without connecting wires. The facility was never fully operational and was not completed due to economic problems, so no attempt to reproduce his first result was ever carried out.<ref>Cheney, Margaret (1999), ''Tesla, Master of Lightning'', New York: Barnes & Noble Books, {{ISBN\|0-7607-1005-8}}, pp. 107.; "Unable to overcome his financial burdens, he was forced to close the laboratory in 1905."</ref>		[[Nikola Tesla]] claimed as early as 1899 to have used a high frequency current to light gas-filled lamps from over {{convert\|25\|mi\|km}} away [[Wireless energy transfer\|without using wires]]. In 1904 he built [[Wardenclyffe Tower]] on [[Shoreham, New York\|Long Island]] to demonstrate means to send and receive power without connecting wires. The facility was never fully operational and was not completed due to economic problems, so no attempt to reproduce his first result was ever carried out.<ref>[[Margaret Cheney (author)\|Cheney, Margaret]] (1999), ''Tesla, Master of Lightning'', New York: Barnes & Noble Books, {{ISBN\|0-7607-1005-8}}, pp. 107.; "Unable to overcome his financial burdens, he was forced to close the laboratory in 1905."</ref>

	Other examples which contrary evidence has refuted the original claim:		Other examples which contrary evidence has refuted the original claim: