Skip to Main Content
HBS Home
  • About
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
Faculty & Research
  • Faculty
  • Research
  • Featured Topics
  • Academic Units
  • …→
  • Harvard Business School→
  • Faculty & Research→
Publications
Publications
  • November 9, 2019
  • Article
  • BMJ: British Medical Journal

Effect of Revealing Authors' Conflicts of Interests in Peer Review: Randomized Controlled Trial

By: Leslie K. John, George Loewenstein, Andrew Marder and Michael Callaham
  • Format:Electronic
ShareBar

Abstract

Objective: To assess the impact of disclosing authors’ conflict of interest declarations to peer reviewers at a medical journal.
Design: Randomised controlled trial.

Setting: The study was conducted within the manuscript review process at the Annals of Emergency Medicine.

Participants: Reviewers (n=885 reviewers) who reviewed manuscripts submitted between June 2, 2014 and January 23, 2018 inclusive (n=1480 manuscripts).

Intervention: Reviewers were randomised to either receive (treatment) or to not receive (control) authors’ full ICMJE-format conflict of interest disclosures before performing their review. Reviewers rated the manuscripts as usual on eight quality ratings and were then surveyed to obtain “counterfactual scores”—i.e., the scores they believed they would have given had they been assigned to the opposite arm—as well as attitudes toward conflicts of interest. .

Main outcome measures: The primary outcome measure was the overall quality score that reviewers assigned to the manuscript upon submitting their review (1 to 5 scale). The trial had 99% power to detect a .4 point difference, determined previously by editors to be the minimal significance threshold for a difference in score that would influence their editorial decision. Secondary outcomes looked at the scores the reviewers submitted for the seven more specific quality ratings as well as counterfactual scores elicited in the follow-up survey.

Results: Providing authors’ conflict of interest disclosures did not affect reviewers’ ratings of manuscript quality (Mcontrol=2.70 out of 5, SD=1.11; Mtreatment=2.74 out of 5, SD=1.13; mean difference=0.04, 95% CI=-0.05 to 0.14), even for manuscripts with disclosed conflicts (Mcontrol=2.85 out of 5, SD=1.12; Mtreatment=2.96 out of 5, SD=1.16; mean difference=0.11, 95% CI=-0.05 to 0.26). Similarly, there was no effect of the treatment on any of the other seven quality ratings the reviewers assigned. Reviewers acknowledged conflicts of interest as an important issue and believed they could correct for them when disclosed. Yet, their counterfactual scores did not differ from actual scores (Mactual=2.69, Mcounterfactual=2.67; difference in means=0.02, 95% CI=-0.02 to -0.01). When conflicts were reported, a comparison of different source types (e.g., government, for-profit corporation, etc.) found no difference in impact.

Conclusions: Current ethical standards require conflict of interest disclosure for all scientific reports. As currently implemented, this practice demonstrated no impact on any quality ratings of real manuscripts being evaluated for publication by real peer reviewers.

Keywords

Conflicts Of Interest; Peer Review; Randomized Controlled Trial; Scientific Publication; Conflict of Interests; Journals and Magazines; Science

Citation

John, Leslie K., George Loewenstein, Andrew Marder, and Michael Callaham. "Effect of Revealing Authors' Conflicts of Interests in Peer Review: Randomized Controlled Trial." BMJ: British Medical Journal 367, no. 8221 (November 9, 2019).
  • Find it at Harvard
  • Read Now

About The Author

Leslie K. John

Negotiation, Organizations & Markets
→More Publications

More from the Authors

    • March 2023
    • Marketing Letters

    Reaching for Rigor and Relevance: Better Marketing Research for a Better World

    By: Shilpa Madan, Gita Venkataramani Johar, Jonah Berger, Pierre Chandon, Rajesh Chandy, Rebecca Hamilton, Leslie John, Aparna Labroo, Peggy J. Liu, John G. Lynch, Nina Mazar, Nicole Mead, Vikas Mittal, Christine Moorman, Michael I. Norton, John Roberts, Dilip Soman, Madhu Viswanathan and Katherine White
    • 2023
    • American Journal of Health Promotion

    A Randomized Trial of Behavioral Nudges Delivered through Text Messages to Increase Influenza Vaccination among Patients with an Upcoming Primary Care Visit

    By: Mitesh S. Patel, Katherine L. Milkman, Linnea Gandhi, Heather N. Graci, Dena Gromet, Hung Ho, Joseph S. Kay, Timothy W. Lee, Jake Rothschild, Modupe Akinola, John Beshears, Jonathan E. Bogard, Alison Buttenheim, Christopher Chabris, Gretchen B. Chapman, James J. Choi, Hengchen Dai, Craig R. Fox, Amir Goren, Matthew D. Hilchey, Jillian Hmurovic, Leslie John, Dean Karlan, Melanie Kim, David Laibson, Cait Lamberton, Brigitte C. Madrian, Michelle N. Meyer, Maria Modanu, Jimin Nam, Todd Rogers, Renante Rondina, Silvia Saccardo, Maheen Shermohammed, Dilip Soman, Jehan Sparks, Caleb Warren, Megan Weber, Ron Berman, Chalanda N. Evans, Seung Hyeong Lee, Christopher K. Snider, Eli Tsukayama, Christophe Van den Bulte, Kevin G. Volpp and Angela L. Duckworth
    • January 2023
    • Journal of the Association for Consumer Research

    Calculators for Women: When Identity-Based Appeals Backfire

    By: Tami Kim, Kate Barasz, Michael I. Norton and Leslie K. John
More from the Authors
  • Reaching for Rigor and Relevance: Better Marketing Research for a Better World By: Shilpa Madan, Gita Venkataramani Johar, Jonah Berger, Pierre Chandon, Rajesh Chandy, Rebecca Hamilton, Leslie John, Aparna Labroo, Peggy J. Liu, John G. Lynch, Nina Mazar, Nicole Mead, Vikas Mittal, Christine Moorman, Michael I. Norton, John Roberts, Dilip Soman, Madhu Viswanathan and Katherine White
  • A Randomized Trial of Behavioral Nudges Delivered through Text Messages to Increase Influenza Vaccination among Patients with an Upcoming Primary Care Visit By: Mitesh S. Patel, Katherine L. Milkman, Linnea Gandhi, Heather N. Graci, Dena Gromet, Hung Ho, Joseph S. Kay, Timothy W. Lee, Jake Rothschild, Modupe Akinola, John Beshears, Jonathan E. Bogard, Alison Buttenheim, Christopher Chabris, Gretchen B. Chapman, James J. Choi, Hengchen Dai, Craig R. Fox, Amir Goren, Matthew D. Hilchey, Jillian Hmurovic, Leslie John, Dean Karlan, Melanie Kim, David Laibson, Cait Lamberton, Brigitte C. Madrian, Michelle N. Meyer, Maria Modanu, Jimin Nam, Todd Rogers, Renante Rondina, Silvia Saccardo, Maheen Shermohammed, Dilip Soman, Jehan Sparks, Caleb Warren, Megan Weber, Ron Berman, Chalanda N. Evans, Seung Hyeong Lee, Christopher K. Snider, Eli Tsukayama, Christophe Van den Bulte, Kevin G. Volpp and Angela L. Duckworth
  • Calculators for Women: When Identity-Based Appeals Backfire By: Tami Kim, Kate Barasz, Michael I. Norton and Leslie K. John
ǁ
Campus Map
Harvard Business School
Soldiers Field
Boston, MA 02163
→Map & Directions
→More Contact Information
  • Make a Gift
  • Site Map
  • Jobs
  • Harvard University
  • Trademarks
  • Policies
  • Accessibility
  • Digital Accessibility
Copyright © President & Fellows of Harvard College