Essential reading on transparency and reproducibility in pharmacoepidemiology research

We all recognize that, without transparency, trust in research evidence can falter. Most of us, therefore, feel a responsibility to increase the transparency and reproducibility of our work. However, good intentions aren’t always enough. We need concrete ideas, practical tools and examples that show us what better practice looks like and guide us to improve our processes.

Real-world evidence (RWE) increasingly plays a role in shaping clinical, regulatory and policy decisions. This, in turn, has led to an increasing focus on transparency. To build momentum to support our efforts, I’ve put together a list of 10 papers that I consider ‘must-reads’ to increase understanding of the issue of transparency and reproducibility in pharmacoepidemiology.¹ The list is not meant to be definitive or exhaustive. Rather, it is a curated selection of recent publications that highlight best practices, innovative frameworks and useful tools related to transparency and reproducibility in pharmacoepidemiology.

Here, I revisit the list and outline why the articles will be of interest to pharma professionals wishing to contribute to a more trustworthy and robust research evidence base.

Understanding the issue

Improving transparency to build trust in real-world secondary data studies for hypothesis testing—why, what, and how: recommendations and a road map from the real-world evidence transparency initiative² via Value in Health | 30-minute read

Transparency in real-world evidence processes is crucial to support the evaluation of research quality. This position paper – endorsed by the International Society for Pharmacoepidemiology (ISPE) – outlines the short-, medium- and long-term objectives of the RWE Transparency Initiative and advocates for the preregistration of ‘hypothesis evaluating treatment effectiveness’ study protocols to improve the transparency, traceability and credibility of RWE studies.

Reproducibility of real-world evidence studies using clinical practice data to inform regulatory and coverage decisions³ via Nature Communications | 40-minute read

In this article, Shirley Wang et al. report the results of REPEAT, a project committed to understanding the current state of reproducibility through meta-research and direct research replication. Shirley and coauthors replicated 150 RWE studies and found that, although many results are reproducible, greater methodological transparency is urgently needed to make it easier to reproduce and assess the validity of pharmacoepidemiology research. This article is a must-read for all pharma professionals wishing to appreciate the importance of transparent reporting.

Planning transparent research

HARmonized Protocol Template to Enhance Reproducibility of hypothesis evaluating real-world evidence studies on treatment effects: A good practices report of a joint ISPE/ISPOR task force⁴ via Pharmacoepidemiology and Drug Safety | 30-minute read

In 2021, ISPE and The Professional Society for Health Economics and Outcomes Research (ISPOR) convened a taskforce to address ambiguity in the communication of RWE study parameters, a problem that had limited the utility of RWE in healthcare decision-making. The HARmonized Protocol Template to Enhance Reproducibility (HARPER) provides a standardized structure for writing, registering and reporting reproducible RWE study protocols and is intended to be used throughout the research life cycle to encourage clear, consistent RWE reporting.

Process guide for inferential studies using healthcare data from routine clinical practice to evaluate causal effects of drugs (PRINCIPLED): considerations from the FDA Sentinel Innovation Center⁵ via BMJ | 30-minute read

PRINCIPLED is a structured process for planning, executing and reporting RWE studies with transparency and reproducibility in mind. This article summarizes the PRINCIPLED process and provides helpful flow diagrams to support pharma professionals who are interested in integrating transparency and reproducibility into drug development.

Conducting transparent research

Checklist and guidance on creating codelists for routinely collected health data research⁶ via NIHR Open Research | 25-minute read

Do you or your team contribute to the development of codelists? Make sure that you are following best practice by reviewing this guidance developed by Julian Matthewman et al. The paper presents a practical 10-step checklist for the production and transparent sharing of codelists used in studies based on routine health data.

Reporting transparent research

The reporting of studies conducted using observational routinely collected health data statement for pharmacoepidemiology (RECORD-PE)⁷ via BMJ | 1-hour read

How can you make sure that your pharmacoepidemiology manuscripts adhere to reporting requirements? This article introduces the RECORD-PE checklist for reporting pharmacoepidemiology research that uses non-randomized, routinely collected data. The guidelines should be used in parallel with the RECORD statement and STROBE guidelines to ensure that pharmacoepidemiology studies are fully and transparently reported.

Describing diversity of real-world data sources in pharmacoepidemiologic studies: The DIVERSE scoping review⁸ via Pharmacoepidemiology and Drug Safety | 25-minute read

Real-world data sources are frequently used in pharmacoepidemiology research. However, electronic databases have varying structures, formats and data sources, making it challenging to reproduce and validate results. In this scoping review, Rosa Gini et al. identify nine domains that can be used to guide transparent data source reporting in pharmacoepidemiology research. Including these domains in your reporting procedures is an easy way to enhance the replicability of your RWE studies.

Visualizations throughout pharmacoepidemiology study planning, implementation, and reporting⁹ via Pharmacoepidemiology and Drug Safety | 25-minute read

“Graphics and data visualizations support transparency by aiding communication and understanding” begins this 2022 article by Nicolle Gatto et al. This article demonstrates the importance of visual tools for enhancing transparency throughout the study life cycle. From documenting study design decisions to reporting research findings, graphics and data visualizations support clear communication and can support confidence in RWE. The recommendations for visualizing results are essential reading for anyone who develops or reviews publications.

Sharing is caring? International Society for Pharmacoepidemiology review and recommendations for sharing programming code¹⁰ via Pharmacoepidemiology and Drug Safety | 35-minute read

Between 2017 and 2022, just 4.8% of research articles published in Pharmacoepidemiology and Drug Safety shared programming code. Recognizing the need to improve code availability, John Tazare et al. recommend using persistent identifiers and appropriate licences, alongside adhering to good software practices to support standardizing code-sharing practices. Read this article to understand how to adopt best practice in your publications.  

Declaring transparent research

Building transparency and reproducibility into the practice of pharmacoepidemiology and outcomes research¹¹ via American Journal of Epidemiology | 20-minute read

How can publication professionals foster an environment that encourages and expects transparency? In this article, Shirley Wang and I propose a framework for explicit transparency declarations across five domains and encourage researchers and journals to embrace open practices. More information about our statement and its relevance for publication professionals can be found in this recent blog post.

Preparing for the future

Transparency and reproducibility in pharmacoepidemiology often remain as abstract ideals. However, although they may be ideals, they are not abstract. Instead, they are built out of practices we can adopt and develop.

I hope the list above is not just reading material, but serves as a starting point for action, collaboration and progress in building more trustworthy pharmacoepidemiology evidence. I’d be delighted to hear about how you are improving the transparency and reproducibility of RWE in your professional network – let me know by connecting with me on LinkedIn (Anton Pottegård: https://www.linkedin.com/in/pottegard/).

References

1. Pottegård A. Ten must-read papers on transparency and reproducibility in pharmacoepidemiology [letter]. Pharmacoepidemiol Drug Saf 2025;34:e70119. https://doi.org/10.1002/pds.70119.
2. Orsini LS, Berger M, Crown W et al. Improving Transparency to Build Trust in Real-World Secondary Data Studies for Hypothesis Testing—Why, What, and How: Recommendations and a Road Map from the Real-World Evidence Transparency Initiative. Value Health 2020;23:1128–36. https://doi.org/10.1016/j.jval.2020.04.002.
3. Wang SV, Sreedhara SK, Schneeweiss S et al. Reproducibility of real-world evidence studies using clinical practice data to inform regulatory and coverage decisions. Nat Comm 2022;13:5126. https://doi.org/10.1038/s41467-022-32310-3.
4. Wang SV, Pottegård A, Crown W et al. HARmonized Protocol Template to Enhance Reproducibility of hypothesis evaluating real-world evidence studies on treatment effects: A good practices report of a joint ISPE/ISPOR task force. Pharmacoepidemiol Drug Saf 2023;32:44–5. https://doi.org/10.1002/pds.5507.
5. Desai RJ, Wang SV, Sreedhara SK et al. Process guide for inferential studies using healthcare data from routine clinical practice to evaluate causal effects of drugs (PRINCIPLED): considerations from the FDA Sentinel Innovation Center. BMJ 2024;384:e076460. https://doi.org/10.1136/bmj-2023-076460.
6. Matthewman J, Andreson K, Suffel A et al. Checklist and guidance on creating codelists for routinely collected health data research. NIHR Open Res 2024;4:20. https://doi.org/10.3310/nihropenres.13550.2.
7. Langan SM, Schmist SAJ, Wing K et al. The reporting of studies conducted using observational routinely collected health data statement for pharmacoepidemiology (RECORD-PE). BMJ 2018;363:k3532. https://doi.org/10.1136/bmj.k3532.
8. Gini R, Pajouheshnia R, Gardarsdottir H et al. Describing diversity of real world data sources in pharmacoepidemiologic studies: The DIVERSE scoping review. Pharmacoepidemiol Drug Saf 2024;33:e5787. https://doi.org/10.1002/pds.5787.
9. Gatto NM, Wang SV, Murk W et al. Visualizations throughout pharmacoepidemiology study planning, implementation, and reporting. Pharmacoepidemiol Drug Saf 2022;31:1140–52. https://doi.org/10.1002/pds.5529.
10. Tazare J, Wang SV, Gini R et al. Sharing Is Caring? International Society for Pharmacoepidemiology Review and Recommendations for Sharing Programming Code. Pharmacoepidemiol Drug Saf 2024;33:e5856. https://doi.org/10.1002/pds.5856.
11. Wang SV and Pottegård A. Building transparency and reproducibility into the practice of pharmacoepidemiology and outcomes research. Am J Epidemiol 2024;193:1625–31. https://doi.org/10.1093/aje/kwae087.

---

The views expressed in this blog post are those of the authors and do not necessarily reflect those of Open Pharma or its Members and Supporters.

Anton Pottegård is a pharmacoepidemiologist and Professor at the University of Southern Denmark. He leads a team of more than 40 researchers and his research focuses on the use of drugs in the broadest sense.