Bias in the reporting of harms in clinical trials of second-generation antidepressants for depression and anxiety: A meta-analysis

https://doi.org/10.1016/j.euroneuro.2016.09.370Get rights and content

Abstract

Previous research has shown that reporting bias has inflated the apparent efficacy of antidepressants. We investigated whether apparent safety was also affected. We included 133 trials, involving 31,296 patients, of second-generation antidepressants for the treatment of major depressive disorder (MDD) or anxiety disorders, obtained from Food and Drug Administration (FDA) reviews. We extracted data on overall discontinuation, discontinuation due to adverse events, and serious adverse events (SAEs). Meta-analysis was used to compare discontinuation rates between FDA reviews and matching journal articles, while SAEs were compared qualitatively. The odds ratio for overall discontinuation, comparing drug to placebo, was 1.0 for both sources, while that for discontinuation due to adverse events was 2.4 for both sources. Seventy-seven of 97 (79%) journal articles provided incomplete information on SAEs; sixty-one (63%) articles made no mention of SAEs at all. Of 21 articles which could be compared to the FDA, only 6 (29%) had full reporting without discrepancies. Nine (43%) articles reported a discrepant number of SAEs. Descriptions were absent or discrepant in 6 (29%) additional articles, even for important SAEs such as suicide attempts. In conclusion, reporting bias has not affected average discontinuation rates over trials. However, SAE reporting is not only very poor, with over half of articles failing to discuss SAEs altogether, but discrepancies between the FDA and articles were common and often led to a more favorable drug-placebo comparison. These findings suggest that journal articles are an unreliable source of data on SAEs in antidepressant trials.

Introduction

A significant fraction of all studies are never published in peer-reviewed journals (Dwan et al., 2013). Even within the subset of studies that are published, the (primary) analyses and outcomes reported in journal articles frequently deviate from the protocol (Chan et al., 2004, Dwan et al., 2014). As a consequence, statistically significant (positive) studies or outcomes are more likely to be published than non-significant (negative) studies (Dwan et al., 2013) or outcomes (Chan et al., 2004). While it is often difficult to assess the presence of reporting bias, the United States Food and Drug Administration (FDA) maintains an independent results database for drug trials, which can be used to examine the presence of reporting bias within a set of trials (Turner, 2004). This database has previously been used to assess reporting bias in trials of antipsychotics for schizophrenia (Turner et al., 2012) and antidepressants for major depressive disorder (MDD) (Turner et al., 2008) and anxiety disorders (Roest et al., 2015).

Second-generation antidepressants have been found to be effective for MDD (Turner et al., 2008) and anxiety disorders (Hidalgo et al., 2007, Blanco et al., 2013, Soomro et al., 2008, Andrisano 2013, Stein et al., 2006, Bandelow et al., 2015). They are considered to have a favorable risk-benefit profile and hence are widely prescribed (Olfson and Marcus, 2009). While both studies examining the FDA database of antidepressant trials confirmed their efficacy for MDD and anxiety disorders, they also revealed substantial reporting bias (Turner et al., 2008, Roest et al., 2015). Although nearly all published trials (94–96%) reported positive results, only 51% of all submitted trials for MDD, and 72% of those for anxiety disorders, were judged to be positive by the FDA. As a consequence of reporting bias, the effect size of antidepressant treatment was overestimated by 32% and 15% for MDD and anxiety disorders, respectively.

An accurate assessment of the risk-benefit ratio of antidepressants requires an unbiased understanding of safety as well as efficacy, but this other side of the coin has not, thus far, been examined as comprehensively. Previous research has indicated that reporting of harms in journal articles is incomplete and inadequate in various medical fields (Ioannidis and Lau, 2001, Wieseler et al., 2013, Loke and Derry, 2001), including psychiatry (Papanikolaou et al., 2004). The case of reboxetine demonstrates the impact that reporting bias can have on apparent safety as well as efficacy: inclusion of unpublished data not only shifted the difference in efficacy between reboxetine and placebo from significant to non-significant, but it also showed that reboxetine was significantly inferior to placebo in terms of selected harm outcomes, while the published trials suggested they were equivalent (Eyding and Lelgemann, 2010). Poor reporting of harms has also been found in trials of two other antidepressants (sertraline and duloxetine) and several antipsychotics with serious adverse events (SAEs) not always reported fully or accurately in journal articles (Maund et al., 2014, Hughes et al., 2014).

The work on antidepressant trials was limited to relatively recent trials of three antidepressants, and only the reboxetine study quantified the possible impact of bias on an important harm outcome, discontinuation from the trial. In the present study, we assessed the presence of reporting bias, and its impact on several harm outcomes, within a comprehensive set of trials of second-generation antidepressants for both MDD and anxiety disorders.

Section snippets

Data from FDA reviews and journal articles

We previously obtained FDA reviews of second-generation antidepressants approved for MDD (Turner et al., 2008) and/or anxiety disorders (Roest et al., 2015) (specifically generalized anxiety disorder (GAD), social anxiety disorder (SAD), obsessive-compulsive disorder (OCD), post-traumatic stress disorder (PTSD) and panic disorder (PD)). We defined second-generation antidepressants as including selective serotonin reuptake inhibitors (SSRIs), serotonin-norepinephrine reuptake inhibitors (SNRIs),

Discontinuation rates

As seen in Table 1, the average discontinuation rate (weighted by sample size) in the placebo group was 30.5% according to the FDA versus 30.1% according to the journal articles. For the antidepressant groups, the average discontinuation rate was 31.9% according to the FDA and 30.4% according to the journal articles. As shown in Table 2, random-effects meta-analysis of the FDA data yielded an odds ratio (OR) of discontinuation for the drug group compared with placebo of 1.02 (95% CI 0.96–1.08),

Principal findings

We found no evidence for bias in the reporting of overall discontinuation rates or discontinuation rates due to adverse events. However, discontinuation rates were high, averaging approximately 30% in both placebo and drug groups, especially given the short (6–12 weeks) duration of most of the included trials. Furthermore, participants receiving an antidepressant were 2.4 times more likely to discontinue the trial due to adverse events than participants receiving placebo.

Reporting of SAEs was

Role of the funding source

This study received no specific funding from any source.

Contributors

Ms. de Vries contributed to the study design, extracted and analyzed data, and drafted the manuscript. Dr. Roest contributed to the study design, extracted data, and critically revised the manuscript. Ms. Beijers extracted data and critically revised the manuscript. Dr. Turner and Dr. de Jonge contributed to the study design and critically revised the manuscript. All authors have read and approved the final version of the manuscript.

Conflict of interest

All authors declare that they have no conflicts of interest.

References (50)

  • H. Vartiainen et al.

    Double-blind study of mirtazapine and placebo in hospitalized patients with major depression

    Eur. Neuropsychopharmacol.

    (1994)
  • C. Andrisano et al.

    Newer antidepressants and panic disorder: a meta-analysis

    Int. Clin. Psychopharmacol.

    (2013)
  • D. Baldwin et al.

    Paroxetine in social phobia/social anxiety disorder: randomised, double- blind, placebo-controlled study

    Br. J. Psychiatry

    (1999)
  • B. Bandelow et al.

    Efficacy of treatments for anxiety disorders: a meta-analysis

    Int. Clin. Psychopharmacol.

    (2015)
  • C. Blanco et al.

    The evidence-based pharmacotherapy of social anxiety disorder

    Int. J. Neuropsychopharmacol.

    (2013)
  • J. Bradwejn et al.

    Venlafaxine extended-release capsules in panic disorder: flexible-dose, double-blind, placebo-controlled study

    Br. J. Psychiatry

    (2005)
  • K. Brady et al.

    Efficacy and safety of sertraline treatment of posttraumatic stress disorder

    J. Am. Med. Assoc.

    (2000)
  • A.-W. Chan et al.

    Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles

    J. Am. Med. Assoc.

    (2004)
  • K. Dwan et al.

    Systematic review of the empirical evidence of study publication bias and outcome reporting bias – an updated review

    PLoS One

    (2013)
  • K. Dwan et al.

    Evidence for the selective reporting of analyses and discrepancies in clinical trials: a systematic review of cohort studies of clinical trials

    PLoS Med

    (2014)
  • D. Eyding et al.

    Reboxetine for acute treatment of major depression: systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trials

    BMJ

    (2010)
  • R.N. Golden et al.

    Efficacy and tolerability of controlled-release and immediate-release paroxetine in the treatment of depression

    J. Clin. Psychiatry

    (2002)
  • T.A. Hammad et al.

    Suicidality in pediatric patients treated with antidepressant drugs

    Arch. Gen. Psychiatry

    (2006)
  • D. Hartung et al.

    Reporting discrepancies between the ClinicalTrials.gov results database and peer-reviewed publications

    Ann. Intern. Med.

    (2014)
  • D. Healy et al.

    Antidepressants and violence: problems at the interface of medicine and law

    PLoS Med.

    (2006)
  • R.B. Hidalgo et al.

    An effect-size analysis of pharmacologic treatments for generalized anxiety disorder

    J. Psychopharmacol.

    (2007)
  • E. Hollander et al.

    A double-blind, placebo-controlled study of the efficacy and safety of controlled-release fluvoxamine in patients with obsessive-compulsive disorder

    J. Clin. Psychiatry

    (2003)
  • S. Hughes et al.

    Differences in reporting serious adverse events in industry sponsored clinical trial registries and journal articles on antidepressant and antipsychotic drugs: a cross-sectional study

    BMJ Open

    (2014)
  • J.P.A. Ioannidis et al.

    Completeness of safety reporting in randomized trials

    J. Am. Med. Assoc.

    (2001)
  • J.P.A. Ioannidis et al.

    Better reporting of harms in randomized trials: an extension of the CONSORT statement

    Ann. Intern. Med.

    (2004)
  • J. Le Noury et al.

    Restoring study 329: efficacy and harms of paroxetine and imipramine in treatment of major depression in adolescence

    BMJ

    (2015)
  • M.R. Liebowitz et al.

    A randomized controlled trial of venlafaxine extended release in generalized social anxiety disorder

    J. Clin. Psychiatry

    (2005)
  • M.R. Liebowitz et al.

    A randomized, double-blind, fixed-dose comparison of paroxetine and placebo in the treatment of generalized social anxiety disorder

    J. Clin. Psychiatry

    (2002)
  • Y.K. Loke et al.

    Reporting of adverse drug reactions in randomised controlled trials – a systematic survey

    BMC Clin. Pharmacol.

    (2001)
  • R.D. Marshall et al.

    Efficacy and safety of paroxetine treatment for chronic PTSD: a fixed-dose, placebo-controlled study

    Am. J. Psychiatry

    (2001)
  • Cited by (23)

    • Comparison of consistency and complementarity of reporting biosimilar quality attributes between regulatory and scientific communities: An adalimumab case study

      2021, Biologicals
      Citation Excerpt :

      As expected, reporting on QAs varied between the two sources among the included biosimilars. This variation was in part due to the different aims of the two sources and was consistent with previous findings of substantial differences in reporting safety and efficacy information in regulatory reports and scientific publications [8–14]. Therefore, both sources should be systematically consulted to obtain comprehensive information on QAs for an improved understanding of how biosimilarity was established at the molecular level.

    • How to improve the integrity of clinical trial articles

      2018, Revista de Psiquiatria y Salud Mental
    View all citing articles on Scopus
    View full text