David Healy: Do randomized clinical trials add or subtract from clinical knowledge


Carlos Morra: Alternative methodology to the RCT for evaluating the efficacy of therapeutic interventions for major depressive disorder:

The CODE system



        Major Depressive Disorder (MDD) is a highly prevalent psychiatric disorder that is considered one of the leading causes of the burden of global diseases by the World Health Organization (Malhi and Mann 2018; Bains and Abdijadid 2021). It affects around 300 million people worldwide, with an average lifetime prevalence of 12% and occurs nearly twice as often in women than in men (Otte, Gold, Penninx et al. 2016; Bains and Abdijadid 2021). MDD is a heterogenic disorder, with many subtypes that seem to behave as different diseases in terms of age of onset, symptoms, treatment response and prognosis (van Borkulo, Boschloo, Borsboom et al. 2015). It is a commonly held view that there is no alternative methodology to evaluate the clinical efficacy of drugs in MDD by the RCT.

        This essay will survey some relevant clinical studies that reviewed the efficacy of modern psychopharmacological and psychological interventions; it will analyze the methodologies utilized and the problems derived from their application. Finally, it will present alternative strategies to evaluate the efficacy of specific interventions for MDD.

        Several meta-analyses have been published comparing the efficacy of modern antidepressants and therapies; some of the most representative results will be presented. For example, a metanalysis by Vöhringer and Ghaemi (2011), which reviewed an extensive number of randomized controlled trials (RCTs), concluded that the antidepressants showed efficacy in moderate and severe, but not in mild MDD (Vöhringer and Ghaemi 2011). In addition, Nierenberg, Ostacher, Huffman et al. (2008) published a brief review of the efficacy and effectiveness of antidepressants for MDD and included several psychotherapies, i.e., cognitive behavioral therapy, mindfulness-based cognitive therapy, and concluded that these therapies showed efficacy for both acute treatment and relapse prevention in MDD (Nierenberg, Ostacher, Huffman  et al. 2008).

        In their famous metanalysis, Cipriani and collaborators compared the efficacy and tolerability of 21 antidepressants, including some of the newest ones such as vortioxetine, vilazodone and agomelatine (Cipriani, Furukawa, Salanti et al. 2018). The authors incorporated the number of trials and randomized patients in a complex graphic that allows visualizing the size of the evidence by proportionally increasing the width of the connectors or the size of the circle representing each of the treatments and the placebo (Cipriani, Furukawa, Salanti et al. 2018). The conclusions described a similar efficacy and side-effects profile to most treatments for MDD, with better tolerability for newer antidepressants than tricyclics. In contrast, amitriptyline and mirtazapine were the most efficacious and reboxetine the least efficacious compound (Cipriani, Furukawa, Salanti et al. 2018). However, the compared efficacy and tolerability variability increased in head-to-head studies, possibly associated with commercial interest biases and the possibility of guessing which drug the patient was on by their side effect profile (Cipriani, Furukawa, Salanti et al. 2018). In addition, in a metanalysis by Andrade (2018), when an antidepressant was the active compound, it was significantly more effective than the same drug when it was the active comparator (Andrade 2018).

        Another metanalysis published by Pim Cuijpers and collaborators (2013) studied the efficacy of several pharmacological treatments and psychotherapies in depressive and anxiety disorders. They concluded that pharmacotherapies and psychotherapies had a comparable effect in MDD (Cuijpers, Sijbrandij, Koole et al. 2013). However, some specific interventions such as tricyclic antidepressants and non-directive supportive counselling showed less efficacy for MDD than the other interventions (Cuijpers, Sijbrandij, Koole et al. 2013).

        Recently, several other meta-analyses have been published studying the efficacy of different psychotherapies. They have described far more complex problems and intrinsic limitations on the instruments and techniques utilized than pharmacologic interventions. For example, multiple variables are involved in the administration of psychotherapies, like the number and duration of sessions, the therapist's experience and the requirements of specific adaptations in some of the techniques to fit patient's needs (Serfaty, Csipke, Haworth et al. 2011). Moreover, in a metanalysis by Zhou et al. (2020), the differences between interpersonal therapy (IPT) and cognitive-behavioral therapy (CBT) depended on the scales used in the trials, with the Hamilton Depression Rating Scale (HDRS) and the Beck Depression Inventory (BDI) being the most frequently utilized. The low specificity of the HDRS can explain this phenomenon for MDD; moreover, several of its items assess anxiety and patients with generalized anxiety disorder without depressive symptoms can reach a high score using this scale (Olden, Rosenfeld, Pessin and Breitbart 2009).

        Several criticisms have been raised from authors like Khan and collaborators (2012) about the RCT methodology to evaluate psychotherapies, especially in the procedures of double-blinding, randomization and discriminating the effect of the intervention from placebo or active controls (Khan, Faucett, Lichtenberg et al. 2012; Steinert, Munder, Rabung et al. 2017). In addition, Steinert and collaborators (2017) found variable results in the efficacy of active treatments and controls, but they concluded that all the interventions, including psychodynamic psychotherapy, had similar efficacy (Steinert, Munder, Rabung et al. 2017). However, this type of study has several limitations which arise from the heterogeneity of the trials analyzed; they included 23 trials, 16 compared psychodynamic therapy (PT) with different forms of CBT, two trials compared PT with antidepressants and the remaining studies had no clear definition of the therapy utilized (Steinert, Munder, Rabung et al. 2017). 

        Finally, Zhou and colleagues (2020), analyzed 71 trials selected from 20,366 publications on treatments that included pharmacological and psychotherapeutic interventions in children and adolescents (Zhou, Teng, Zhang et al. 2020). They found that interpersonal therapy was more effective within the therapies than all the other psychological interventions and fluoxetine alone or combined with CBT seemed to be the best intervention for moderate to severe MDD (Zhou, Teng, Zhang et al. 2020). However, this study had several limitations, one of the most important of which was related to the low quality of the studies included, with several problems in implementing and maintaining the double-blind condition in the studies that included psychotherapies (Zhou, Teng, Zhang et al. 2020)

        There seems to be an unmet need to identify strategies to evaluate the efficacy of interventions for treating MDD (Stolk, Ten Berg, Hemels and Einarson 2003). However, several studies included in the Stolk’s group report provided evidence that comparing psychotherapies to pharmacologic treatments is at least questionable. First, they pointed that there are two interventions of a different nature that allegedly act by different mechanisms on different brain areas, like comparing apples and oranges (Layous, Chancellor, Lyubomirsky et al. 2011). Moreover, they have proved to be complementary interventions and their efficacy seems to increase when they are combined; such evidence was well established by trials either for acute treatment or for relapse prevention (Guidi, Fava, Fava and Papakostas 2011; Zhou, Teng, Zhang et al. 2020).

        Furthermore, several authors suggested that combining them with other interventions, such as physical activity and nutritional improvements, would achieve complete functional recovery (Layous, Chancellor, Lyubomirsky et al. 2011). Thus, the comparison of complementary interventions could not reflect their efficacy for MDD but represent the degree of affectation of the pathways by the disease, which could explain the variability of results between MDD subtypes.

        In summary, several relevant concluding remarks arise from the studies reviewed, such as that all the interventions have variable kinds of supporting evidence. However, some psychotherapeutic interventions have failed to achieve significance compared to placebo, although this might be related to intrinsic RCTs or psychometric issues, so careful analysis should be advised before translating their conclusions to clinical practice. Furthermore, the limited efficacy of these methodologies in heterogeneous disorders like MDD suggests that a different approach might be required than general RCTs.

        Furthermore, comparing the efficacy of these interventions of different natures seems to be a fallacy and experience has proved the advantages of combining them to achieve better efficacy and prevent relapses and recurrences. Moreover, the diagnosis of MDD is symptom-based with no objective measures (biomarkers) included in its diagnostic criteria and RCTs are focused on proving the efficacy of pharmacologic and psychotherapeutic interventions in the whole subject sample of MDD. In clinical practice, however, it is more beneficial to know which intervention is more effective on a patient's specific subtype of the disorder (Gaynes 2009).

        The efficacy of the interventions in specific subtypes and groups has not been extensively analyzed and requires more studies to identify each intervention's treatment response population. For example, one study by O'Brien and collaborators (1993) intended to identify treatment response profiles of antidepressants. However, despite failing to prove a better comparative response between the antidepressants, they found that endogenous depression had a better response rate than neurotic depression in all the treatment groups (O'Brien, McKeon and O'Regan 1993).

        Finally, there seems to be an unmet need to identify alternative strategies to evaluate the efficacy of interventions for treating MDD (Stolk, Ten Berg, Hemels and Einarson 2003). However, this appears impossible without identifying homogenous subtypes of MDD patients (Rapaport 2007). Thus, polydiagnostic (multiple diagnoses) instruments, such as CODE-DD, introduced a newer strategy for assessing the patient using the classifications and diagnoses of different authors simultaneously (Ban 1989). The procedure of this system consists of performing a general interview that identifies variables/symptoms which can then be processed simultaneously using 25 different diagnostic algorithms, manually or using the computer program, and the system provides the diagnoses for each classification concurrently.

        Furthermore, the system has several applications, such as identifying specific groups of patients with similar phenotypes that respond better or worse to specific treatments (Morra and Kreiker 2020). In terms of their predominant symptoms and treatment response profiles, these homogenous groups constitute what Tom Ban called "nosological homotypes," which he described as a leap forward for identifying biological categories of the disease (Ban 2007; Morra and Kreiker 2020). However, there is a need for more studies using this methodology before obtaining conclusive results and a revised version of the system, named CODE-UD (Composite Diagnostic Evaluation for Unipolar Depression), has been developed.

        CODE-UD includes 84 diagnostic classifications, including the International Classification of Diseases (ICDs), Diagnostic and Statistical Manual of Mental Disorders (DSMs) and five severity scales. It is compatible with artificial intelligence and predictive models and could help analyze the results of studies on biomarkers at a significant scale (Belsher, Smolenski, Pruitt et al. 2019; Morra and Kreiker 2020).




Andrade C. Relative Efficacy and Acceptability of Antidepressant Drugs in Adults With Major Depressive Disorder: Commentary on a Network Meta-Analysis. J Clin Psychiatry 2018;79(2):18f12254. 

Bains N, Abdijadid S. Major Depressive Disorder. In: StatPearls [Internet]. Treasure Island (FL): StatPearls Publishing; 2021. 

Ban TA. CODE-DD. Composite Diagnostic Evaluation of Depressive Disorders. Nashville: JM Productions; 1989. 

Ban TA. Towards a clinical methodology for neuropsychopharmacological research. Neuropsychopharmacologia Hungarica: a Magyar Pszichofarmakologiai Egyesulet lapja, official journal of the Hungarian Association of Psychopharmacology 2007;9(2):81–90. 

Belsher BE, Smolenski DJ, Pruitt LD, Bush NE, Beech EH, Workman DE, Morgan RL, Evatt DP, Tucker J, Skopp NA. Prediction Models for Suicide Attempts and Deaths: A Systematic Review and Simulation. JAMA Psychiatry 2019;76(6):642-51. 

Cipriani A, Furukawa TA, Salanti G, Chaimani A, Atkinson LZ, Ogawa Y, Leucht S, Ruhe HG, Turner EH, Higgins J, Egger M, Takeshima N, Hayasaka Y, Imai H, Shinohara K, Tajika A, Ioannidis J, Geddes JR. Comparative Efficacy and Acceptability of 21 Antidepressant Drugs for the Acute Treatment of Adults with Major Depressive Disorder: A Systematic Review and Network Meta-Analysis Lancet 2018;391(10128):1357-66.

Cuijpers P, Sijbrandij M, Koole SL, Andersson G, Beekman AT, Reynolds CF 3rd. The efficacy of psychotherapy and pharmacotherapy in treating depressive and anxiety disorders: a meta-analysis of direct comparisons. World Psychiatry 2013;12(2):137-48. 

Gaynes BN. Identifying difficult-to-treat depression: differential diagnosis, subtypes, and comorbidities. J Clin Psychiatry 2009;70 Suppl 6:10-5. 

Guidi J, Fava GA, Fava M, Papakostas GI. (2011). Efficacy of the sequential integration of psychotherapy and pharmacotherapy in major depressive disorder: a preliminary meta-analysis. Psychol Med 2011;41(2):321-31. 

Rapaport MH. Translating the evidence on atypical depression into clinical practice. J Clin Psychiatry 2007;68(4):e11. 

Khan A, Faucett J, Lichtenberg P, Kirsch I, Brown WA. A systematic review of comparative efficacy of treatments and controls for depression. PLoS One 2012;7(7):e41778. 

Layous K, Chancellor J, Lyubomirsky S, Wang L, Doraiswamy PM. Delivering happiness: translating positive psychology intervention research for treating major and minor depressive disorders. J Altern Complement Med 2011;17(8):675-83. 

Malhi GS, Mann JJ. Depression. Lancet 2018;392(10161):2299-2312. 

Morra C, Kreiker M. General Psychopathology 21. (Thomas A. Ban: Towards a clinical methodology for neuropsychological research). inhn.org.centraloffice. December 3, 2020. 

Nierenberg A, Ostacher M, Huffman J, Ametrano R, Fava M, Perlis R. A Brief Review of Antidepressant Efficacy, Effectiveness, Indications, and Usage for Major Depressive Disorder.  J Occup Environ Med 2008;50(4):428-36. 

O'Brien S, McKeon P, O'Regan M. The efficacy and tolerability of combined antidepressant treatment in different depressive subgroups. Br J Psychiatry 1993;162:363-8. 

Olden M, Rosenfeld B, Pessin H, Breitbart W. Measuring depression at the end of life: is the Hamilton Depression Rating Scale a valid instrument? Assessment 2009;16(1):43-54. 

Otte C, Gold SM, Penninx BW, Pariante CM, Etkin A, Fava M, Mohr DC, Schatzberg AF. Major depressive disorder. Nat Rev Dis Primers 2016;2:16065. 

Serfaty M, Csipke E, Haworth D, Murad S, King M. A talking control for use in evaluating the effectiveness of cognitive-behavioral therapy. Behav Res Ther 2011;49(8):433-40. 

Steinert C, Munder T, Rabung S, Hoyer J, Leichsenring F. Psychodynamic Therapy: As Efficacious as Other Empirically Supported Treatments? A Meta-Analysis Testing Equivalence of Outcomes. Am J Psychiatry 2017;174(10):943-53. 

Stolk P, Ten Berg MJ, Hemels ME, Einarson TR. Meta-analysis of placebo rates in major depressive disorder trials. Ann Pharmacother 2003;37(12):1891-9. 

van Borkulo C, Boschloo L, Borsboom D, Penninx BW, Waldorp LJ, Schoevers RA. Association of Symptom Network Structure With the Course of [corrected] Depression. JAMA Psychiatry 2015;72(12), 1219–26.  

Vöhringer PA, Ghaemi SN. Solving the antidepressant efficacy question: effect sizes in major depressive disorder. Clin Ther 2011;33(12):B49-61.

Zhou X, Teng T, Zhang Y, Del Giovane C, Furukawa TA, Weisz JR, Li X, Cuijpers P, Coghill D, Xiang Y, Hetrick SE, Leucht S, Qin M, Barth J, Ravindran AV, Yang L, Curry J, Fan L, Silva SG, Cipriani A, Xie P. Comparative efficacy and acceptability of antidepressants, psychotherapies, and their combination for acute treatment of children and adolescents with depressive disorder: a systematic review and network meta-analysis. Lancet Psychiatry 2020;7(7):581-601.


January 6, 2022