Charles M. Beasley, Jr and Roy Tamura: What We Know and Do Not Know by Conventional Statistical Standards About Whether a Drug Does or Does Not Cause a Specific Side Effect (Adverse Drug Reaction)
Charles Beasley’s additional reply to Edward Shorter’s comments on his “introductory comments”
We thank Prof. Shorter for raising the topic of dechallenge – rechallenge approaches to determining which adverse events (AEs) experienced by patients while taking a medication are adverse drug reactions (ADRs) to that medication. The basic approach is one of discontinuing medication after observing the occurrence of an AE; if the AE resolves in close temporal association to the discontinuation and clearance of the medication, there is some suggestion that the AE might be an ADR. The patient is then treated again with the medication, and if the AE recurs shortly after restarting the medication, that observation is considered stronger evidence that the AE is an ADR.
Our focus was on randomized clinical trials (RCTs) and the extent to which RCTs can (or cannot) establish, with the same robustness of scientific evidence required for a demonstration of efficacy for a medication to gain regulatory approval, that an AE is (or is not) an ADR. In Section 8 of our commentary, we briefly suggested that all parties should be aware of the limitations on the robustness of evidence for infrequent and rare AEs regarding whether or not they are ADRs. We firmly believe that the gold standard for such evidence is the placebo-controlled RCT (or a set of such trials). Acceptance of the uncertainty as to whether an infrequent or rare AE is or is not an ADR is a practical necessity if we are to continue to develop new medications. We also briefly alluded to the need for work directed at developing alternative methods to RCTs for substantially robust ascertainment of what are (and are not) the ADRs associated with a medication. The goal is identifying all ADRs, even those that are quite rare, without false identification. Our brief discussion focused on evolving epidemiological methods using large, collaborative databases.
We were remiss in not discussing the dechallenge – rechallenge approach in Section 8. Properly conducted, a dechallenge – rechallenge approach can provide as strong or stronger scientific/statistical evidence, relative to RCTs, as to whether an AE is or is not an ADR associated with a medication for an individual patient. If the dechallenge – rechallenge approach proves that the AE is an ADR for one patient, then the general case (the AE is an ADR) is proven for the medication. But for some individual patients, the AE could have alternative etiologies, as is always the case for an AE that has been demonstrated with robust evidence to be an ADR for a given medication. If the dechallenge – rechallenge approach either fails to prove that the AE is an ADR or proves that the AE is not an ADR for the patient, the AE could still be an ADR for other patients and therefore for the drug.
From our perspective, to be robust, the dechallenge – rechallenge approach should be conducted as a blinded “N-of-1” experiment with inferential statistical analyses (Kravitz and Dunn 2014). Conceptually, an “N-of-1” is a sequential set of cross-over trials in a single patient where test drug and placebo are administered multiple times in random order with a withdrawal period between the sequential trial treatment administrations. There are two limitations on the nature of AEs that can be studied with dechallenge – rechallenge methods. First, the AE must have a relatively rapid progression to maximal severity and, more importantly, resolve virtually completely on discontinuation of the causal medication. When such a trial design is used to study efficacy, statistical methods exist to consider continuous and ordinal assessments of the disorder being treated. However, for the assessment as to whether an AE is an ADR, we believe that the dependent (outcome) variable should be binary, the presence of the AE of any severity, or complete absence. If an AE only changes in severity with dechallenge and rechallenge, it remains present, and it would be difficult to interpret such results concerning the test medication as being an etiological contributor to the AE. An anaphylactic reaction would be an example of an AE that would be a good candidate for assessment through an “N-of-1” trial. Cases of agranulocytosis and aplastic anemia might be good candidates for study in an “N-of-1” trial.
Some AEs, once they occur, do not resolve, although the medication contribution to the pathophysiology leading to the AE would resolve on discontinuation of the medication. An example of such an AE would be myocardial infarction, where the medication accelerated coronary artery atherosclerosis as the pathophysiological process. Although the resolution of atherosclerotic plaques can occur and plaques can stabilize, lowering the probability of a subsequent myocardial infarction, these are slow processes. Such a progression and resolution process would not lend itself well to assessment with an “N-of-1” trial. Some other important AEs (malignancies, diabetes mellitus to some extent) would have the same difficulties in using an “N-of-1” trial approach, slow development and no resolution or delayed resolution following medication discontinuation.
The second limitation on the use of the “N-of-1” trial design is the matter of the ethics of potentially causing a serious AE in a patient. Closely related to the somewhat abstract matter of the ethics of such a study is the practical matter of whether patients would consent to participate in such a study where the AE is clinically serious and might be associated with permanent harm. Those AEs of greatest need for improved methods of assessment, infrequent and rare events, tend to be clinically serious events.
Beasley is aware of a few instances of simple (unblinded, single sequence) dechallenge – rechallenge evaluation of AEs temporally associated with medications on which he worked while employed by Lilly. These simple dechallenge – rechallenge studies added important information to the assessment of individual cases. This important information went well beyond the quality of the information provided by simply observing an AE and discontinuing the medication. However, the quality of this information still falls short of that provided by formal inferential comparison in a controlled, parallel RCT or a formal “N-of-1” trial.
Medical disorders can be episodic in their course. Disorders with such a course underscore the relative superiority of formal “N-of-1” trials over simple dechallenge observation and even decahallenge - rechallenge observation. Beasley recalls a personally very important example case that shaped his thinking about the quality of evidence in attributing an AE to treatment and declaring it an ADR. During a Phase 3 development study of a new molecular entity (NME), a subject developed significant neutropenia, approaching neutrophil indices values consistent with agranulocytosis but without symptoms of infection. The trial medication was discontinued, and the condition resolved. On internal unblinding of study medication for regulatory reporting purposes, it was found that the patient was being treated with placebo. It was ultimately concluded that this patient probably suffered from the rare condition of cyclic neutropenia (Dale, Bolyard, Marrero et al. 2012; Dale, Bolyard, Leung et al. 2017). According to Dale and colleagues (2012), this disorder was first described in 1910. However, its genetic etiology was described in 1999 (Horowitz, Benson, Person et al. 1999), several years after this case occurred.
If the patient had been rechallenged in a single rechallenge episode, recurrence would likely have been observed. If treatment assignment had been to the NME, such a sequence, the resolution on dechallenge and recurrence on rechallenge, could have been interpreted as evidence of treatment causation. As odds were that only one out of five of assignments was to placebo, it was likely that this case would have occurred on the NME (or another active drug included in the study); without an “N-of-1” study design, it could have been easily concluded that there was strong evidence for the neutropenia to be a treatment effect of the NME (or active comparator). This interpretation would have been reinforced by the experience with other drugs known to cause neutropenia and agranulocytosis if the NME (or active comparator) was considered to have a somewhat comparable molecular structure to that of clozapine. Except for a rather unlikely treatment assignment and especially if a simple dechallenge – rechallenge evaluation had been carried out, this one case could have had a profound negative impact on the development of the NME.
Better quality data is always preferable to lesser quality data, and we agree that for a certain domain of AEs, dechallenge – rechallenge studies, even those without multiple, controlled, sequential dechallenge – rechallenge periods, can add useful data in separating AEs from ADRs. However, for the long-term good of patients, the limitations on any method of study must be kept clearly in mind when interpreting study results and drawing conclusions about treatment effects.
References:
Dale DC, Bolyard AA, Marrero TM, Bonilla MA, Link DC, Newburger PE, Shimamura A, Boxer LA for The Severe Chronic Neutropenia International Registry and Repository. The natural history of cyclic neutropenia: long-term prospective observations and current perspectives. Blood. 2012; 120:2141.
Dale DC, Bolyard AA, Leung J, Tran E, Marrero TM, Newburger PE. Cyclic neutropenia, congenital and idiopathic neutropenia. Blood. 2017; 130:Suppl; 1:2275.
Horowitz M, Benson KF, Person RE, Aprikyan AG, Dale DC. Mutations in the ELA2, encoding neutrophil elastase, define a 21-day biological clock in cyclic haematopoiesis. Nat Genet. 1999; 23:433.
Kravitz RL, Duan N, eds, and the DEcIDE Methods Center N-of-1 Guidance Panel (Duan N, Eslick I, Gabler NB, Kaplan HC, Kravitz RL, Larson EB, Pace WD, Schmid CH, Sim I, Vohra S). Design and Implementation of N-of-1 Trials: A User’s Guide. AHRQ Publication No. 13(14)-EHC122-EF. Rockville, MD: Agency for Healthcare Research and Quality; January 2014.
May 21, 2020