Clinical Researcher—January 2020 (Volume 34, Issue 1)

PEER REVIEWED

Robert Jeanfreau, MD, CPI; Nathan Best

 

Attributes of the thorough documentation of research data, endorsed worldwide by the U.S. Food and Drug Administration (FDA), the International Council for Harmonization (ICH), and the World Health Organization (WHO), are embodied in the ALCOA acronym, first used by an FDA Bioresearch Monitoring staff member.{1} “A” stands for accurate; “L” for legible; “C” for contemporaneous; “O” for original; and “A” for attributable. In 2010, the European Medicines Agency (EMA) added four additional attributes: Complete, Consistent, Enduring, and Available, thereby creating the more cumbersome acronym, ALCOACCEA.

The importance of thorough documentation is perhaps best appreciated by a closer inspection of the underlying rationale. As described in the following sections, there are three major, albeit related, reasons why adverse events (AEs), like all research data, should be thoroughly and consistently documented.

The Evaluation of Severity and Causality

First, a complete description of an AE is critical for the principal investigator’s (PI’s) evaluation of an AE’s relatedness, or causality, and severity to the investigational product (IP). A one-word description is useless in this regard.

Even in those situations in which a very brief description might appear to be adequate to determine causality, a careful description is still necessary. Let’s take as an example a study involving an IP to control atrial fibrillation. During the study, a subject is involved in a motor vehicle crash and reports that event to the coordinator. The subject denies any injury and provides no further description.

If no additional information is solicited, the PI may conclude that there was no AE and that the crash could not have been related to the IP. However, upon careful questioning, the subject thinks that he may have fallen asleep at the wheel. The subject is instructed to come to the site for an evaluation that reveals that the subject is having episodes of intermittent, complete heart block, which likely caused the subject to briefly lose consciousness.

Similarly, the PI cannot determine severity without an adequate description of the AE and its effect on the subject. Furthermore, based upon a good description of the AE, coupled with a thorough knowledge of the IP (including the mechanism of action, the half-life, and other information from the Investigator’s Brochure), along with background information on the subject’s medical history and any available laboratory studies, the PI may be able to formulate a differential diagnosis which then serves as a defensible basis for determining causality.

The Statistics of AEs

Secondly, this information serves as the starting point for the future statistical evaluation of a drug’s safety profile.

The pursuit of medication safety is a complicated and a never-ending process that encompasses the entire lifespan of a drug. Understandably, the younger the drug, the more intensive the scrutiny. The greatest number of safeguards are in place for experimental drugs in their infancy, since safety information on risks and benefits is at its nadir.

Although the specific process will vary depending upon the stage of the drug’s lifecycle, statistics plays an indispensable role at every step. Statistics require solid data, and critical to every phase is the reliable garnering of data which first begins with the collection of AEs at the research site level. The entire subsequent process of safety evaluation could be flawed if the initial data are inconsistent or incomplete.

One cannot help but wonder if various problems encountered in pharmacovigilance, at least in part, could be traced back to the inadequate description of AEs obtained during clinical studies. According to one source, “In an attempt to solve this problem, many systems have been developed for a structured and harmonized assessment of causality. None of these systems, however, have been shown to produce a precise and reliable quantitative estimation of relationship likelihood. Nevertheless, causality assessment has become a common routine procedure in pharmacovigilance.”{2}

Although the description of AEs is an essential component of source documentation, the far-reaching importance of completeness in the description of AEs and the subsequent causality assessment are not only underappreciated, but also being questioned outright. An article from 2017 suggests that the PI process of determining causality is so subjective that the practice should be abandoned. According to the authors, their analyses demonstrate that assigning causality to AEs “is a complex and difficult process that produces unreliable and subjective data. In randomized double-blind placebo-controlled trials where data are available to objectively assess relatedness of AE to treatment, attribution assignment should be eliminated.”{3}

The determination of causality would then become purely a statistical exercise utilizing double-blind placebo-controlled studies. This view is supported in the FDA’s Clinical Investigator Course of 2018. One slide states: “Individual assessment (is) unlikely to help determine attribution for common AEs, i.e. headache, nausea, MI in elderly. Such AEs require aggregate analyses using a population approach (risk or rate with study drug vs. control).”{4}

Subject Safety

Even if causality assessment of AEs by PIs is abandoned in the future, there remains a third, compelling reason for the thorough description of AEs. A complete description is necessary for the PI to ensure the totality of a subject’s safety.

The PI’s responsibility extends beyond ensuring that the potential risks attributable to the IP are minimized. In the ICH’s Good Clinical Practice (GCP) guideline, ICH-GCP 4.3.2 states: “During and following a subject’s participation in a trial, the investigator/institution should ensure that adequate medical care is provided to a subject for any adverse events, including clinically significant laboratory values, related to the trial. The investigator/institution should inform a subject when medical care is needed for intercurrent illness(es) of which the investigator becomes aware.”

To determine if a patient-reported AE is indicative of a significant health threat, the PI must have an adequate description of the AE. Let’s take, as an example, a subject having a headache. Causes of headaches are legion, and range from the relatively innocuous tension headache to the potentially life-threatening headache of a sentinel bleed.

Although the PI may feel that the IP is not causing the subject’s symptom, the PI’s involvement does not end there. If the subject’s description is suggestive of a sentinel bleed, the PI must act. As one source observes, “Patients with subarachnoid hemorrhage (SAH) frequently describe the occurrence of an underestimated or even ignored severe headache in the days or weeks preceding the bleeding. If recognized early, this warning headache might lead to specific investigations and, if indicated, a surgical approach might avoid a dramatic hemorrhagic event.”{5}

The Current Practice

Despite the clear importance of the proper collection of AEs, there are a number of challenges to its implementation, including the skill and time required.

Coordinators at research sites collect the vast majority of the data on subject-reported AEs and then present the information to the PI for evaluation of severity and causality. Interviewing subjects to obtain a complete description of an AE requires a complex skill set. Not only must the coordinator know which questions to ask for any given symptom, but also how to ask them and in what order. The result is lack of uniformity in how AEs are documented.

Bias, which can be nearly impossible to detect during an interview, can be bi-directional and requires considerable skill to avoid.

Admittedly, there has been an increasing emphasis on education and certification for coordinators. Moreover, many coordinators are nurses with the requisite skills. Nonetheless, the need for reproducible uniformity remains a concern. The amount and quality of information regarding AEs can vary widely within an organization’s sites and may even vary within a single site.

Another roadblock is the amount of time required for this process. The duties of coordinators are broad and increasing. The thorough collection of information for an AE can be a time-consuming process for a coordinator who is already overburdened with other duties. The frequent result is that the coordinator attempts to collect the information as quickly as possible. This pressured approach sometimes results in incomplete data. The frequency with which the PI, when presented with inadequate data, requests additional necessary information underscores the inefficiency of the current method.

A Different Approach

A potential solution is the use of well-crafted, electronic, self-administered questionnaires for the most common AEs. The questionnaires are loaded onto tablets available to subjects in the waiting room before a scheduled visit. These same questionnaires could also be accessed as an app on the subject’s phone for home use. The information is then presented as a summary for the PI’s review.

Importantly, in collecting subject data, the questionnaire presents qualifier questions with appropriate descriptions that a subject can easily process. Questions are layered into symptom tiers which are further layered into specific inquiries. The system is interactive in that answers to a question can alter which questions are subsequently presented.

The information within the questions is parceled to avoid overwhelming the subjects. The questions are also sequenced in such a way to make the flow of information intuitive. The questions and the sequence of questions are also fashioned to minimize the introduction of bias.

These goals, in part, are accomplished through logic trees and by presenting subjects with rational follow-up questions. The information is then processed in the background and sent to the PI in a narrative format, as can be seen in a sample questionnaire available online for a subject’s reported symptom of fever.

Advantages

This system offers a number of other advantages over the current approach.

First, because the template could be utilized across a wide spectrum of clinical trials, this type of approach presents a much-needed standard for the uniform collection of data for AEs. There is a long-standing, recognized need for the standardized characterization of AEs. In 2001, in an article addressing variability in the assessment of AEs, the authors conclude: “There was considerable variability in categorization of AEs in an exercise following training for AE data collection. Type of report, relatedness, and severity were found to have more variability in reporting than did action taken or outcome.”{6}

The past nearly two decades since that article have brought little progress in resolving this issue. In a recent review article of the analysis and reporting of AEs in randomized controlled trials (RCTs), the authors present similar conclusions: “This review highlighted that the collection, reporting, and analysis of AE data in clinical trials is inconsistent and RCTs as a source of safety data are underused. Areas to improve include reducing information loss when analyzing at patient level and inappropriate practice of underpowered multiple hypothesis testing. Implementation of standard reporting practices could enable a more accurate synthesis of safety data and development of guidance for statistical methodology to assess causality of AEs could facilitate better statistical practice.”{7}

A second advantage is that the system will save time for research staff, resulting in decreased sponsor costs. The technology, as used in electronic diaries for patient-reported outcomes, is readily available. Data security is ensured by not entering any identifiable subject information and by using vendors with a secure data exchange.

Lastly, such a platform offers a key component for the evolution of virtual clinical trials, which hold the promise of decreased trial costs, greater access to volunteers, and improved data quality.

For use on a wider scale, the questions could be approved by a panel of experts in the respective fields, with input from those in pharmacovigilance as well.

Conclusion

Laboring in an environment where there is no standard for approaching causality, the PI has no option but to rely on his or her own subjective approach. The first step in formalizing the approach to determining relatedness is to systematize the description of AEs.

Presently, the responsibility for the safety of current subjects and future patients rests squarely on the shoulders of the PI. A uniform system for collecting data will hopefully advance the industry’s search for safety.

References

1. Source Documentation. 2011. FCR – FDA Good Clinical Practice (GCP) Q&A. http://firstclinical.com/fda-gcp/?show=2011%2FRE+source+documentation
2. Naidu RP. 2013. Causality assessment: a brief insight into practices in pharmaceutical industry. Persp Clin Res 4(4):233–6. doi:10.4103/2229-3485.120173 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3835968/
3. Le-Rademacher J, Hillman SL, Meyers J, Loprinzi CL, Limburg PJ, Mandrekar SJ. 2017. Statistical controversies in clinical research: value of adverse events relatedness to study treatment: analyses of data from randomized double-blind placebo-controlled clinical trials. Ann Oncol 28(6):1183–90. doi:10.1093/annonc/mdx043 https://www.ncbi.nlm.nih.gov/pubmed/28184420
4. Yasinskaya Y. 2012. Safety Assessment in Clinical Trials and Beyond (U.S. Food and Drug Administration). https://www.fda.gov/downloads/Drugs/NewsEvents/UCM440805.pdf
5. Falco FAD. 2004. Sentinel headache. Neuro Sci 25(S3). doi:10.1007/s10072-004-0289-1 https://www.ncbi.nlm.nih.gov/pubmed/15549540
6. Raisch D, Troutman W, Sather M, Fudala P. 2001. Variability in the assessment of adverse events in a multicenter clinical trial. Clin Theraps 23(12):2011– doi.org/10.1016/S0149-2918(01)80153-3 https://www.sciencedirect.com/science/article/abs/pii/S0149291801801533
7. Phillips R, Hazell l, Sauzet O, Cornelius V. 2019. Analysis and reporting of adverse events in randomised controlled trials: a review. BMJ Open 9(2). https://bmjopen.bmj.com/content/9/2/e024537

Robert Jeanfreau, MD, CPI, (robertjeanfreau@medpharmics.com) is a Board Certified Internist affiliated with multiple hospitals and serves as the President and Medical Director of Medpharmics in Metairie, La.

Nathan Best is a Computer Consultant and Graphic Designer with Gnome Computer Workshop in the Greater New Orleans area.