Clinical Researcher—January 2018 (Volume 32, Issue 1)
Abbe Steel, MSc; Bert Hartog, PhD; Tom Howard; Jennifer Frasier; Jennifer Kelly; Elise Felicione, MPH, MBA
Global clinical trial enrollment rates across all therapeutic areas decreased from 75% in 2000 to 59% in 2006. Furthermore, retention of enrolled patients fell 21% over the same period.1 According to the Center for Information and Study on Clinical Research Participation, the percentage of screened patients who completed a clinical trial dropped from 49% during the 1999–2003 time period to 25% during 2009–13.
The requirements of a study are often more demanding than standard medical care, in that patients must invest significant time in making frequent office visits for blood tests and procedures, and in completing lengthy questionnaires. Total median procedures per protocol increased from 105.9 in 2000–03 to 166.6 in 2008–11.2
Given the need for new medicines, and the critical role that clinical trials play in the drug approval process, it is necessary for today’s pharmaceutical companies to design trials that fit with patients’ lifestyles and medical needs. Sponsors recognize the vast recruitment and retention challenges; however, very few include patient feedback as part of the study design or implementation.
This article describes a two-stage project that sought to capture patients’ voices in a clinical trial–specific survey, ultimately to guide programs that can improve patient enrollment, satisfaction, and retention.
There are many standardized general satisfaction surveys in healthcare with good validity and reliability, including the Patient Satisfaction Questionnaire Short Form (PSQ-18),3 the Picker Patient Experience Questionnaire (PPEQ -15),4 and the consumer assessment health plans (CAHPS).5 However, these surveys were not specifically developed for the clinical trial setting.
Because the overall clinical trial experience is impacted by so many unique factors related to study protocol and standards of care (i.e., research site staff, unapproved medication/placebo, need for increased tests/procedures, etc.), any survey that seeks to effectively assess the clinical trial participant experience must be designed for this setting.
HealthiVibe, LLC designed and administered a U.S.-based survey development study from April 2015 to August 2015, which allowed for development, pre-testing, and refinement of the survey instrument. The company then partnered with Janssen Research & Development, LLC to jointly conduct a global implementation study to assess logistics and proof-of-concept implementation in 10 countries. The global study was conducted from December 2015 to June 2016.
Survey Development Study
- Assess patient satisfaction with the clinical trial process
- Isolate drivers of satisfaction or dissatisfaction
- Identify what barriers hinder participation and completion, and any other aspects of the trial process the patients felt could be improved to positively impact their experience
- Ascertain the role played by site-specific considerations on trial participation, completion, and satisfaction
Issues explored as part of the survey development process included:
- The informed consent process
- Friendliness, preparation, and effectiveness of site staff
- Positive and negative impressions of waiting areas and exam rooms
- Flexibility and convenience
- Obstacles related to travel, child care, and other factors
- Study compensation
- Reasons for trial participation
- Likelihood of participation in additional trials
By administering the survey to clinical trial participants, Janssen’s goal was to identify areas of relative strength and weakness within the clinical trial, target areas for improvement, and gain a more substantive understanding of the patient experience. This would also provide insight into how the survey could be rolled out and standardized to a broader audience, and ultimately be validated as a benchmarking tool for comparison across trial types (phase, therapeutic area, complexity, etc.). This understanding could then be leveraged to help drive protocol design, site selection, and study execution moving forward.
A literature review was undertaken in April and May 2015 to evaluate trial participant experience topics/issues in published papers, reports, blogs, and videos.3,4,6–15
Detailed, open-ended, face-to-face and phone interviews were conducted with 24 U.S.-based past trial participants to understand their perspective on challenges they faced and how the clinical trial experience could be improved. The goal was to identify areas of concern from a patient perspective and improve the overall experience before, during, and after the trial, including any challenges they may have faced during enrollment in the trial, execution of the protocol, and study completion. These interviews were geared toward gathering information and expressions that could be used to develop potential survey questions (see Figure 1).
Figure 1: Sample Questions from Patient Interviews
|How did you first learn about the trial?|
|How long was the trial in terms of the length of time that you were involved?|
|How did the study site keep you informed about your test results, the number of people participating in the trial, or other trial-related details?|
|What questions or concerns did you have for the study doctor or study staff after you finished reading the informed consent form?|
|Looking back, do you think the information provided in the consent form was adequate given your experience in the study?|
|How did the doctors or nurses talk to you about your study medication? What types of questions did you have about the medication?|
|How did you get to and from your study (or doctor’s) visits when you were enrolled?|
|What challenges did you encounter during the enrollment period?|
|What was challenging or did you find to be particularly burdensome once you were enrolled and actively participating in the study?|
|Once the study ended, how did you feel about finishing your involvement in the clinical trial?|
|Did you ever consider dropping out of the study? (If yes, PROBES: painful tests, transportation, etc.)|
|Was there anything that could have been done differently that might have improved your clinical study experience?|
Draft Survey Implementation
The draft survey instrument was created on the basis of these 24 patient interviews, along with the literature search, and included a final selection of 50 multiple-choice questions. It was programmed in an online format in July 2015, and was fielded to 100 additional respondents who had participated in a Phase II or III clinical trial from 16 distinct disease areas within the past 10 years (n=100). The respondents were all located in the U.S. and ranged in age from 18 to 69 (62% male; 38% female).
The goal of the draft survey implementation was to test the comprehensibility and clarity of the survey instrument. Respondents completed the survey between July and August 2015.
Telephone Depth Interviews
Following the administration of the online survey, a sample of clinical trial survey respondents (n=11) participated in 45-minute telephone interviews in August 2015. The purpose of these interviews was to gain insight into the participant response to the survey content, formatting, clarity of questions, and user-friendliness.
Site Staff Interviews
Telephone interviews were also conducted in August 2015 with a representative sample of investigators and study coordinators (n=5) from U.S.-based research sites to obtain feedback on the survey. Site staff took the survey as part of the interview process, then provided specific feedback to questions and response options.
The draft instrument was revised, based on comprehensive review of patient and site feedback. Revisions included elimination of some items and rewording of others, due to such factors as a high non-response rate or response scales that showed little variability.
Themes and Question Selection
Six consistent themes were found across different sources of patient feedback. The final survey questions align with these themes. Additional context for each of these themes is provided in Table 1.
Table 1: Survey Themes
|Theme||Number of Questions||Examples of Topic Areas|
|Relationship Building & Support||7||
|Helping Self & Others||7||
The final survey resulted in 50 questions presented sequentially in several formats:
- Choose one or multiple responses from a defined list of possible statements (e.g., “Select all that apply”)
- Provide a rating using a 5-point Likert scale
In addition, the survey was designed without open-ended questions to avoid the possibility of adverse event reporting.
There was consensus among the site staff interviewed that measuring patient satisfaction in the clinical trial setting is critical for site/patient relationships, patient retention, and compliance. None of the interviewees had implemented a patient satisfaction questionnaire, nor did they have any process in place to assess patient satisfaction. All of those interviewed stated they would willingly offer a patient satisfaction survey to their research participants if one was provided.
Drivers to Satisfaction
Of the measures collected from participants, the top driver of participant overall satisfaction was having their health concerns addressed by staff during the study (0.688 Pearson correlation coefficient, accounting for 47% of the shared variance between the measures) (see Table 2).
Other top drivers included satisfaction with the answers to questions during the informed consent process (0.585 Pearson correlation coefficient, 34% of shared variance) and the opportunity to ask questions throughout the study (0.563 correlation coefficient, 32% of shared variance).
Three drivers were identified with a negative correlation to overall satisfaction. Respondents who considered stopping participation in the clinical trial for personal reasons were more likely to be dissatisfied (-0.558 correlation coefficient, 31% of shared variance), with two of the most common drivers of dissatisfaction being staff failing to keep the patient informed about his or her health (-0.508 correlation coefficient, 26% of shared variance) and the patient having to undergo tests seen as painful or scary (-0.425 correlation coefficient, 18% of shared variance).
Table 2: Top 10 Drivers of Overall Satisfaction in Clinical Trials
|Aspect of Clinical Trial||Correlation to Overall Satisfaction||Shared Variance|
|1. Health concerns addressed by staff||0.688||47%|
|2. Answers to questions during informed consent process||0.585||34%|
|3. Opportunity to ask questions||0.563||32%|
|4. Having a personal reason to consider stopping||-0.558||31%|
|5. Staff friendliness||0.542||29%|
|6. Helpfulness of participation to oneself||0.535||29%|
|7. Failure to inform patient about their health||-0.508||26%|
|8. Explanations during informed consent process||0.491||24%|
|9. Painful/scary tests||-0.425||18%|
|10. Satisfied with instructions regarding medicine||0.411||17%|
Ease of Use
Patients were asked about any challenges with taking the survey and to share their perspective on ease of use and navigating the survey site (see examples in Figure 2).
Figure 2: Patient Quotes Regarding Survey Ease of Use
|“I just had to press Next. It was very smooth to get through”|
|“It was fairly easy to use. I remember afterwards in my head I thought that wasn’t bad. When I saw the subject I thought it would be pretty clunky to take but it was well written and well laid out.”|
|“Yeah, it was real easy to navigate.”|
Global Implementation Study
Janssen contracted with HealthiVibe to conduct a clinical trial participant survey for a Phase III, multicenter, randomized, double-blind study in subjects with moderate to severe plaque-type psoriasis upon completion of a Week 60 follow-up visit. The survey launched initially in the U.S. in December 2015. Additional surveys were launched in the first half of 2016 in Australia, Canada, Germany, Korea, Poland, Russia, Spain, Taiwan, and United Kingdom. For this pilot, the surveys were limited to just those patients who completed the study. The survey closed in June 2016.
Ethics Committee Approvals
One month prior to launch, site staff received an e-mail that included a link to a five-minute training video and a link to an English version of the survey. Following the training, site staff were sent a Welcome Packet that included a supply of EC-approved, language-specific patient invitation postcards. Janssen’s local trial managers were centrally trained on key operational aspects to allow them to work with site staff for implementation.
During their Week 60 study follow-up visit, participants at sites in participating countries received a survey invitation via a language-specific postcard with a QR code allowing easy access to the survey website in the participants’ native languages.
The implementation survey was made available to 148 patients left in the study across 10 countries, at the time of EC approval. It included a subset of the questions refined through the earlier survey development study. Questions that were not relevant based on the trial design (i.e., compensation and medication use questions) were removed.
Respondents completed the web-based survey consisting of 35–40 questions, with the exact number varying based on skip logic. Answers were required to all questions.
Response Identification and De-Duplication
Responses were anonymous, and both Internet-based and location-based tracking were disabled to protect privacy and anonymity. No patient-identifiable information was collected. Response deduplication was therefore not possible.
Monthly reports were provided to the sponsor via an online reporting portal. Reports included response distribution frequencies, aggregate mean scores between 1.0 and 5.0 for all Likert-based questions, and aggregate scores for each themed group of questions and for the survey sample population as a whole. Site- and country-level responses were also made available through the online portal.
In consideration of patient privacy and to maintain anonymity of participants, scores were not reported to the sponsor unless certain thresholds were met at the site, country, and overall survey level.
For sites reporting fewer than five total respondents, the site’s aggregate score was not reported, and response distribution frequencies were not made available. For countries reporting fewer than five respondents and fewer than two reporting sites, the country’s aggregate score was not reported, and response distribution frequencies were not made available.
A total of 57 respondents took the survey, including 46 complete and 11 partial responses representing 25 different study sites across the 10 participating countries. Response rates varied by country, with Spain having the highest (50%) and Russia the lowest (1%). The average response rate for completed surveys across the 10 countries was 31%.
Among all respondents, 63% were male and 37% were female. The average survey completion time was 9.5 minutes, exclusive of the slowest and fastest 10% of respondents.
Ninety percent of respondents said they were “very” or “completely” satisfied with their overall clinical trial experience; 87% said the clinical trial experience “very much” or “completely” met their expectations; and no patients reported being only “slightly” or “not at all” satisfied by the overall trial experience.
Responses by Theme
“Relationship Building and Support” had the highest aggregate theme score (4.59 out of a possible 5.0) among respondents. The themes of “Communication,” “Helping Self and Others,” “Overall Satisfaction,” and “Site Experience,” while less highly rated, also exceeded the 4.0 threshold. The lowest aggregate theme score (3.62) was for “Convenience” (note that the “Compensation” theme was out of scope for this clinical trial).
As part of the pilot study, participating sites and sponsor staff were asked their feedback on the survey and the survey process, to help support the design and implementation of patient experience surveys for future trials. Insights were generated and feedback obtained in multiple areas, as described in the following sections.
Post-Study Site Feedback
Forty-six sites from eight countries provided their feedback regarding the implementation of the survey study. They were asked about their role in supporting the implementation and the value they believe the survey could bring.
The survey was perceived as “easy to implement” by 68% of sites, and 52% indicated they could see the value in it. Nearly 64% of sites indicated they were “satisfied” or “very satisfied” with the questions used in the patient satisfaction survey. Further, nearly 80% of respondents said they would recommend patient satisfaction surveys be conducted for all clinical trials.
There were clear learnings related to site communications, including the following:
- Clear and concise patient communication is needed.
- Introduction letters must be short and concise.
- Communication should come from country’s lead clinical research associate/site monitor.
- Staff should encourage patients to complete surveys onsite.
Additionally, there were clear learnings related to patient communications, including the following:
- One touch is not enough; multiple reminders support higher response rates.
- Encouragement from the site is critical.
The response rate in the pilot study could have been higher, had it not been for the fact that it was introduced late in an already ongoing clinical trial; sites had not been prepared during study start up. Therefore, it is recommended to make the survey part of the original submission package, including mention of the survey(s) in the study informed consent.
The survey instrument, as designed, is understood to be a baseline or template for patient experience surveys. IRB and EC feedback on using the survey has been positive and, given the novelty of this approach, our recommendation is to continue to inform IRBs and ECs about planned use of such surveys for reasons of transparency.
Meanwhile, sponsor modifications to the survey are to be expected, in order to capture datapoints specific to any given trial and to discard inapplicable questions, while retaining core questions that will help shape future benchmarking capabilities. It is important, from a patient experience perspective and from a response analytics perspective, that modifications are made with consistent question wording and response scales while paying attention to the ordering of questions and the overall length of the survey.
Additional changes were made to the survey instrument following the development study and the survey’s global trial implementation. These changes were implemented to extend the analytical capabilities of the survey response dataset, further simplify and clarify the content, and allow for industry benchmarking.
Survey Development Study: Ethical & Independent Review Services (Kansas City, Mo.)
Pilot Study: U.S. central IRB and country/site-level ECs
Conflicts of Interest
HealthiVibe was paid by Janssen to execute the pilot study.
- Lo C. 2014. The numbers game: boosting clinical trial enrolment. Pharm Tech www.pharmaceutical-technology.com/features/featurethe-numbers-game-boosting-clinical-trial-enrolment-4171654/
- Center for Information and Study on Clinical Research Participation. https://www.ciscrp.org/download/our-full-set-of-graphs-and-tables-in-pdf-format/?wpdmdl=4951
- Dawn AG, Lee PP. 2003. Patient satisfaction instruments used at academic medical centers: results of a survey. Am J Med Qual 18(6):265–9.
- Jenkinson C, Coulter A, Bruster S. 2002. The Picker Patient Experience Questionnaire: development and validation using data from in-patient surveys in five countries. Int J Qual Health Care 14(5):353–8.
- Agency for Healthcare Research and Quality. https://www.ahrq.gov/cahps/
- TEDx Talks. Fred Lee presentation at TEDxMaastricht 2011: The Future of Health. Patient satisfaction or patient experience? https://www.youtube.com/watch?v=tylvc9dY400
- Health Talks Online. healthtalksonline.com
- Gorski D. 2010. Integrating patient experience into research and clinical medicine: towards true “personalized medicine”. Sci-Based Med https://sciencebasedmedicine.org/integrating-patient-experience-into-research-and-clinical-medicine-towards-true-personalized-medicine/
- National Public Radio. 2012. One woman’s experience as a clinical trial “lab rat”. https://www.npr.org/2012/02/07/146531530/one-womans-experience-as-a-clinical-trial-lab-rat
- International Society for Pharmaceutical Engineering. 2014. Assessing patient experience with clinical trial medicine kits. https://www.ispe.org/ispeak/assessing-patient-experience-with-clinical-trial-medicine-kits
- Eli Lilly & Company. 2014. Improving the patient experience of clinical trials. http://docplayer.net/16455005-Improving-the-patient-experience-of-clinical-trials-eli-lilly-company.html
- Hussain-Gambles M. 2004. South Asian patients’ views and experiences of clinical trial participation. Family Practice 21:636–42.
- Nelson A, Fenlon D, Morris J, et al. 2013. QUALZICE: A QUALitative exploration of the experiences of the participants from the ZICE clinical trial (metastatic breast cancer) receiving intravenous or oral bisphosphonates. Trials 14:325.
- Ferguson PR. 2001. Patients’ experiences and views of clinical trials. Med Law 20:143–52.
- Heaven B, Murtagh M, Rapley T, et al. 2006. Patients or research subjects? A qualitative study of participation in a randomised controlled trial of a complex intervention. Patient Educ Couns 62:260–70.
Abbe Steel, MSc, (firstname.lastname@example.org) is Chief Executive Officer of HealthiVibe, LLC.
Bert Hartog, PhD, (BHartog@its.jnj.com) is a Senior Director for Janssen Clinical Innovation at Janssen Research & Development.
Tom Howard (email@example.com) is Chief Information Officer at HealthiVibe, LLC.
Jennifer Frasier (JFrasier@its.jnj.com) is Clinical Innovation Manager at PRA Health Sciences, performing services on behalf of Janssen Research & Development.
Jennifer Kelly (firstname.lastname@example.org) is Vice President at HealthiVibe, LLC.
Elise Felicione, MPH, MBA, (EFelicio@its.jnj.com) is a Senior Director for Janssen Clinical Innovation at Janssen Research & Development.