Skip to main content

Estimating attrition in mild-to-moderate Alzheimer’s disease and mild cognitive impairment clinical trials

Abstract

Background

Participant retention is a key factor that affects clinical trial integrity. Trial protocols estimate attrition as a function of sample size calculations. Alzheimer’s disease (AD) is an area of active treatment development. We aimed to quantify the association between trial duration and completion rates and provide guidance for estimating attrition in AD trial protocols.

Methods

Using the Alzforum and ClinicalTrials.gov databases, we analyzed retention data from 125 mild-to-moderate AD and 12 mild cognitive impairment (MCI) clinical trials. We compared the rates of completion between trial arms (active vs. control) and ran regression models to test the hypothesis that trials with longer study duration have lower trial completion using all available data and restricting to placebo data. Our primary outcome was the odds of trial completion for a 6-month increase in trial duration. From the regression model, we estimated the proportion of participants completing 6-, 12-, and 18-month trials.

Results

We found that 21 (17%) mild-to-moderate AD trials and 1 (8%) MCI trial demonstrated greater dropout in treatment compared to placebo arms. For every 6-month increase in trial duration, there was a 27% decrease in the odds of trial completion (OR = 0.73; 95% CI 0.66, 0.81; p < 0.001) among participants in mild-to-moderate AD trials and a 55% decrease (OR = 0.45; 95% CI 0.36, 0.57; p < 0.001) among participants in MCI trials. The proportion of participants in the placebo group completing 6-, 12-, and 18-month trials were estimated to be 85.2%, 80.0%, and 73.3% for mild-to-moderate AD trials and 91.9%, 84.2%, and 71.3% for MCI trials, respectively.

Conclusions

Longer duration trials may be underpowered to demonstrate estimated treatment effects and may suffer from a greater risk of bias than do shorter trials.

Introduction

Participant retention directly affects the validity and generalizability of clinical trial results. Greater than anticipated dropout can lead to failed futility analyses, extended or prematurely terminated trials, and invalid or biased results [1,2,3]. Trials with greater than anticipated dropout are also at risk of being underpowered and unethical [4]. Guidelines from the American Academy of Neurology classify trial results with lower than 80% participant retention to be lower-quality evidence [5]. Accounting for anticipated dropout is critical to ensuring sufficient statistical power to detect an intervention effect if one exists [4, 6]. Research investigating how trial design features impact retention, however, remains limited. For example, limited information is available for expected attrition for trials of varying durations. There is a need for evidence-based retention estimates to guide trialists. To provide such estimates, we examined the impact of trial duration on study completion in mild-to-moderate Alzheimer’s disease (AD) and mild cognitive impairment (MCI) trials. AD is an active area of drug development and trials in this area face unique challenges in participant retention [7].

Methods

Using the AlzForum and ClinicalTrials.gov databases, we reviewed phase II and phase III placebo-controlled trials conducted between 1995 and 2022 (Table 1). We included parallel design trials with results available on ClinicalTrials.gov or in a peer-reviewed publication and excluded trials that terminated before completion. Trials with primary outcome measures that could add heterogeneity to our estimates (i.e., depression, sleep disturbance, agitation, psychosis) were excluded. We also excluded randomized withdrawal trials, crossover design studies, and trials with a primary outcome of time to conversion. When trials had a range of possible follow-up durations, we used the average reported duration. Trials with two cohorts of different trial durations were included as separate data points. We assessed only the double-blind phase of a study. We categorized trials into MCI or mild-to-moderate AD. The MCI analyses included studies that enrolled participants with MCI (based exclusively on clinical and cognitive diagnostic criteria) or “prodromal AD” (diagnostic criteria for MCI plus a biomarker for AD). Mild-to-moderate AD analyses included trials enrolling participants with “early AD” (frequently including mild dementia and MCI with a biomarker for AD), “mild AD dementia,” and “mild-to-moderate AD dementia” (typically based on diagnostic criteria and scores on the Mini-Mental State Exam).

Table 1 Criteria for inclusion

Statistical analyses

For trials that reported results by trial arm, we assessed the frequency with which trial arms differed in overall retention. When trials included more than one active arm, we combined those arms into a single group. To model associations between trial duration and retention, we used binomial regression. Robust variance estimates were used for all inferences to account for potential deviations from the binomial model mean–variance relationship. We ran the model using data from all participants (in the treatment and placebo groups) and with only the placebo group to remove potential bias due to treatment effects. Trials with no breakdown of participant trial completion by treatment group were removed from the placebo analysis. Based on the regression model, we estimated the proportion of participants completing 6-, 12-, and 18-month trials for MCI and mild-to-moderate AD trials. As exploratory analyses, we ran subgroup analyses by trial characteristics including, trial phase (phase II vs. III), therapeutic purpose (disease-modifying vs. symptomatic treatment), and trial site (single- vs. multi-site).

Results

One hundred twenty-five mild-to-moderate AD and twelve MCI trials met the criteria for inclusion in this study. Three mild-to-moderate AD trials did not report differences in completion by treatment arms. We found that 21 mild-to-moderate AD trials and 1 MCI trial demonstrated greater dropout in the treatment arm (Fig. 1). No trials were observed to have greater dropout in the placebo arm.

Fig. 1
figure 1

Treatment and placebo group differences in the observed proportion of trial completion

Trials with longer study duration had lower retention (Fig. 2). Using binomial regression with robust variance estimates to account for within-trial correlation, we estimated that a 6-month increase in trial duration was associated with a 27% decrease in the odds of trial completion (OR = 0.73; 95% CI 0.66, 0.81; p < 0.001) among mild-to-moderate AD trials and a 55% decrease (OR = 0.45; 95% CI 0.36, 0.57; p < 0.001) among MCI trials. In regression models including participants in both treatment and placebo groups, completion rates for 6-, 12-, and 18-month trials were estimated to be 82.6%, 77.5%, and 71.4% for mild-to-moderate AD trials and 91.7%, 83.3%, and 69.3% for MCI trials, respectively. In models including only participants in placebo groups, the completion rates for 6-, 12-, and 18-month trials were estimated to be 85.2%, 80.0%, and 73.3% for mild-to-moderate AD trials and 91.9%, 84.2%, and 71.3% for MCI trials, respectively. In exploratory analyses, we observed fairly consistent associations between trial duration and completion rates for subgroups of mild-to-moderate AD trials (sample size prevented subgroup analyses in MCI trials). Rates were similar for phase II (n = 84; OR = 0.72; 95% CI 0.59, 0.88; p = 0.002) and phase III (n = 39; OR = 0.74; 95% CI 0.66, 0.83; p < 0.001) trials; for disease-modifying (n = 89; OR = 0.77; 95% CI 0.69, 0.87; p < 0.001) and symptomatic (n = 35; OR = 0.42; 95% CI 0.15, 1.22; p = 0.1094) trials; and for single-site (n = 17; OR = 0.81; 95% CI 0.60, 1.08; p = 0.153) and multi-site (n = 107; OR = 0.73; 95% CI 0.66, 0.81; p < 0.001) trials (Additional file 1: Fig. S1).

Fig. 2
figure 2

Observed proportion of trial completion by trial duration

Discussion

In this analysis of MCI and mild-to-moderate AD trials, we found that participant retention was negatively associated with trial duration. The proportions of participants in the placebo group completing 6-, 12-, and 18-month trials were estimated to be 85.2%, 80.0%, and 73.3% for mild-to-moderate AD trials and 91.9%, 84.2%, and 71.3% for MCI trials, respectively. Trials with a duration greater than 6 months that incorporate common attrition estimates of 10% per year or 20% overall, therefore, may be at risk of being underpowered. This is concerning, since AD trials have increased in length over time [8].

While trials in both diagnostic groups presented similar patterns of attrition with increasing duration, mild-to-moderate AD trials were estimated to have lower completion estimates at 6- and 12-months while MCI trials were estimated to have lower completion rates at 18 months. Age may be a risk factor for participant dropout [9, 10], and participants in the mild-to-moderate AD trials are likely to be older than those in MCI trials [11]. AD is also a progressive disease. Previous examinations of data from NIH-funded Alzheimer’s Disease Research Centers (ADRC) identified worsening cognitive impairment as a risk factor for dropout [12]. This may be most relevant in longer MCI trials, where cognitive decline may be accompanied by the onset of functional impairment.

AD trial participants must co-enroll with a study partner [7]. Longer trials bring an added burden for dyads and may require more careful planning and resources to reduce modifiable barriers to trial completion [13]. For example, a previous study observed an association between the number of retention tactics used at NIH-funded ADRCs and rates of retention at 1 and 2 years [14]. Trials that require longer participation may also benefit from adjusting other study design features like reducing/replacing high-burden assessments (e.g., lumbar puncture) and increasing motivating factors such as returning test results [15] and even financial bonuses for trial completion [16].

While our study underscores the association between trial duration and retention, trial duration decisions are complex. For example, financial considerations may influence duration decisions [17]. Trials with objectives related to pharmacodynamics and pharmacokinetics may require shorter durations. Trials that enroll participants at earlier stages of the disease (e.g., MCI or early AD) may observe less change over time, therefore requiring longer durations [18, 19]. Our results suggest relatively similar associations between duration and retention between MCI and dementia trials. Symptomatic agents may produce measurable effects more rapidly, requiring shorter durations, while disease-modifying treatments may require a longer duration to detect measurable attenuation of AD progression [18]. Though the association with duration was similar between symptomatic and disease-modifying trials here, we note that symptomatic trials were heavily skewed toward shorter durations. We also found similar associations between phase II and phase III trials and between single- and multi-site trials. Overall, trial duration should not be determined based on anticipated retention rates; however, our findings suggest that duration does impact retention and can provide valuable insights into projected retention rates for investigators during the design stage.

Limitations

The retention estimates in this study may not be representative of all MCI and AD trials due to publication bias. We did not adjust the models in our study for other potential confounding factors that may be associated with retention, such as personal characteristics of the participants (e.g., study partner types, race and ethnicity, education) or trial design features (e.g., number of visits, alternate allocation, mode of treatment administration). Trials that were terminated before completion were excluded from our analysis, which could have biased our estimates, particularly if high dropout contributed to early termination. Some of the trials included in our analyses were ongoing during the COVID-19 pandemic, potentially resulting in a change in their protocols, including adopting a decentralized approach or extending trial duration. We were unable to assess how such changes affected retention estimates.

Conclusions

While decisions related to expected trial attrition are guided by several factors, these estimates may assist investigators in designing trials and suggest that attrition is associated with trial duration and may frequently exceed protocolized expectations.

Availability of data and materials

The datasets analyzed during this study are available on ClinicalTrials.gov or in peer-reviewed publications and its supplementary information files.

Abbreviations

AD:

Alzheimer’s disease

ADRC:

Alzheimer’s Disease Research Centers

CI:

Confidence interval

MCI:

Mild cognitive impairment

OR:

Odds ratio

PET:

Positron emission tomography

NIH:

National Institutes of Health

References

  1. French J, Gronseth G. Lost in a jungle of evidence: we need a compass. Neurology. 2008;71(20):1634–8.

    Article  PubMed  Google Scholar 

  2. Weuve J, Proust-Lima C, Power MC, Gross AL, Hofer SM, Thiébaut R, et al. Guidelines for reporting methodological challenges and evaluating potential bias in dementia research. Alzheimers Dement J Alzheimers Assoc. 2015;11(9):1098–109.

    Article  Google Scholar 

  3. Little RJ, D’Agostino R, Cohen ML, Dickersin K, Emerson SS, Farrar JT, et al. The prevention and treatment of missing data in clinical trials. N Engl J Med. 2012;367(14):1355–60.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  4. Halpern SD, Karlawish JHT, Berlin JA. The continuing unethical conduct of underpowered clinical trials. JAMA. 2002;288(3):358–62.

    Article  PubMed  Google Scholar 

  5. Edlund W, Gronseth G, So Y, Franklin G. Clinical practice guideline process manual. St. Paul, MN: Am Acad Neurol. 2004:1–57.

  6. Emanuel EJ, Wendler D, Grady C. What makes clinical research ethical? JAMA. 2000;283(20):2701–11.

    Article  PubMed  CAS  Google Scholar 

  7. Grill JD, Karlawish J. Addressing the challenges to successful recruitment and retention in Alzheimer’s disease clinical trials. Alzheimers Res Ther. 2010;2(6):34.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  8. Brøgger-Mikkelsen M, Zibert JR, Andersen AD, Lassen U, Hædersdal M, Ali Z, et al. Changes in key recruitment performance metrics from 2008–2019 in industry-sponsored phase III clinical trials registered at ClinicalTrials.gov. PLOS ONE. 2022;17(7):e0271819.

  9. Henley DB, Sundell KL, Sethuraman G, Schneider LS. Adverse events and dropouts in Alzheimer’s disease studies: what can we learn? Alzheimers Dement. 2015;11(1):24–31.

    Article  PubMed  Google Scholar 

  10. Bernstein OM, Grill JD, Gillen DL. Recruitment and retention of participant and study partner dyads in two multinational Alzheimer’s disease registration trials. Alzheimers Res Ther. 2021;13(1):16.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  11. William-Faltaos D, Chen Y, Wang Y, Gobburu J, Zhu H. Quantification of disease progression and dropout for Alzheimer’s disease. Int J Clin Pharmacol Ther. 2013;51(2):120–31.

    Article  PubMed  Google Scholar 

  12. Burke SL, Hu T, Naseh M, Fava NM, O’Driscoll J, Alvarez D, et al. Factors influencing attrition in 35 Alzheimer’s disease centers across the USA: a longitudinal examination of the National Alzheimer’s Coordinating Center’s Uniform Data Set. Aging Clin Exp Res. 2019;31(9):1283–97.

    Article  PubMed  Google Scholar 

  13. Gabel M, Bollinger RM, Knox M, Coble DW, Grill JD, Edwards DF, et al. Perceptions of research burden and retention among participants in ADRC cohorts. Alzheimer Dis Assoc Disord. 2022;36(4):281–7.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Grill JD, Kwon J, Teylan MA, Pierce A, Vidoni ED, Burns JM, et al. Retention of Alzheimer’s disease research participants. Alzheimer Dis Assoc Disord. 2019;33(4):299–306.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Ottenhoff L, Vijverberg EGB, Visser LNC, Verijp M, Prins ND, Van der Flier WM, et al. Experiences of and recommendations on clinical trial design in Alzheimer’s disease from the participant’s point of view: a mixed-methods study in two clinical trial centers in the Netherlands. Alzheimers Res Ther. 2023;15(1):72.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Largent EA, Fernandez LH. Making the case for completion bonuses in clinical trials. Clin Trials Lond Engl. 2019;16(2):176–82.

    Article  Google Scholar 

  17. Cummings JL, Goldman DP, Simmons-Stern NR, Ponton E. The costs of developing treatments for Alzheimer’s disease: a retrospective exploration. Alzheimers Dement. 2022;18(3):469–77. https://doi.org/10.1002/alz.12450.

    Article  PubMed  Google Scholar 

  18. Schneider LS. The potential and limits for clinical trials for early Alzheimer’s disease and some recommendations. J Nutr Health Aging. 2010;14(4):295–8. https://doi.org/10.1007/s12603-010-0066-1.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  19. Storandt M, Grant EA, Miller JP, Morris JC. Rates of progression in mild cognitive impairment and early Alzheimer’s disease. Neurology. 2002;59(7):1034–41. https://doi.org/10.1212/wnl.59.7.1034.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We thank the participants and study partners in these trials.

Funding

This study was supported by RF1 AG059407 and P30 AG066519.

Author information

Authors and Affiliations

Authors

Contributions

All authors participated in the design of the study. M.R. completed the statistical analysis with guidance from D.L.G. All authors contributed to the interpretation of the results and to the writing of the manuscript.

Corresponding author

Correspondence to Marina Ritchie.

Ethics declarations

Ethics approval and consent to participate

The analyses for the current study only used de-identified data and therefore do not meet the criteria for human subjects research.

Consent for publication

Not applicable.

Competing interests

M.R. and D.L.G. declare that they have no competing interests. J.D.G. reports research support from Eli Lilly, Genentech, Biogen, Eisai, NIA, the Alzheimer’s Association, and BrightFocus Foundation; he has provided consulting to SiteRx, Cogniciti, and Flint Rehab.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: Fig. S1.

The effect of trial duration on the observed proportion of participants completing a trial stratified by a) trial phase (phase II vs. phase III), b) therapeutic purpose (disease-modifying vs. symptomatic treatments), and c) site (multi- vs. single-site).

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ritchie, M., Gillen, D.L. & Grill, J.D. Estimating attrition in mild-to-moderate Alzheimer’s disease and mild cognitive impairment clinical trials. Alz Res Therapy 15, 203 (2023). https://doi.org/10.1186/s13195-023-01352-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13195-023-01352-0

Keywords