Assessing cognition and daily function in early dementia using the cognitive-functional composite: findings from the Catch-Cog study cohort

Alzheimer’s disease (AD) is the leading cause of dementia worldwide and has been the target of clinical trials and intervention studies for many years [1]. In the past decade, the research field has shifted towards earlier clinical stages of dementia and to predementia stages such as mild cognitive impairment (MCI) [2]. Remarkably, the selection of cognitive and functional outcome measures to evaluate treatment effects has not been adapted to the shift in treatment target populations. Measures originally designed for mild to severe dementia, such as the Alzheimer’s Disease Assessment Scale–Cognitive subscale (ADAS-Cog [3]), are still widely used as primary endpoints in MCI and early AD dementia trials. Several studies have shown that those older, traditional measures are insensitive to change over time in MCI and mild dementia [4, 5], as they focus on cognitive domains and everyday activities that are unaffected in those disease stages [5–7]. This limits their clinical relevance in the predementia window and leads to range restrictions in scoring (i.e., floor and ceiling effects) [4, 8]. Hence, researchers and regulatory agencies have expressed the urgent need for a sensitive measure that is capable of detecting clinically relevant changes at early clinical stages of AD [9–13]. The same holds for dementia with Lewy bodies (DLB), the second most common cause of dementia, of which both pathology and clinical manifestations show considerable overlap with AD [14, 15].

The Capturing Changes in Cognition (Catch-Cog) project was initiated to fulfill the need for a sensitive, clinically relevant outcome measure for use in MCI and mild dementia. We designed a novel cognitive-functional composite (CFC) measure in our expert working group, basing our selections on previously published work and input from MCI and dementia patients and caregivers [16]. The resulting CFC consists of a short cognitive test battery focusing on memory and executive functioning (EF) [17], as these are the cognitive domains that have been shown to decline in predementia and early stages of dementia [13]. The rationale of specific cognitive tests included in this battery has been described in more detail elsewhere [16]. Briefly, the selection was based on empirical evidence on their sensitivity to change, as reflected by the absence of floor and ceiling effects in MCI and mild AD [17]. To amplify its clinical relevance, we augmented the cognitive test battery with a previously validated everyday functioning measure: the Amsterdam Instrumental Activities of Daily Living Questionnaire (A-IADL-Q) [18]. The A-IADL-Q assesses the problems in cognitively complex everyday activities such as cooking, managing finances, and using technological devices [19, 20]. In item response theory analyses, these activities were found to be the activities most sensitive to cognitive decline [21]. The A-IADL-Q was previously demonstrated to be sensitive to the decline in dementia, as well as able to capture difficulties in instrumental activities of daily living (IADL) functioning in MCI and subjective cognitive decline (SCD) [21, 22].

Combining these cognitive and functional measures into the CFC summarizes cognitive and everyday performance and may thereby provide a single clinically relevant score. Both the Food and Drug Association (FDA) and the European Medical Agency (EMA) encourage the use of such composite endpoints to evaluate novel drug therapies and interventions [11, 12]. These agencies also stipulate that composite measures should be (1) carefully designed, (2) validated in an independent prospective cohort study, and (3) “bear some relevance to existing tools for which historical experience exists” [12]. In the Catch-Cog study, we have sought to meet these criteria by performing an extensive construct validation of the CFC. In a previous report, we demonstrated that the CFC exhibits high test-retest reliability and good feasibility of use [23], which are pivotal prerequisites for a reliable and valid outcome measure [24]. Having demonstrated this, we embarked on a longitudinal construct validation in an independent prospective cohort across the clinical spectrum from SCD to mild dementia. The main aim of this study is to validate the CFC in MCI and mild AD dementia stages, but we will also explore whether the CFC could be of use in individuals with SCD and DLB.

In the current study, we performed a psychometric evaluation of the CFC using baseline data of the Catch-Cog study. We aimed to evaluate several quality aspects of the CFC, including the construct validity, clinical relevance, and suitability for the target population. Therefore, we investigated the CFC’s factor structure and compared the CFC score with reference measures of disease severity such as informant reports and global cortical atrophy [16]. We also examined CFC score distributions in direct comparison to currently used tests, including the ADAS-Cog [3], the Clinical Dementia Rating scale Sum of Boxes (CDR-SB) [25], the Alzheimer’s Disease Cooperative Study–Activities of Daily Living scale (ADCS-ADL) [26], and the Alzheimer’s Disease Composite Score (ADCOMS) procedure [27].

We performed a psychometric evaluation of the novel CFC. Factor analyses confirmed the underlying structure of the CFC, reflecting the domain memory, EF, and IADL. The associations that we found between the CFC score and reference measures of disease severity further supported the construct validity and clinical relevance of the CFC. We also demonstrated that the CFC scores yielded fewer range restrictions in scoring as compared to traditional tests of cognition and function, indicating a better quality for the target population.

Construct validity, clinical relevance, and suitability for the target population are important quality aspects for clinical outcome measures [41]. Construct validity refers to whether an instrument measures what it intends to measure [24]. As a first step to assess this, we performed a CFA which confirmed that our novel combination of tests can be described by a memory, EF, and IADL component. Interestingly, we observed that the two cognition factors contributed differentially to the overall CFC score as seen in AD versus DLB, with the AD group scoring worse on the memory component compared to the EF component and the DLB group vice versa. Notably, this is in line with the clinical pictures of both diseases with memory problems as the most prominent symptoms of early AD as opposed to more predominant EF problems in early DLB [14]. These results further confirm that the CFC score is an adequate reflection of the dimensionality of the construct to be measured.

Another aspect of construct validity is testing whether scores on an instrument are associated with scores on instruments that measure a similar construct [24]. We found moderate-to-strong associations with the traditional measures of cognition and function, which supports the construct validity of the CFC. It should, however, be noted that cognitive composite included three ADAS-Cog measures, which probably accounts for the strong association we found between these measures. We also demonstrated that the CFC is associated with other clinically relevant measures of disease severity, such as cognitive decline as reported by the informant, quality of life, caregiver burden, and apathy. Additionally, the association between CFC score and GM volume, which has shown to be a good biomarker of neurodegeneration in AD [42, 43], illustrates that the CFC assesses a construct that is related to the underlying disease process. Taken together, these associations suggest that several clinically relevant aspects of the disease and its severity are captured by the CFC.

Quality for the target population was evaluated by inspecting range restrictions in scoring in our MCI and early dementia sample. This population largely corresponds with “stage 3 patients” as described in the recently proposed NIA-AA clinical staging scheme and aligned FDA guidance [11, 44]. We found that both the traditional tests and the recently designed ADCOMS showed ceiling effects in scoring in this stage, indicating that these participants showed a high level of functioning. These ceiling effects hamper the measurement of changes and especially the measurement of potential improvement due to treatment. In contrast, the score distributions of the CFC component were normally distributed, showing potential to indicate both decline and improvement with respect to baseline measurements. Therefore, these cross-sectional results support that the CFC is a promising measure with which to assess changes over time without exhibiting the limitations of traditionally used tests.

Several other endeavors have been undertaken to design and validate composite measures that are more appropriate to assess changes earlier clinical stages of AD. These composites range from purely statistically driven [27, 45] to more theoretically based such as the preclinical Alzheimer’s cognitive composite (PACC) [46] and the Alzheimer prevention initiative cognitive composite (APCC) [47]. The PACC and APCC focus on preclinical stages of AD and do not include a measure of daily function. However, measuring everyday functioning is highly relevant in the MCI and dementia stages, as evolving IADL problems are an important clinical hallmark in the transitional stage from MCI to dementia and predict a future decline in dementia [7]. The ADCOMS procedure includes a functional measure (CDR-SB), and the fact that previously changes were detected in a clinical trial [27] supports the idea that adding a functional measure advances a cognitive outcome measure. However, the ADCOMS selection has been largely driven by statistical considerations rather than its content, and therefore, its clinical relevance is as yet uncertain. It is our view that the CFC can contribute to the existing initiatives to improve cognitive measurement in the MCI and mild dementia stages, as its composition has been based on both theoretical constructs (i.e., the combination of cognition and IADL measure) and empirical research [17, 22].

There are some limitations to consider. First, not all MCI and AD participants had AD biomarkers available, and therefore, it is unknown whether they had AD pathology. However, in these circumstances, we relied on an extensive clinical assessment and excluded participants with other conditions that could have caused or contributed to the cognitive or functional symptoms. Second, some heterogeneity in our sample may have been caused by minor differences in the recruitment strategies employed across the centers as well as from including SCD and DLB participants. Consequently, our sample may not perfectly mirror the composition of an ideal clinical trial sample. It should also be noted that the SCD and DLB samples were relatively small and that the CFC investigations in these groups were of explorative nature. This limits the interpretation of the CFC results in those groups. Additionally, the sample size of participants with an MRI scan available was relatively small, as this was not required for participation in our study. Therefore, we should be careful with interpreting these findings, particularly in the SCD group. A further limitation is that we have only investigated a single weighting method to create the CFC score, whereas it is likely that the optimal scoring method involves different weights for the different components. For example, our data showed that the memory component seems to be relatively easy compared to the EF component, which might be something that we need to account for when tracking the changes over time, in particular for the MCI group. Lastly, the fact that the CFC in its current composition focuses less on other domains than episodic memory and EF may limit its usefulness for measuring progression at more severe stages of dementia.

Strengths of this study include our construct validity approach, which is a unique aspect of the Catch-Cog study. Given the lack of a gold standard for “disease severity,” we used different reference measures of disease severity to compare the CFC with, which led to converging evidence for the clinical relevance of the CFC. Additionally, we were able to perform a direct head-to-head comparison between the CFC and traditional tests, which, to our knowledge, has never been done in previous studies. The advantage of this is to reveal both the strengths and weaknesses of different clinical measurements. Furthermore, our investigation of the CFC in an independent, prospective cohort is an essential aspect of this study. Although the CFC consists of tests that have been validated as part of other test batteries and across several study cohorts, it cannot be assumed that all measures perform similarly in a novel composition. For example, tests may perform differently when assessed in a different test order. An independent validation of a novel composite measure such as the CFC measure is thus needed to enhance future implementation. We are currently assessing the CFC longitudinally in our Catch-Cog prospective study cohort, and we will investigate its sensitivity to change after 3, 6, and 12 months and compare its sensitivity with that of the traditional tests. This longitudinal data will also enable us to explore whether different weights for the subtests can improve the sensitivity to change over time, as well as whether different weights are useful when tracking a change in different diagnostic groups. For example, putting more weight on the cognitive parts and activities of daily living that decline relatively late in the disease course may enhance the use of the CFC to track progression in later disease stages.

We demonstrated that the CFC has good construct validity and captures clinically relevant aspects of disease progression. We also showed its improved suitability for the target population as compared to traditional tests, as reflected by fewer range restrictions in scoring. These findings illustrate that the CFC has good potential to be a sensitive, clinically meaningful outcome measure. It is therefore better indicated for use to evaluate cognition and function as compared to traditional tests, as in line with the recent FDA recommendations. Ultimately, the CFC can yield a more accurate and useful measurement of clinically relevant changes, which will aid the monitoring of disease progression and evaluation of novel treatments.