Comparison of Instrumental Activities of Daily Living assessment by in-person or phone interviews: A randomized, crossover trial

Background The functional autonomy assessment is essential to manage patients with a neurodegenerative disease, but is therefore not always possible during consultation. To optimize ambulatory autonomy assessment, the Lawton IADL questionnaire assessment between phone and in-person interviews was compared. Methods A randomized, crossover trial was carried-out in patients in memory clinic. The IADL questionnaire was collected for patients during phone and in-person interviews between nurses and patients’caregivers. The agreement between the two methods was measured with the proportion of participants giving the same response, the Cohen’s kappa, the intraclass correlation (ICC) coe�cient and Bland and Altman’s method. The associations between patients’characteristics, events occurring between the two measures and agreement were assessed. Results Among the 292 patients with two IADL measures, the agreement rate between the two modes ranged from 75% to 92% depending the IADL items, and from 84% to 97% depending of the level of autonomy. Kappa score was 0.3 for total score and ranged from 0.5 to 0.8 for the analysis by item or level of IADL. ICC score was 0.91 for total IADL score. Overall, 96.9% of measures lied within the 95% limits of agreement. The occurrence of fall was less likely associated with the probability to lie within the 95% limits of agreement (OR=0.07 [0.02-0.27], p<0.0001). Conclusions The administration of IADL by phone with the caregiver appears as an acceptable way of assessment for patients followed by MC compared to in-person interview. The events which could occur in a time close to the evaluation should be reported.


Background
The assessment of functional autonomy, an essential step in evaluating and caring patients in Memory Clinics (MC), is included in the diagnosis procedure to determine the stage of diseases.The major neurocognitive disorder (NCD) is thus de ned as a syndrome characterized by a decline in cognitive functions with impact patient's ability to perform everyday activities according to the DSM-V, while in mild NCD, the patient's abilities are not signi cantly impacted.Although there is no standard measurement of functional status, the Lawton IADL questionnaire is commonly used in MC during a in-person interview [1].The Lawton IADL questionnaire assesses the patients' ability to perform daily tasks considered as complex activities for which different cognitive processes are involved, and was shown to be a valid and reliable tool.The Lawton IADL scale has also been previsouly identi ed among the most appropriate outcomes to assess e cacy of new interventions in Alzheimer's disease [2].However, in current practice the systematic collection of IADL questionnaire can be problematic due to limited time available in the MC.Minimizing the rate of missing data is essential to manage patients and to ensure data quality for registries, cohorts and research.The collection of data with others modes of administration such as phone interview has been previously proposed to achieve such objective and while several previous studies have compared the phone and telehealth administration vs in-person interviews for different cognitive scales, none studied the Lawton IADL questionnaire [3,4].However the mode of administration can in uence the quality data [5].Before proposing such alternative in current practice or for research purpose, it is therefore essential to study the comparability of the Lawton IADL questionnaire collected by phone and in-person.We conducted a randomized crossover trial to measure the agreement between two modes of administration i.e phone or in-person interview of the IADL questionnaire.Furthermore, we assessed whether patient's characteristics may impact the degree of agreement.

Study design
We carried out a randomized, open-label, crossover trial, with two study periods (sequences AB/BA) and four assessment points (Figure 1).The trial was conducted in the same context of the MEMORA cohort which aims to study the relationship between patient characteristics and functional autonomy change over time among patients attending a MC [6,7].

Participants and setting
Eligibility criteria for participants were patients attending a memory visit for the rst time at the MC, aged 50 years or older, living at home, accompanied by an informal caregiver, and who agreed to participate in the study.Exclusion criteria were patients whose caregiver did not wish to participate in the study, patients whose caregiver did not provide a phone contact, and patients for whom the health status would require institutionalization during the period of the study.The study was conducted at a MC of the Memory Research Center of Lyon (France), between November 2014 and April 2016.

IADL questionnaire and modes of administration
The questionnaire used to assess the level of functionnal independence (or dependence) was the French version of the Lawton instrumental activities of daily living including the 8 items: Ability to use the phone, to go shopping, preparing food, do its housekeeping, do its activities bleaching, to use transportation, to ensure accountability in respect of salary and handle money [1].The questionnaire answers have been considered as a total score ranging from 0 (dependent) to 8 (independent), as well as 2 sub-scores of 4 based on previous research [8].The rst sub-score included the abilibies for phone, transportation, medications and nance, the second sub-score included the others 4 abilities.In addition, each item of the IADL questionnaire has been scored as binary variables (1: abillity to conduct the activity, 0: no ability).
The IADL questionnaire has been collected for the same patients using two modes of administration: by phone and in-person interview.Both interviews consisted of a collection of answers given by the caregiver to the nurses trained for this procedure.The questionnaire was identical in both modes of administration, the questions were asked in the same order and the nurses had to read the questionnaire exactly as it was written to ensure similar conditions for collection of data.
The potentially eligible patients were selected in the list of scheduled appointments in the MC.A letter to inform both patient and caregiver of the possibility to participate to the study was sent with the convocation for the clinical memory visit.The nurse contacted the patients and caregivers depending on the phone number available, presented the study, checked the eligible criteria and asked whether they agreed to participate.If they agreed to participate, they were assigned randomly in one of the branches of the study.For the patients in the 1st branch, the phone appointment was planned with the caregiver one month after the memory consultation.For the patient in the 2 nd branch for whom the phone appointment was the rst IADL measure, a phone appointment was planned with the caregiver one month before the memory consultation.In case the call did not succeed, it was expected that the nurse could call back the caregiver up to 4 times maximum, whatever the branch.

Patients and study characteristics
Additionaly to the IADL questionnaire collected during the MC visit, we considered the following patients' characteristics collected using the eCRF of the MEMORA study during the MC visit.We focused on the following patients' characteristics: education level, marital status, cognitive status, aetiology, the MMSE score ranging from 0 to 30 and evaluating the overall cognitive performance, and the relationship between the caregiver and the patient.Details of the collection of the data in the MEMORA study are available in previous open-access manuscript.
Additional information was collected at the second interview, either by in-person interview, either by phone in order to detect possible changes between the two measures: change of caregiver respondent between the 2 visits, change of nurse who administered the questionnaire, and the following events: entry in emergency department, hospitalization, occurrence of a fall, change of place of life, occurance of a death in the family.The number of calls needed to reach the participants, and the reason why the questionnaire could not be administered after randomization.

Randomisation
The randomization was performed using a computer generated random list (Excel).It was a centralized and restricted randomization with an allocation ratio of 2:2 in xed block of 4 individuals.The washout period between the two modes of administration was approximatively 1 month.The random allocation branch was generated by a statistician.The nurses, who enrolled the patients, assigned each of them to one of the branches according to the random allocation and successively in the order of inclusion.

Sample size
The sample size required was determinated with the STATA software (SSQDL function).To obtain a Cohen's kappa coe cient of 0.8 between the two methods of administration of the questionnaire.By assuming a proportion of patients dependant for at least 2 items of the IADL questionnaire at 50%, with a power of 80% and a risk alpha of 0.05, the sample size required was 138 patients per branch.With an expected loss of follow-up or missing value rate of 30%, the total sampe size was estimated at 197 patients per branch.

Data collection and management
The IADL questionnaire and the additional information collected at the second interview were collected in paper form by the nurses and entered in an electronic case report form by the nurses after the interviews.The others characteristics of the patients were collected directly in the eCRF of the MEMORA study by the nurses.At the end of the study, the data of both eCRF were matched using the unique identi cation number allocated to each patient.

Statistical analysis
A ow-chart has been made to describe the selection of the population.Missing data were not replaced, nevertheless we considered that the data were analysed by intention-to-treat approach as the patients were kept in the branch to which they were originally assigned by randomization.
The characteristics of patients who had the rst visit completed were compared between those who had the second visit and those who had not using the Pearson Chi-squared test or Fisher Exact test to compare proportions or independent Student t-test to compare means.The characteristics of the nal study population were compared between the branches.
In our main analysis, the extent of agreement between the two modes of administration was measured (1) by calculating the proportion of participants who gave the same response for both modes of administration, (2) by estimating the Cohen's kappa coe cient for the total IADL score, for the 2 subscores of IADL, for each item of the IADL score (coded as binary variables), and each level of independence according to the total IADL score e.g.independent of 8 abilities vs. 0, independent for > = 7 abilities vs less, and (3) by estimating the Intraclass correlation coe cient (ICC) for the total IADL score.The proportion of patients according to the different cases was compared using the McNemar Chisquared test.The method of Bland and Altman was also used to evaluate the agreement between the two modes of administration for the total IADL score [9].These analyses were conducted in all the study population, and separately in both branches.
In an additional analysis, the associations between the patients included in the 95% limits of the Bland and Altman agreement vs. those outside, and the patient's characteristics were assessed using logistic regression models.Analyses were performed using SPSS (Statistical Package for the Social Sciences) version 19.0 for Windows (SPSS Inc., Chicago, Illinois, USA).

Characteristics of study population
Overall, 420 participants were selected and randomized (Fig. 1).Among them, the IADL could be collected at the rst visit for 365 patients.After the second visit, the IADL assessment was available for both modes of administration for 292 patients (80%).The majority of incomplete data at visit 2 was explained by unreachable participants in the branch 1 (in-person -phone) and by cancelled or postponed visits in the branch 2 (phone-in-person).Supplementary le s.Table 1 shows that patients with IADL measures with the 2 modes of administration had similar baseline pro le than those without IADL measure at the second visit in terms of age, sex, aetiology, MMSE, and IADL, except that they had a lower education level, were more frequently married or in couple, and were less often with no neurocognitive disorders.
The study population (mean age 81.5±7) was characterized by a higher proportion of female (58.9%), a higher proportion of patients with less than 12 years of education (59.3%), a higher proportion of patients maried or in couple (42.8%), a majority of patients with major NCD (39.7%), and a majority of patients with awaiting etiological diagnosis (62.3%) or with probable Alzheimer's disease (22.3%), a mean MMSE at 19.6±6.1, a mean IADL by phone at 3.6±2.2,and a mean IADL in person at 3.3±2.2(Table 1).The type of caregiver was mostly the child of the patient (60.3%).This pro le of characteristics was similar in both branches.The proportion of reported events tended to be higher in the branch 1 (In-person-Phone interview) compared to the branch 2 (Phone-In-person interview) in particular there was a higher frequency of falls (9.7%) and change of place of life (3%) between the 2 visits in branch 1 compared to branch 2 (3.2%, 0% respectively).The average number of calls to succefully reach the participants was higher in branch 1 (1.5 ±0.8) compared to branch 2 (1.1 ±0.3).

Comparability between the 2 modes of administration of the IADL
Kappa scores were 0.3 for the total IADL score, 0.5 for the 1 st sub-score 1 and 0.4 for the 2 nd sub-score, re ecting fair to moderate agreement between the 2 modes of administrations of IADL questionnaire (Table 2).The analysis by item of the IADL or according to the level of autonomy highlighted higher Kappa scores from 0.5 to 0.8, indicating moderate to substantial agreement between the 2 modes.The ICC score for total IADL scores by phone and in-person was 0.91 (95% CI of ICC score: 0.89-0.93)for the total study population, 0.91 (0.88-0.94) in branch 1 and 0.91 (0.89-0.94) in branch 2.
The rate of agreement between the two modes ranged from 75% (ability to han e nances) to 92% (ability to use phone) depending the item of the IADL and 84% (autonomy of 3 IADL vs. less) to 97% (autonomy for 8 IADL vs. 0) depending of the level of autonomy.Supplementary le s.Table 2 shows results with the same order of magnitude irrespective of whether the IADL questionnaire was administered by phone or in-person rst.
With the Bland and Altman method, mean difference between the IADL total scores by phone or in-person was 0.32 (Fig. 2).Of the 292 patients, 96.9% lied within the 95% limits of agreement [-2.06-2.70].Supplementary le s.Fig. 1 and s.Fig. 2 shows the analysis of agreement with Bland and Altman method in both branches.The mean difference between the total IADL scores by phone or in-person was 0.27 in branch 1 with 97.8% of the 134 patients lying within the limits of agreement [-2.10-2.64],and respectively 0.37 in branch 2 with 96.2% of the 158 patients within the limits of agreement [-2.03-2.76].

Factors associated with agreement between the 2 modes of administration
The study of the factors associated with the probability to lie within the limits of agreement showed that age, sex, education, marital status, cognitive status, aetiology, the MMSE, the type of relationship between the caregiver and the patient, the change of respondent between the two visits, the occurence of death among family of change of place of life between the 2 visits did not contribute in the model (Table 3).An entry in emergency department, the occurence of a hospitalization or a fall between the two visits were less likely associated with the probability to lie within the limits of agreement.When included together in the same logistic regression model, only the occurrence of fall remained less likely associated with the probability to lie within the limits of agreement (OR = 0.07 [0.02-0.27],p<0.0001).

Discussion
In this randomized crossover trial conducted in a MC, we compared two modes of administration of the French version of Lawton IADL questionnaire "in-person" and "by phone" conducted among caregivers and found that these two modes provide similar assessment of the functional level of the patients when there are no events potentially in uencing the IADL occurred between the measurements i.e. falls, entry in emergency, and hospitalization.Despite somewhat a low Kappa coe cient when comparing the total IADL score with both modes of administration, the analysis by ICC and with the Bland and Altman method showed high value of agreement with around 97% of the sample lying within the 95% limits of agreement.This paradox between the Kappa statistic and the agreement analysis has been previously described and Kappa statistic is known to be sensitive to the distribution of the distribution of the marginal totals which limits its interpretation [10].
Another interesting result of this trial is that the agreement was not in uenced by the baseline characteristics of the patients, in particular the cognitive impairment.The administration of the IADL questionnaire with proxy respondent: the caregivers instead of the patient itself, whom could experience agnonosia along with the cognitive impairment, may explain this result [11].

Strengths and weaknesses of the study
To our knowledge, this is the rst study comparing Lawton IADL administration by phone and in-person.The randomized crossover design of the study has allowed to show that the order of administration does not in uence the magnitude of agreement.The collection of data was conducted both speci cally for the trial and as part of the MEMORA cohort in order to reduce the cost of carrying out the trial (the Lawton IADL is collected routinely in-person in the MEMORA cohort), and allowing to be close to the real context which is often a limit to clinical trial.The match with the MEMORA data allowed to account for various potential confounding factors that could impact the agreement.The trial was completed by collection of events that could occur during the wash-out period.Data were collected prospectively for patients visiting the MC for the rst time which should avoid recall bias.
After selection of the patients, loss of participants occurred before each visit and the trial could nally be carried out among 69.5% of the selected participants, either mainly because the visit was cancelled or postponed in branch 1, either because it was not possible to reach the caregiver in branch 2. These missing evaluations were expected in this population, the patient having at a least memory complaint and the caregiver possibly experiencing burden making its participation di cult [12].As we anticipated this loss, the sample size was corrected a priori, nevertheless it constitutes an attrition biais.It was obviously not possible to evaluate the agreement in patients with not double measurements of IADL but we compared patients' characteristics according to completeness of IADL data at the 2 nd visit and slight differences were highlighted in terms of education, marital status and cognitive status.For recall, these factors were not associated with agreement between the 2 modes of administration of the IADL questionnaire, which make us suppose that this attrition biais could have little in uence in the validity of this analysis [13].
Comparison with the literature Our study extents results of previous researchs which have compared phone and in-person interview for the evaluation of cognitive impairment in elderly people with various scales, and which generally found that phone interview provides an adequate medium to collect data [3].We noted that prior studies often included small sample sizes, and they often used correlation to compare the different modes of administration instead of studying the degree of agreement which is a more appropriate methodology to achieve the objective.
The administration of the IADL questionnaire with caregivers instead of the patients themselves in our study is even more justi ed for patients who can experience cognitive impairment than previous studies found that answers by patients may be in uenced by their cognitive status, pharmaceutical treatments and presence of behavioral disorders [4,14,15].
The effect of falls and in a lesser extent hospitalization, and entry in emergency department on a worse degree of agreement between the two modes of administration of the IADL questionnaire was not surprising as these events can lead directly to a decrease in functional abilities or be a marker of health conditions associated with functional impairment [16].

Conclusions
Our results provide evidence that the administration of Lawton IADL questionnaire by phone is an acceptable way of assessment for patients of MC in comparison to in-person interview.However, to avoid biases of measurement between patients, the administration should be done with a proxy who knows su ciently well the patient, and the events which have occurred in a time close to the evaluation should be reported.The administration of the Lawton IADL questionnaire by phone could be implemented in clinical practice in complement to the memory visit in order to improve the completeness of health status assessments.

Abbreviations
Instrumental activities of daily living ICC Intraclass correlation coe cient MC Memory clinic MMSE Mini-mental stade examination NCD Neurocognitive disorder DeclarationsEthics approval and consent to participateThe study protocol has been approved by the regional ethics committee (Comité de protection des personnes Sud Est IV) and by the National Data Protection Commission (Commission Nationale de l'Informatique et des Libertés -CNIL) the 23rd April 2014.Information was individually provided to the patients and caregivers at inclusion.

Figures
Figures

Table 1
Characteristics of the study population in total and according to branches /+ : number of patients with autonomy in in-person mode and autonomy in phone mode.
-/-: number of patients with no autonomy in in-person mode and no autonomy in phone mode; -/+ : number of patients with no autonomy in in-person mode and autonomy in phone mode; +/-: number of patients with autonomy in in-person mode and no autonomy in phone mode;Page 16/19 +* Agreement was calculated as : (a+d)/(a+b+c+d)

Table 3
Relationship between the characteristics of the patients and the odds to be included in the 95% limits of * OR was not calculated due to cells at 0.