Final Report on the Effects of Sample Attrition on Estimates of Channeling's Impacts. B. Response Rates and Reasons for Attrition

01/13/1986

In Table II.1, we present the full sample size and percent of the full sample included in each of the analysis samples. Figures are presented separately by model and experimental status.

The results indicate that at 6 months, 88 percent of the full sample was in the Medicare sample. Loss of observations was due almost entirely to sample members' failure to complete the baseline (as we shall see later); nonverifiability of Medicare eligibility was responsible for only about one-tenth of the cases excluded from this analysis sample.

Only 73 percent of the full sample was included in the nursing home sample, with the additional loss of observation arising because of sample member death and nonresponse to the 6 month interview. The followup sample included two thirds of the full sample, while the in-community sample at six months was comprised of the 55 percent of sample members who completed the 6 month followup and were residing in the community.

The proportions of the full sample included in the analysis samples at 12 months were comparable to those at 6 months. The proportion in the followup sample was lower, as expected, given the substantial fraction of sample members who died within the 7 to 12 month period. The proportion of the full sample included in the nursing home sample actually increased between the first and second six month periods, however, because some sample members who died within the first period were excluded from the 6 month analysis (since their utilization of-nursing homes was unknown) but included in the 12-month analysis (because their utilization was known to be zero).

Two figures are presented for the in-community sample at 12 months. The first is the proportion of the full sample that was included in this sample; the second is the proportion of those alive at the beginning of the period. The latter is the relevant measure of sample inclusion in this sample, since the in-community sample is used to estimate impacts on use of formal and informal care for sample members during the time they spent in the community. Since sample members who were deceased at the beginning of the period could never be in the community, this outcome is undefined rather than just missing for this sample. The proportion of those alive at the beginning of the 7 to 12 month period who are included in this sample is by chance the same (55 percent) as the proportion of those alive at the beginning of the 1 to 6 month period (the full sample) who are included in the 6 month in-community sample.

TABLE II.1: Percent of Full Sample Included in Analysis Samples
  Basic Model Financial Control Model Full Sample
  Treatments     Controls     Total     Treatments     Controls     Total     Treatments     Controls     Total  
6 MONTH OUTCOMES
Number of Observations in Full Sample 1,779 1,345 3,124 1,923 1,279 3,202 3,702 2,624 6,326
Percent of Full Sample Included in:
   Medicare sample 90.4 82.1 86.8 93.3 81.9 88.8 91.9 82.0 87.8
   Nursing home sample 72.0 67.1 69.9 80.5 67.3 75.2 76.4 67.2 72.6
   Followup sample 66.4 62.0 64.5 73.1 59.2 67.5 69.9 60.6 66.0
   In-community sample 54.8 51.5 53.3 62.3 48.9 56.9 58.7 50.2 55.2
12 MONTH OUTCOMES
Percent of Full Sample Included In:
   Medicare sample 90.4 82.1 86.8 93.3 81.9 88.8 91.9 82.0 87.8
   Nursing home sample 76.4 69.5 73.4 82.0 68.9 76.8 79.3 69.2 75.1
   Followup sample 59.1 52.1 56.1 63.0 51.4 58.4 61.2 51.8 57.3
   In-community sample 47.1 41.0 44.5 50.7 40.7 46.7 49.0 40.9 45.6
   In-community sample as % of those alive at beginning of perioda   56.9
(1,472)
50.6
(1,091)
54.2
(2,563)
60.9
(1,600)
48.9
(1,065)
56.1
(2,665)
59.0
(3,072)
49.8
(2,156)
55.2
(5,228)
18 MONTH OUTCOMES
Number of Observations in 18-Month Cohort 922 697 1,619 926 620 1,546 1,848 1,317 3,165
Percent of Full Sample Included In:
   Medicare sample 89.3 84.9 87.4 94.1 80.8 88.8 91.7 83.0 88.1
   Nursing home sample 69.8 68.1 69.1 78.8 64.4 73.0 74.4 66.4 71.0
   Followup sample 43.8 40.3 42.3 50.9 40.2 46.6 47.4 40.2 44.4
   In-community sample 33.6 31.3 32.6 38.8 31.5 35.8 36.2 31.4 34.2
   In-community sample as % of those alive at beginning of perioda 46.5
(667)
44.9
(486)
45.8
(1,153)
53.8
(667)
42.5
(459)
49.2
(1,126)
50.1
(1,334)
43.7
(645)
47.5
(2,279)
  1. Numbers in parentheses are the number of full sample members alive at the beginning of the analysis period.

For the 18 month sample the sample retention rates are similar to the rates in other periods for the Medicare and nursing home samples but considerably lower for the followup and in-community samples because of additional deaths and the requirement that sample members complete both of the earlier interviews.

Comparing treatment and control groups we see that the differences in the proportion of observations available for analysis are substantial and fairly constant (about 10 percentage points) across all periods and analysis samples. Thus, the treatment/control difference for all samples appears to be due to the differential response rates at baseline. The difference is especially pronounced for the financial control model (about 13 percentage points, compared to 6 to 8 points for most samples in the basic model).

1. Reasons for Attrition

To obtain a somewhat more detailed picture of the reasons for sample loss and how it differs across experimental groups and models, we present in Table II.2a and Table II.2b a breakdown of the total attrition from the full sample by the reason data were unavailable, for each analysis sample. The results are discussed below.

The Medicare Sample. Most of the attrition from the Medicare sample occurred at baseline--11.4 percent of the screen sample overall were excluded from the Medicare sample because they did not complete a baseline interview, while an additional 0.8 percent were dropped because Medicare entitlement could not be verified or Medicare records were not obtained. Of those who attrited at baseline, 20 percent died before the interview could be conducted, and the rest either refused to complete the interview or could not be reached. Similar attrition rates occurred in the 18-month Medicare sample.

As noted above, the overall attrition rates were substantially higher in the control group than in the treatment group, especially in the financial contol model. These higher attrition rates among control group members are attributable to their higher baseline nonresponse rates. Breakdowns of the reasons for baseline nonresponse, presented later, show that controls were much more likely than treatments to refuse to participate in the baseline interview. This higher rate of refusal for controls is due to the incentives of the treatment group to respond (completion of the baseline was a requirement to receive channeling services) and to the fact that some control members were angry about being excluded from receiving channeling services.11

TABLE II.2a: Attrition From the 6- and 12-Month Medicare, Nursing Home, and Followup Analysis Samples as a Percent of the Full Sample
  Basic Model Financial Control Model   Total  
  Treatment     Control     Treatment     Control  
FULL SAMPLE 1779 1345 1923 1279 6326
MEDICARE SAMPLE
   Deceased prior to baseline 2.0 3.3 1.6 2.5 2.3
   Other baseline nonresponse 6.2 13.6 4.4 15.4 9.1
   Medicare coverage unknown 1.3 1.0 0.7 0.2 0.8
Total Attrition Rate 9.6 17.9 6.7 18.1 12.2
Observations Available for Analysis   1608 1104 1795 1047 5554
6 MONTH NURSING HOME SAMPLE
   Attrition from Medicare sample 9.6 17.9 6.7 18.1 12.2
   Died in period, no nursing home data 9.6 8.1 7.7 6.0 8.0
   Alive at 6 months, no data 8.8 6.8 5.1 8.5 7.2
Total Attrition Rate 28.0 32.9 19.5 32.7 27.4
Observations Available for Analysis 1281 903 1548 861 4593
12 MONTH NURSING HOME SAMPLE
   Attrition from Medicare sample 9.6 17.9 6.7 18.1 12.2
   Died in period, no nursing home data 6.6 5.8 6.7 4.8 6.1
   Alive at 12 months, no data 7.4 6.8 4.7 8.2 6.6
Total Attrition Rate 23.6 30.5 18.0 31.1 24.9
Observations Available for Analysis 1359 935 1577 881 4752
6 MONTH FOLLOWUP SAMPLE
   Attrition at baseline 8.2 16.9 6.0 17.9 11.0
   Died in period 14.4 12.7 14.4 12.3 13.6
   Other followup nonresponse 11.0 8.4 6.5 10.6 9.4
Total Attrition Rate 33.6 38.0 26.9 40.8 34.0
Observations Available for Analysis 1181 834 1405 757 4177
12 MONTH FOLLOWUP SAMPLE
   Attrition at baseline 8.2 16.9 6.0 17.9 11.0
   Died before 12 month interview 23.6 22.2 25.0 20.6 23.1
   Other followup nonresponse 9.1 8.8 6.0 10.1 8.6
Total Attrition Rate 40.9 47.9 37.0 48.6 42.7
Observations Available for Analysis 1052 701 1212 658 3623


TABLE II.2b: Attrition From the 18-Month Medicare, Nursing Home, and Followup Analysis Samples as a Percent of the Full Sample
  Basic Model Financial Control Model   Total  
  Treatment     Control     Treatment     Control 
18 MONTH COHORT OF FULL SAMPLE 922 697 926 620 3165
18 MONTH MEDICARE SAMPLE
   Deceased prior to baseline 2.4 3.0 1.1 2.3 2.1
   Other baseline nonresponse 6.6 11.0 4.1 16.5 8.8
   Medicare coverage unknown 1.7 1.0 0.8 0.5 0.7
Total Attrition Rate 10.7 15.1 5.9 19.2 11.6
Observations Available for Analysis   823 592 871 501 2787
18 MONTH NURSING HOME SAMPLE
   Attrition from Medicare sample 10.7 15.1 5.9 19.2 11.9
   Died in period, no nursing home data 6.4 6.0 7.7 4.0 6.2
   Alive at 18 months, no data 13.0 10.8 7.6 12.4 10.8
Total Attrition Rate 30.2 31.9 21.2 35.6 28.9
Observations Available for Analysis 644 475 730 399 2248
18 MONTH FOLLOWUP SAMPLE
   Deceased prior to baseline 2.4 3.0 1.1 2.3 2.1
   Other baseline nonresponse 6.6 11.0 4.1 16.5 8.8
   Deceased before scheduled 18 month interview 30.8 32.3 34.7 24.8 31.1
   Nonresponse at 6 or 12 months followup 12.4 10.2 5.6 12.7 10.0
   Other followup nonresponse 4.0 3.2 3.6 3.5 3.6
Total Attrition Rate 56.2 59.7 49.1 59.8 55.6
Observations Available for Analysis 404 281 471 249 1405

The Nursing Home Sample. Attrition rates from these samples are relatively high, at 27 percent, 25 percent, and 29 percent for the 6-, 12-, and 18-month samples, respectively. As shown in Table II.2a and Table II.2b, the set of cases omitted from a particular nursing home sample includes all of those omitted from the Medicare sample (for the reasons given above), as well as two other groups of approximately equal size. Between 6 and 8 percent of the screen sample members were omitted from the analysis samples because they died during a period, but were not Medicaid-covered throughout the period and did not have a caregiver interview. Another 6 to 11 percent, approximately, were alive throughout a given six month period, but were excluded from the analysis sample because they did not complete the followup and did not have Medicaid coverage throughout the period.

Differences in rates of attrition from the nursing home samples by treatment status and by model exhibit patterns similar to those observed in the Medicare samples. In particular: (1) attrition rates are much higher among control group members than among treatment group members in both models; (2) control group attrition rates are similar across models; and (3) treatment group attrition rates are higher in the basic sites than in the financial control sites. When categories of attrition are considered, it becomes obvious that most of the differences between treatments and controls occurred as attrition from the Medicare samples for the reasons described above. Treatment/control differences in the remaining two categories are small, and differ little across models.

The Followup Samples. As seen in Table II.2a and Table II.2b, attrition from these samples is quite high, at about 34 percent, 43 percent, and 56 percent overall for the 6-, 12-, and 18-month samples, respectively. Sample members who failed to complete followup interviews include those who were nonrespondents at baseline (as described above and therefore ineligible for followup interviews), those who died after completing the baseline interview but before the followup interview could be attempted, and others who failed to respond to the interview because they refused to complete the interview or could not be reached. For the 6-month interview, about 11 percent of the screen sample were eliminated because they did not complete a baseline, an additional 14 percent had died by the 6-month anniversary, and about 9 percent did not respond for other reasons. Death accounts for the increasing attrition rates over time--about 23 percent died before the 12-month followup and about 31 percent of the 18-month cohort were deceased before the 18 month interview could be conducted. The proportion of the sample that dropped out for reasons other than death was fairly similar over time, although a greater proportion was excluded from the 18-month sample because of failure to complete one or both of the earlier followups.

Again, the overall attrition rates are higher for control group members than the treatment group members, and these differences are greater in the financial control model than in the basic model. Differential baseline nonresponse again accounts almost entirely for the treatment/control differences in the proportion of the samples with incomplete followups.

The In-Community Samples. Finally, rates of attrition from the in-community samples are broken down by reason, and displayed in Table II.2c. In addition to attrition due to lack of a completed followup, an additional 10 to 12 percent of the full sample were lost to analysis because the sample member was in a hospital or nursing home. This proportion of the sample who responded but were not in the community on their anniversary date was very similar across models, experimental groups, and time periods, especially for the 6 and 12 month periods.

TABLE II.2c: Attrition From the 6, 12, and 18 Month in Community Samples as a Percent of the Full Sample
  Basic Model Financial Control Model   Total  
  Treatment     Control     Treatment     Control  
FULL SAMPLE 922 697 926 620 6326
6-Month In-Community Sample
   Attrition at baseline 8.2 16.9 6.0 17.9 11.0
   Died in period 14.4 12.7 14.4 12.3 13.6
   Other followup nonresponse 11.0 8.4 6.5 10.6 9.4
   Respondent not in community 11.6 10.5 10.8 10.3 10.8
Total Attrition Rate 45.2 48.5 37.7 51.1 44.8
Observations Available for Analysis   974 692 1198 625 3489
12-Month In Community Sample
   Attrition at baseline 8.2 16.9 6.0 17.9 11.0
   Died before 12 month interview 23.6 22.2 25.0 20.6 23.1
   Other followup nonresponse 9.1 8.8 6.0 10.1 8.6
   Respondent not in community 12.0 11.1 12.3 10.7 11.7
Total Attrition Rate 52.9 59.0 49.3 59.3 54.4
Observations Available for Analysis 838 552 974 521 2885
18-Month Cohort of Full Sample 922 697 926 620 3165
   Attrition at baseline 9.0 14.0 5.2 18.8 11.9
   Deceased before scheduled 18 month interview 30.8 32.3 34.7 24.8 31.1
   Nonresponse at 6 or 12 month followup 12.4 10.2 5.6 12.7 10.0
   Other followup nonresponse 4.0 3.2 3.6 3.5 3.6
   Respondent not in community 10.2 9.0 12.1 8.7 9.2
Total Attrition Rate 66.4 68.7 61.2 68.5 65.8
Observations Available for Analysis 310 218 359 195 1082

2. Reasons for Interview Nonresponse

The previous discussion indicated that nonresponse to the interviews, especially the baseline, was the primary reason for treatment/control differences in the proportion of the full sample that was available for analysis in any area. To understand these differences in nonresponse, Table II.3 disaggregates the nonresponse category by the reasons for nonresponse, again by model and by treatment status. From this table we learn that control group members are considerably more likely than treatment group members to refuse the baseline interview in both models.12 In none of the followup interviews do we find such large treatment/control differences in reasons for nonresponse as in the baseline. As expected, death accounts for most of the nonresponse at each of the followup interviews for all groups, ranging from 12.3 percent of the sample at 6 months to 34.8 percent at 18 months.13

TABLE II.3: Reasons for Incomplete Interviews at Baseline and at 6, 12, and 18 Month Followup
(Percent of Full Sample)
  Basic Case Management Model Financial Control Model
  Treatment     Control     Total     Treatment     Control     Total  
BASELINE
Completed Baseline 92.1 83.2 88.3 94.4 82.4 89.6
Not Completed Baseline Due to:
   Deceased prior to baseline 2.0 3.3 2.6 1.6 2.5 2.0
   Refusal 2.6 10.9 6.2 1.4 10.6 5.1
   Moved out of area 0.3 0.1 0.3 0.2 0.2 0.2
   Unable to locate respondent or proxy   0.2 0.4 0.3 0.2 0.9 0.4
   Othera 2.8 1.9 2.4 2.3 3.4 2.7
   Total Attrition 7.9 16.8 11.7 5.6 17.6 10.4
   Total 100.0 100.0 100.0 100.0 100.0 100.0
   Sample Size 1779 1345 3124 1923 1279 3202
6-MONTH FOLLOWUP
Completed Followup 66.4 62.0 64.5 73.1 59.2 67.5
Not Completed Baseline Due to:
   No interview attemptedc 7.9 16.8 11.7 5.6 17.6 10.4
   Deceasedb 14.4 12.6 13.6 14.5 12.3 13.6
   Refusal 5.3 4.1 4.8 2.1 5.8 3.6
   Moved out of area 1.8 1.0 1.4 1.0 1.6 1.2
   Unable to locate respondent or proxy 0.7 0.7 0.7 0.5 1.2 0.8
   Othera 3.5 2.8 3.2 3.3 2.4 2.9
   Total Attrition 33.6 38.0 35.5 26.9 40.8 32.5
   Total 100.0 100.0 100.0 100.0 100.0 100.0
   Sample Size 1779 1345 3124 1923 1279 3202
12-MONTH FOLLOWUP
Completed Followup 59.1 52.1 56.1 63.0 51.4 58.4
Not Completed Baseline Due to:
   No interview attemptedc 7.9 16.8 11.7 5.6 17.6 10.4
   Deceased 23.6 22.2 23.0 25.0 20.6 23.2
   Refusal 4.7 5.1 4.8 2.1 5.4 3.4
   Moved out of area 2.1 1.3 1.8 1.7 1.6 1.7
   Unable to locate respondent or proxy 0.9 0.9 0.9 0.3 1.0 0.6
   Othera 1.6 1.6 1.6 2.2 2.3 2.2
   Total Attrition 40.9 47.9 43.9 37.0 48.6 41.6
   Total 100.0 100.0 100.0 100.0 100.0 100.0
   Sample Size 1779 1345 3124 1923 1279 3202
18-MONTH FOLLOWUP
Completed Followup 43.8 40.3 42.3 50.9 40.2 46.6
Not Completed Baseline Due to:
   No interview attemptedc 21.4 24.2 22.6 10.8 31.5 19.1
   Deceased 30.8 32.3 31.4 34.8 24.8 30.8
   Refusal 1.2 0.4 0.9 0.6 1.6 1.0
   Moved out of area 0.4 1.1 0.7 0.4 0.2 0.3
   Unable to locate respondent or proxy 0.0 0.4 0.2 0.4 0.2 0.3
   Othera 2.4 1.1 1.9 2.1 1.6 1.9
   Total Attrition 56.2 59.7 57.7 49.1 59.8 53.4
   Total 100.0 100.0 100.0 100.0 100.0 100.0
   Sample Sized 922 697 1619 926 620 1546
NOTE: The source for this table is 6,326 completed screen interviews, and information gathered through followup interviews, client tracking, and contact sheets.
  1. “Other” includes institutions refusing access or other failure to gather information about the sample member.
  2. The “deceased” category includes only sample members for whom a followup interview was scheduled. Since some sample members for whom no interview was attempted (due to baseline nonresponse) will also be deceased at the time of the followup, the nubers in this category cannot be interpreted as death rates.
  3. Reasons for not attempting a followup interview were missing baseline interview, or, in the case of the 18 month followup sample, a missing 6 or 12 month interview.
  4. The basis of the percentages for the 18 month followup interview is the early cohort.

View full report

Preview
Download

"atritn.pdf" (pdf, 4.74Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®