Final Report on the Effects of Sample Attrition on Estimates of Channeling's Impacts. C. Treatment/Control Group Differences in Characteristics in the Analysis Samples

01/13/1986

As explained in Section A, the experimental design of the evaluation ensured that, subject to chance variation, the treatment and control groups were initially made up of individuals with similar (measured and unmeasured) characteristics. In order to be able to interpret treatment/control differences in outcomes as valid estimates of channeling's impact, it is important to ensure that the initial similarity is not undone by the effects of attrition. In an earlier report (Brown and Harrigan, 1983) the similarity of the treatment and control groups at randomization was confirmed by using data collected at the screen to compare the characteristics of the two groups. That analysis is extended in this chapter by estimating treatment/control differences on screen characteristics for the samples available for analysis at 6, 12, and 18 months after randomization. Because there are so many analysis samples and because the treatment/control differences in the proportion of the sample with available data for any analysis appear to be driven by the treatment/control differences in interview nonresponse, we confine our investigation of initial differences to the followup samples.

Treatment/control differences in the following screen characteristics are presented in Table II.4a (basic sites) and Table II.4b (financial control sites):

  • Impairments on activities of daily living; incontinence
  • How referred to channeling (by hospital or nursing home, home health agency, other)
  • Ethnicity (black, hispanic, white)
  • Sex
  • Age
  • Cognitive impairment (severe, moderate, mild/none)
  • Interviewer-assessed unmet needs (low, medium, high)
  • Whether sample member has Medicaid coverage
  • Whether a proxy completed or helped complete the'screen
  • Whether regular help received with
    • meal preparation
    • housework or shopping
    • taking medicine
    • medical treatments at home
    • personal care
  • Income (0-500, 501-1000, over 1000 dollars per month)
  • Whether on waiting list or applied for nursing home
  • Number of contacts required to obtain screen interview
  • Number of missing items on the screen
  • Whether expect help will be needed to complete the baseline and followup interviews
  • Living arrangement (with child, with spouse but not with child, with other, or alone)

Some item nonresponse occurred for the screen variables. To some extent, item nonresponse was minimized, where appropriate, by imputing the baseline value of the measurement of the same characteristic. Remaining item nonresponse was dealt with by imputing the mean of the nonmissing values if only little nonresponse existed, or including a separate nonresponse category if item nonresponse exceeded 5 percent.

The numbers of primary interest in each of these tables are the treatment/control differences, estimated by regression to control for the different distribution of treatment and control groups across sites.14 The (unadjusted) treatment group means are also given as a reference point, and can be used to obtain a profile of those who remain in the samples over time.

Some differences between treatment and control groups will occur by chance; hence, we concentrate on those for which the probability that a difference of the size observed would occur by chance is less than 5 percent. In each of the tables the first column gives treatment/control differences for the full research sample, which are comparable--except for a slightly different sample definition--to those presented in Brown and Harrigan (1983).

The overall picture presented by Table II.4a and Table II.4b is that the differences between treatment and control groups are quite small, either before or after attrition, and very few are statistically significant. In the basic sites, we note significant differences in three of the 17 variables examined--ethnicity (at 6 and 12 months), number of missing items on the screen (at 6 and 12 months), and living arrangement (at 18 months only). However, the difference in missing items on the screen existed at randomization and thus is not attributable to treatment/control differences in attrition. Continence, on the other hand, was significantly different for the two groups at randomization but not at followup. In the financial control model, however, treatment/control differences in referral source (at 18 months) and income (at 6 and 12 months) widened and became statistically significant. Thus, of the 51 comparisons presented for each model (17 variables at three points in time), only 3 in each model--about what would be expected to occur by chance--were statistically significant that were not explainable by significant differences at the time of randomization. This is about the number of such differences that might be expected to occur by chance in this number of tests.

In addition to the fact that there were few instances of statistically significant differences, the types of variables for which they were found and the lack of pattern in these results increase the belief that attrition did not lead to serious differences between the two groups. The lack of difference between the two groups on ADL, unmet needs, and other indicators of impairment increases our confidence that the two groups do not differ on unobserved dimensions of health status or other .factors that are related to outcomes of interest. The two significant differences which appear only at 18 months seem to be more happenstance than systematic differences in attrition patterns.

Overall, the pattern and magnitude of treatment/control differences on screen characteristics for the followup samples at 6, 12, and 18 months lead us to believe that attrition did not result in treatment and control groups that differ substantially on observed screen characteristics. However, this does not ensure that estimates of channeling impacts are not biased by attrition. The conditions which lead to bias is the topic of the next chapter.

TABLE II.4a: Comparison of Screen Characteristics of Treatments and Controls in Basic Sties Who Completed 6, 12, and 18 Month Followup Interviews
(Percent, unless otherwise indicated)
Screen
  Characteristics  
Full Screen Sample For Sample Members Completing:
6-Month Interview 12-Month Interview 18-Month Interview
  Treatment  
Group
Mean
T/C
  Differences  
t-
  value  
  Treatment  
Group
Mean
T/C
  Differences  
t-
  value  
  Treatment  
Group
Mean
T/C
  Differences  
t-
  value  
  Treatment  
Group
Mean
T/C
  Differences  
t-
  value  
IMPAIRMENT OF ABILITY TO PERFORM ACTIVITIES OF DAILY LIVING (ADL)
Extremely severe 21.3 -1.4 (-0.92) 18.0 -2.6 (-1.46) 16.4 -1.6 (-0.85) 18.1 1.3 (0.45)
Highly severe 34.9 0.6 (0.35) 35.4 2.2 (1.03) 36.0 3.9 (1.66) 34.4 5.8 (1.57)
Moderately severe   23.3 -0.4 (-0.27) 22.9 -2.4 (-1.25) 23.4 -3.8 (-1.82) 22.5 -5.2 (-1.56)
Mild or none 20.5 1.2 (0.84) 23.7 2.8 (1.54) 24.1 1.5 (0.76) 25.0 -1.9 (-0.58)
CONTINENCE
Continent 41.3 0.1 (0.04) 43.4 -0.8 (-0.34) 44.2 -2.4 (-0.98) 46.0 -2.0 (-0.50)
Incontinent 50.5 2.1 (1.18) 50.0 1.9 (0.84) 49.5 2.9 (1.21) 47.0 2.1 (0.54)
Colostomy bag, device, need help   8.1 -2.2* (-1.98) 6.5 -1.1 (-0.91) 6.3 -0.6 (-0.43) 6.9 -0.1 (-0.06)
REFERRAL SOURCE
Hospital or nursing home 28.5 -1.3 (-0.77) 25.7 -0.4 (-0.18) 26.3 -0.4 (-0.19) 27.2 3.9 (1.16)
Home health agency 11.5 0.1 (0.07) 11.6 -0.2 (-0.11) 10.6 0.4 (0.26) 6.9 -3.0 (-1.14)
Neither 60.0 1.2 (0.67) 62.7 0.5 (0.25) 63.0 -0.0 (-0.02) 65.8 -0.9 (0.25)
ETHNICITY
Black 21.9 -1.8 (-1.35) 21.8 -3.2* (-1.97) 21.0 -4.2* (-2.38) 23.0 -4.5 (-1.60)
Hispanic 1.9 0.1 (0.11) 2.4 -0.3 (-0.32) 2.6 -0.4 (-0.45) 3.0 -0.5 (-0.32)
White 76.2 1.7 (1.22) 75.9 3.5* (1.99) 76.4 4.6* (2.43) 74.0 5.0 (1.66)
SEX
Male 28.6 -0.0 (-0.01) 25.0 -0.4 (-0.19) 23.5 0.6 (0.30) 22.0 -0.6 (-0.18)
AGE (in years) 79.1 0.1 (0.48) 78.9 0.1 (0.34) 78.9 0.3 (0.88) 78.9 0.8 (1.35)
COGNITIVE IMPAIRMENT
Severe 14.8 -0.5 (-0.35) 13.6 -1.0 (-0.63) 14.0 -0.2 (-0.09) 14.6 -1.3 (-0.46)
Moderate 27.5 1.0 (0.62) 25.8 1.1 (0.50) 24.3 0.6 (0.29) 25.2 4.3 (1.24)
Mild 49.2 0.1 (0.05) 50.8 1.0 (0.45) 51.8 1.2 (0.50) 48.8 -0.5 (-0.13)
(Missing) 8.5 -0.7 (-0.91) 9.7 -1.0 (-1.07) 10.0 -1.7 (-1.56) 11.4 -2.6 (-1.40)
INTERVIEWER ASSESSED UNMET NEEDS
Low 29.6 -2.9 (-1.80) 29.6 -1.6 (-0.78) 29.8 0.9 (0.40) 30.7 0.5 (0.13)
Medium 33.0 1.6 (0.93) 31.2 -2.3 (-1.07) 31.7 -3.2 (-1.40) 26.5 -0.9 (-0.24)
High 33.1 0.2 (0.14) 34.8 2.9 (1.44) 34.4 1.5 (0.69) 38.4 0.4 (-0.12)
(Missing) 4.4 1.1 (1.32) 4.3 0.9 (0.93) 4.1 0.9 (0.80) 4.5 0.8 (0.47)
MEDICAID INSURANCE 20.6 0.1 (0.23) 21.2 -1.6 (-0.83) 20.8 -2.1 (-1.01) 20.8 -2.0 (-0.57)
PROXY USE AT SCREEN 65.7 -0.4 (-0.23) 61.8 -0.7 (-0.33) 61.2 1.3 (0.58) 59.2 2.1 (0.57)
REGULAR HELP RECEIVED WITH:
Meal preparation 74.4 -1.6 (-1.18) 71.3 -2.9 (-1.61) 70.7 -1.9 (-0.94) 70.3 -3.4 (-1.05)
Housework, shopping 78.2 -1.0 (-0.81) 76.0 -2.6 (-1.56) 75.2 -2.2 (-1.23) 75.3 -1.7 (-0.58)
Taking medicine 56.4 -1.5 (-0.92) 53.0 -0.9 (-0.41) 52.3 0.6 (0.26) 52.1 1.2 (0.34)
Medical treatments at home   43.5 0.0 (0.00) 40.3 -0.5 (-0.23) 39.1 -1.7 (-0.70) 37.4 0.4 (0.11)
Personal care 68.9 -1.2 (-0.77) 65.2 -1.8 (-0.94) 64.3 -0.3 (-0.12) 62.7 2.3 (0.70)
INCOME
<$500/mo. 57.7 -1.3 (-0.76) 60.3 -1.6 (-0.75) 60.3 -2.7 (-1.13) 64.4 3.0 (0.82)
$500-$999/mo. 33.8 -0.0 (-0.02) 31.8 -0.0 (-0.01) 31.9 0.8 (0.34) 27.7 -2.6 (-0.74)
>$1000/mo. 8.5 1.4 (1.45) 8.0 1.7 (1.45) 7.8 1.9 (1.54) 7.9 -0.4 (-0.21)
ON WAITING LIST (or applied for) NURSING HOME 11.3 0.7 (0.66) 9.7 -0.7 (-0.58) 10.0 -0.2 (-0.15) 11.4 0.2 (0.09)
NUMBER OF CONTACTS TO OBTAIN SCREEN INTERVIEWS   2.2 0.0 (0.81) 2.1 0.0 (0.50) 2.1 0.1 (1.04) 2.2 0.1 (0.85)
NUMBER OF MISSING ITEMS ON SCREEN 0.8 -0.2* (-2.52) 0.9 -0.2* (-2.12) 0.9 -0.2* (-2.10) 1.0 -0.0 (-0.02)
NEEDED HELP TO COMPLETE BASELINE 54.1 -0.5 (-0.28) 50.9 -0.8 (-0.38) 49.4 -0.5 (-0.20) 47.4 -2.7 (-0.71)
LIVING ARRANGEMENT
With child 22.9 -1.3 (-0.88) 22.8 -0.7 (-0.40) 22.1 0.1 (0.03) 21.5 3.2 (1.03)
With spouse, not with child 28.7 2.3 (1.40) 28.5 2.7 (1.34) 27.8 4.0 (1.86) 28.7 2.7 (0.78)
With other (no spouse or child) 9.4 -0.4 (-0.41) 8.5 -1.3 (-1.04) 8.5 -1.7 (-1.27) 8.7 -4.7* (-2.14)
Alone 35.8 -0.7 (-0.42) 36.7 -1.0 (-0.46) 38.2 -3.1 (-1.28) 35.9 -4.1 (-1.07)
(Missing) 3.2 0.2 (0.26) 3.5 0.3 (0.44) 3.5 0.7 (0.78) 5.2 2.8* (2.19)
MAXIMUM SAMPLE SIZE (N=3124) (N=2015) (N=1753) (N=685)
NOTE: Treatment/control (T/C) differences were estimated controlling for site. Treatment group means are unadjusted means, provided as a reference.

* Statistically significant at the 5 percent level for a two-tailed test.
** Statistically significant at the 1 percent level for a two-tailed test.


TABLE II.4b: Comparison of Screen Characteristics of Treatments and Controls in Financial Control Sites Who Completed 6, 12, and 18 Month Followup Interviews
(Percent, unless otherwise indicated)
Screen
  Characteristics  
Full Screen Sample For Sample Members Completing:
6-Month Interview 12-Month Interview 18-Month Interview
  Treatment  
Group
Mean
T/C
  Differences  
t-
  value  
  Treatment  
Group
Mean
T/C
  Differences  
t-
  value  
  Treatment  
Group
Mean
T/C
  Differences  
t-
  value  
  Treatment  
Group
Mean
T/C
  Differences  
t-
  value  
IMPAIRMENT OF ABILITY TO PERFORM ACTIVITIES OF DAILY LIVING (ADL)
Extremely severe 26.4 -0.0 (-0.03) 23.8 1.2 (0.65) 21.8 0.6 (0.30) 20.4 -2.0 (-0.66)
Highly severe 35.2 0.7 (0.43) 35.2 1.6 (0.73) 36.1 2.5 (1.05) 36.7 5.9 (1.59)
Moderately severe   20.4 -1.2 (-0.80) 21.8 -3.4 (-1.78) 22.4 -2.3 (-1.09) 24.2 1.3 (0.38)
Mild or none 18.0 0.5 (0.35) 19.1 0.7 (0.38) 19.7 -0.7 (-0.36) 18.7 -5.2 (-1.59)
CONTINENCE
Continent 42.7 0.6 (0.31) 44.6 0.6 (0.27) 47.4 2.3 (0.92) 48.6 6.1 (1.54)
Incontinent 45.2 -0.2 (-0.14) 45.7 -1.2 (-0.53) 43.5 -2.9 (-1.20) 42.3 -6.2 (-1.57)
Colostomy bag, device, need help   12.1 -0.3 (-0.28) 9.7 0.6 (0.46) 9.2 0.7 (0.52) 9.1 0.1 (0.04)
REFERRAL SOURCE
Hospital or nursing home 32.2 -0.9 (-0.52) 28.9 1.2 (0.61) 28.4 1.6 (0.75) 28.9 6.4 (1.86)
Home health agency 21.5 -1.2 (-0.94) 21.8 0.7 (-0.40) 21.1 -1.8 (-1.00) 20.0 -6.9** (-2.59)
Neither 46.3 2.1 (1.20) 49.3 -0.6 (-0.26) 50.5 0.1 (0.06) 51.2 -0.5 (0.14)
ETHNICITY
Black 23.3 -1.0 (-0.77) 22.9 -2.3 (-1.41) 22.1 -2.4 (-1.35) 19.3 -3.7 (-1.29)
Hispanic 5.2 -0.0 (-0.08) 5.7 -1.1 (-1.31) 5.9 -0.6 (-0.69) 7.9 -0.5 (-0.29)
White 71.5 1.1 (0.76) 71.4 3.4 (1.94) 71.9 3.0 (1.59) 72.8 4.1 (1.35)
SEX
Male 29.2 1.6 (0.99) 26.5 0.4 (0.22) 25.3 1.0 (0.48) 24.6 2.0 (0.61)
AGE (in years) 80.1 0.3 (1.08) 80.0 0.5 (1.32) 79.7 0.2 (0.50) 78.9 -0.6 (-1.09)
COGNITIVE IMPAIRMENT
Severe 17.1 1.6 (1.19) 16.6 1.7 (1.04) 16.6 1.4 (0.79) 17.4 3.7 (1.28)
Moderate 36.0 -0.9 (-0.53) 35.7 0.9 (0.42) 35.4 2.9 (1.28) 33.8 1.3 (0.37)
Mild 44.5 -0.7 (-0.41) 44.8 -2.6 (-1.14) 45.0 -4.2 (-1.73) 45.0 -3.8 (-0.98)
(Missing) 2.5 0.0 (0.05) 2.9 -0.0 (-0.03) 3.1 -0.1 (-0.05) 3.8 -1.1 (-0.62)
INTERVIEWER ASSESSED UNMET NEEDS
Low 29.8 -1.3 (-0.78) 30.0 -1.9 (-0.94) 28.9 -2.5 (-1.17) 29.7 -5.1 (-1.45)
Medium 34.9 0.9 (0.52) 36.5 2.8 (1.32) 37.5 4.1 (1.74) 41.2 4.8 (1.31)
High 28.4 0.6 (0.37) 26.9 -0.3 (-0.15) 27.0 -0.5 (-0.24) 22.9 1.4 (0.40)
(Missing) 6.9 -0.2 (-0.27) 6.5 -0.6 (-0.63) 6.6 -1.0 (-0.91) 6.2 -1.1 (-0.64)
MEDICAID INSURANCE 23.4 -0.7 (-0.47) 23.9 -3.5 (-1.85) 24.6 -3.6 (-1.72) 33.3 -3.9 (1.11)
PROXY USE AT SCREEN 68.1 -0.9 (-0.53) 66.6 -0.5 (-0.22) 65.3 -1.0 (-0.91) 61.8 -1.8 (-0.48)
REGULAR HELP RECEIVED WITH:
Meal preparation 80.5 -1.9 (-1.34) 79.0 -1.7 (-0.91) 78.7 -0.1 (-0.03) 76.4 2.1 (0.64)
Housework, shopping 82.7 -1.3 (-1.02) 82.3 -0.5 (-0.28) 81.5 -0.9 (-0.49) 79.7 -1.2 (-0.43)
Taking medicine 64.9 -1.5 (-0.90) 61.7 -1.1 (-0.49) 61.2 0.7 (0.32) 57.7 1.8 (0.49)
Medical treatments at home   54.5 -1.4 (-0.81) 51.0 0.5 (0.24) 49.8 -0.6 (-0.24) 48.5 4.2 (1.10)
Personal care 77.7 -2.4 (-1.63) 76.1 -1.8 (-0.95) 75.3 -2.2 (-1.04) 72.2 -3.5 (-1.04)
INCOME
<$500/mo. 58.0 -1.5 (-0.86) 57.9 -3.4 (-1.55) 58.4 -3.9 (-1.64) 62.8 -1.6 (-0.43)
$500-$999/mo. 36.0 2.6 (1.51) 36.2 4.6* (2.17) 35.6 4.9* (2.15) 31.2 0.8 (0.22)
>$1000/mo. 6.0 -1.1 (-1.14) 5.9 -1.2 (-1.04) 5.9 -1.0 (-0.85) 5.9 0.9 (0.43)
ON WAITING LIST (or applied for) NURSING HOME 8.1 -1.2 (-1.15) 7.6 -0.5 (-0.42) 7.8 -0.3 (-0.24) 7.9 0.3 (0.14)
NUMBER OF CONTACTS TO OBTAIN SCREEN INTERVIEWS   2.4 0.0 (0.73) 2.4 0.0 (0.14) 2.4 0.1 (1.29) 2.4 0.1 (0.75)
NUMBER OF MISSING ITEMS ON SCREEN 1.3 -0.2* (-2.04) 1.3 -0.1 (-1.33) 1.4 -0.2 (-1.90) 1.3 -0.0 (-0.03)
NEEDED HELP TO COMPLETE BASELINE 56.5 -1.6 (-0.89) 56.1 -1.3 (-0.59) 54.9 -0.5 (-0.19) 54.7 4.3 (1.11)
LIVING ARRANGEMENT
With child 21.2 -0.2 (-0.13) 20.8 -1.3 (-0.72) 20.0 -1.2 (-0.62) 18.7 -1.3 (-0.42)
With spouse, not with child 29.1 0.8 (0.47) 28.0 0.2 (0.10) 28.2 2.3 (1.04) 29.7 0.1 (0.03)
With other (no spouse or child)   7.6 0.1 (0.09) 8.0 0.6 (0.49) 7.6 0.8 (0.57) 6.8 -0.9 (-0.43)
Alone 38.9 -0.9 (-0.50) 40.3 -0.1 (-0.04) 41.1 -1.9 (-0.78) 43.3 2.9 (0.75)
(Missing) 3.2 0.2 (0.36) 2.9 0.6 (0.78) 3.1 0.1 (0.09) 1.5 -0.8 (-0.60)
MAXIMUM SAMPLE SIZE (N=3202) (N=2162) (N=1870) (N=720)
NOTE: Treatment/control (T/C) differences were estimated controlling for site. Treatment group means are unadjusted means, provided as a reference.

* Statistically significant at the 5 percent level for a two-tailed test.
** Statistically significant at the 1 percent level for a two-tailed test.


View full report

Preview
Download

"atritn.pdf" (pdf, 4.74Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®