Final Report on the Effects of Sample Attrition on Estimates of Channeling's Impacts. B. Impact Estimates Adjusted for Attrition

01/13/1986

The probit estimates in Table V.1 and those for the other analysis samples were used to construct for each sample member a correction term (M) specific to each of the analysis samples, as defined in Chapter III. This term, when included as an additional control variable in the outcome regression, will control for the effects of attrition on the impact estimate and the other coefficients. The set of auxiliary control variables (X1) used in the outcome equation, in some cases taken from the baseline interview and from the screen in others,26 is the set that were used in the final reports on channeling impacts and includes:

  • Site
  • Impairments on activities of daily living (ADL)
  • Incontinence
  • Medicaid coverage
  • Living arrangement/availability of informal support
  • Whether on a waiting list for a nursing home
  • Cognitive impairment
  • Interviewer-assessed unmet needs
  • Whether referred to channeling by hospital or nursing home
  • Age
  • Ethnicity
  • Marital status
  • Homeownership
  • Life satisfaction
  • Stressful life events within the past year (death of person close to respondent; change in health condition)
  • Number of physician visits during past 2 months
  • Number of hours per week visiting informal caregiver spends in residence
  • Whether formal care received
  • Number of hours per week formal caregiver spends in residence
  • Proxy or self response at baseline
  • Sex

For some of these variables, means have been imputed for missing values, whereas for variables with a substantial number of missing values a separate missing value indicator is included.

In total, the X1-vector consists of 51 separate variables, including the constant. A number of these variables are included in the set of variables used to predict attrition. Others, including informal support, homeownership, life satisfaction, stressful life events, number of physician visits, whether formal care was received, and the number of hours of formal and informal care received were obtained from the baseline and, therefore, were not available for use in predicting response. Still other variables were excluded from the list of auxiliary control variables, but were used to predict attrition (e.g., number of contacts required to complete the screen), as pointed out in the previous section. The appendix contains a comparison of the variables used in the two equations and indicates for the auxiliary control variables whether they were drawn from screen or baseline.

We examine the effects of attrition by estimating channeling impacts on a set of the key outcome measures, with and without adjustment for possible attrition bias. The key outcomes examined (and the analysis samples on which they were estimated) were:

  • Nursing home outcomes (nursing homes samples)

    • whether admitted during months 1-6, 7-13, 13-18
    • number of days in nursing homes in each period
    • nursing home expenditures in each period
  • Well-being outcomes (followup samples)

    • number of unmet needs at'6, 12, and 18 months after randomization
    • number of impairments on activities of daily living at each followup
    • whether dissatisfied with life at each followup
  • Formal and informal care (in-community samples)

    • whether received care from visiting formal caregiver during reference weeks at 6, 12, and 18 months
    • hours of formal in-home care received during reference weeks
    • number of visits from formal caregiver
    • whether received care from visiting informal caregiver during reference week at 6 and 12 months
    • hours of care received from visiting informal caregiver
    • number of visits from visiting informal caregiver

The unadjusted and adjusted impact estimates for the basic and financial control models are presented in Table V.3, Table V.4 and Table V.5. The results are summarized below.

TABLE V.3: Estimates of Channeling Impacts on Nursing Home Outcomes With and Without Correction for Effects of Attrition: 6-, 12-, and 18-Month Nursing Home Samples
  Basic Model Financial Control Model   Rhoa     Sample  
Size
  Uncorrected  
Estimate
  Corrected  
Estimate
  Uncorrected  
Estimate
  Corrected  
Estimate
Any Nursing Home Admission Last 6 Months (percent)
Months 1 to 6 -0.52
(-0.37)
-0.34
(-0.23)
-0.37
(-0.27)
0.08
(0.05)
0.07
(0.37)
4593
Months 7 to 12 -2.23
(-1.88)
-3.03*
(-2.20)
0.29
(0.25)
-1.24
(-0.70)
-0.27
(-1.17)
4752
Months 13 to 18 -0.26
(-0.13)
-0.21
(-0.10)
-0.89
(-0.43)
-0.59
(-0.21)
0.04
(0.16)
2248
Number of Nursing Home Days Last 6 Months
Months 1 to 6 -2.36
(-1.93)
-1.98
(-1.54)
-1.14
(-0.94)
-0.17
(-0.10)
0.18
(0.89)
4593
Months 7 to 12 -1.19
(-0.63)
-2.61
(-1.19)
-2.19
(-1.15)
-4.94
(-1.75)
-0.31
(-1.32)
4752
Months 13 to 18 -1.12
(-0.36)
-0.94
(-0.30)
-0.18
(-0.05)
1.05
(0.24)
0.11
(0.42)
2248
Total Nursing Home Expenditures Last 6 Monthsb
Months 1 to 6 -165*
(-2.15)
-136
(-1.67)
-8
(-0.11)
68
(0.66)
0.22
(1.11)
4593
Months 7 to 12 -58
(-0.56)
-144
(-1.20)
-103
(-0.99)
-270
(-1.74)
-0.34
(-1.46)
4752
NOTE: T-values are reported in parentheses. For the corrected estimates, these are computed from standard errors which have been adjusted for heteroskedasticity using methods developed by Heckman (1979) and Greene (1981).
  1. Rho is the estimated correlation between the disturbance terms in the impact regression (u1) and the attrition equation (u2), obtained by dividing the estimated coefficient on the attrition correction term by the estimated standard error of the disturbance term in the outcome equation. The t-value in this column is the t-value of the coefficient on the correction term in the outcome equation.
  2. Data on nursing home expenditures were not collected for months 13 to 18.

* Statistically significant at the 5 percent level for a two-tailed test.
** Statistically significant at the 1 percent level for a two-tailed test.

1. Nursing Home Outcomes

Impact estimates for nursing home admissions, days, and expenditures before adjustment for possible attrition bias provide little evidence that channeling had any such effects. From Table V.3 we see that in no time period and in neither model were estimates statistically significant, except for nursing home expenditures at 6 months in the basic model (costs reduced by an average of 165 dollars per treatment group member by channeling). Adding the attrition correction term did little to change the overall interpretation of the results. The estimated correlation between unobserved factors affecting attrition and nursing home outcomes was generally small, sometimes positive and sometimes negative, and in all cases statistically insignificant, implying that there was no attrition bias. This finding is also reflected in the general similarity of the impact estimates before and after the attrition correction. There are two instances where the statistical significance of the estimates changes after the attrition correction, both occurring in the basic model. The estimated impact on nursing home admissions at 12 months goes from -2.2 percentage points before correction to -3.0 after correction. The t-statistic of the former is slightly below the critical value for a 5 percent level test while the t-statistic for the latter is slightly above the critical value. However, the point estimates are quite similar. The other instance of a change in significance after controlling for potential bias is similar but reversed: the estimated impact on expenditures at 6 months went from a significant effect of minus 165 dollars to an insignificant estimate of minus 136 dollars.

TABLE V.4: Estimates of Channeling Impacts on Well-Being Outcomes With and Without Correction for Effects of Attrition: 6-, 12 and 18-Month Followup Samples
(t-values in parentheses)
  Basic Model Financial Control Model   Rhoa     Sample  
Size
  Uncorrected  
Estimate
  Corrected  
Estimate
  Uncorrected  
Estimate
  Corrected  
Estimate
Number of Unmet Needs
6 months -0.17*
(-1.96)
-0.19*
(-1.99)
-0.25**
(-2.83)
-0.31*
(-2.01)
-0.16
(-0.45)
4075
12 months -0.31**
(-3.52)
-0.38**
(-3.56)
-0.31**
(-3.52)
-0.43**
(-3.21)
-0.36
(-1.20)
3532
18 months -0.11
(-0.82)
-0.12
(-0.81)
-0.08
(-0.55)
-0.09
(-0.46)
-0.03
(0.07)
1377
Number of Impairments on Activities of Daily Living
6 months 0.04
(0.66)
0.08
(1.10)
0.22**
(3.30)
0.34**
(2.85)
0.38
(1.22)
4094
12 months 0.06
(0.76)
1.16
(1.73)
0.21**
(2.90)
0.39**
(3.37)
0.59*
(2.05)
3539
18 months -0.08
(-0.66)
-0.02
(-0.16)
0.04
(0.35)
0.20
(1.17)
0.47
(1.35)
1381
Global Life Satisfaction (percent dissatisfied)
6 months -5.4*
(-2.49)
-5.8*
(-2.43)
-5.7**
(-2.61)
-7.0
(-1.83)
-0.13
(-0.41)
4022
12 months -2.2
(-0.94)
-3.5
(-1.22)
-5.0*
(-2.07)
-7.1*
(-1.97)
-0.24
(-0.79)
3441
18 months -1.2
(-0.31)
-0.3
(-0.08)
-2.6
(-0.66)
-0.3
(-0.05)
0.24
(0.65)
1325
NOTE: T-values are reported in parentheses. For the corrected estimates, (1) these are computed from standard errors which have been adjusted for heteroskedasticity using methods developed by Heckman (1979) and Greene (1981). For the corrected estimates (2), these are simply the unadjusted t-statistic for the treatment status coefficient and are likely to be close to those adjusted for heteroskedasticity.
  1. Rho is the estimated correlation between the disturbance terms in the impact regression (u1) and the attrition equation (u2), obtained by dividing the estimated coefficient on the attrition correction term by the estimated standard error of the disturbance term in the outcome equation. The t-value in this column is the t-value of the coefficient on the correction term in the outcome equation.

* Statistically significant at the 5 percent level for a two-tailed test.
** Statistically significant at the 1 percent level for a two-tailed test.

These differences are not compelling evidence of attrition bias. In addition to the fact that most of the estimated correlations were low and the estimated changes due to controlling for attrition were small, the two cases where significance levels did change were in different time periods and had estimated correlations of opposite signs. It seems unlikely that if attrition bias were present, it would be positive for one of these variables and negative for a related outcome, or positive in one period and negative in the next. These small changes suggest that bias in estimates of channelings nursing home impacts is unlikely. The conclusion that channeling had little impact on nursing home use in basic sites and none in financial control sites is unchanged when the potential effects of attrition are considered.

2. Well-Being Outcomes

Estimated impacts on well-being, contained in Table V.4, were also relatively unaffected by attrition. Although the estimate of rho in the ADL equations is positive and large in all 3 periods and statistically significant in one of them, the conclusion that channeling led to higher reported impairment on ADL in the financial control sites but not in basic sites is unchanged by the attrition correction. Estimated rhos for the unmet needs and life satisfaction outcomes are statistically insignificant in both models for all three time periods, and impact estimates exhibit only minor changes after the attrition correction term is added.

3. Formal and Informal Care Outcomes

Estimates of rho for these outcomes, given in Table V.5, again are statistically insignificant. The estimated impacts on formal care for the in-community sample are very similar before and after controlling for attrition effects. Statistically significant estimates remain significant and are approximately the same sizes. Insignificant estimates remain insignificant. Thus, despite the difference observed in Chapter IV between the full and in-community samples in estimated impacts on total reimbursements for Medicare-covered community services in the basic model at 6 months, we find no evidence of bias in overall use of formal care, for this time period and model or any other.

The results for informal care lead us to a similar conclusion--the estimated correlations between unobserved determinants of attrition and informal care outcomes are statistically insignificant. However, one substantive difference is observed in the estimated impacts on whether informal care was received from visiting caregivers. The estimate for the financial control model at 6 months is considerably smaller and statistically insignificant after correcting for attrition. Based on the unadjusted estimates, we had concluded (Christianson, forthcoming) that there was some evidence that channeling led to modest reductions in the percent of treatments receiving informal care. The attrition corrected estimates suggest that reductions may be even more modest than the unadjusted estimates show. However, for neither set of estimates are there significant reductions in the amount (hours or visits) of informal care received because of channeling. The lack of significant rhos and the lack of consistent findings across outcome measures or models that attrition corrected estimates differ markedly from uncorrected estimates on t his sample lead us to conclude that estimates of channeling impacts on informal care are not distorted by attrition.


TABLE V.5: Estimates of Channeling Impacts on Formal and Informal Care Use, With and Without Corrections for Attrition Bias: 6- and 12-Month In-Community Samples
  Basic Model Financial Control Model   Rhoa  
  Uncorrected  
Estimate
  Corrected  
Estimate
  Uncorrected  
Estimate
  Corrected  
Estimate
FORMAL CARE
Whether Received in-Home Care from Visiting Formal Caregiver During Reference Week (percent)
6 months after randomization 10.7**
(5.15)
9.9**
(4.57)
22.8**
(10.84)
19.8**
(6.93)
-0.34
(-1.51)
12 months after randomization 10.0**
(4.20)
11.3**
(4.24)
20.1**
(8.48)
22.1**
(7.36)
0.25
(1.06)
Total Hours of Visits by Visiting Formal Caregivers
6 months after randomization 0.82
(0.99)
0.95
(1.11)
7.40**
(8.91)
7.84**
(6.92)
0.13
(0.57)
12 months after randomization 1.74
(1.77)
1.94
(1.77)
6.35**
(6.48)
6.65**
(5.38)
0.10
(0.41)
Number of Visits by Visiting Formal Caregivers
6 months after randomization 0.48**
(3.10)
0.52**
(3.22)
2.15**
(13.75)
2.28**
(10.68)
0.20
(0.88)
12 months after randomization 0.55**
(3.01)
0.71**
(3.47)
2.12**
(11.56)
2.37**
(10.22)
0.40
(1.74)
INFORMAL CARE
Whether Received in-Home Care from Visiting Informal Caregiver During Reference Week
6 months after randomization -2.2
(-0.90)
-1.7
(0.69)
-4.8*
(1.97)
-3.2
(0.96)
0.16
(0.71)
12 months after randomization -0.7
(-0.27)
1.4
(0.48)
-3.9
(-1.46)
-0.5
(-0.14)
0.38
(1.67)
Total Hours of Visits by Visiting Informal Caregivers
6 months after randomization -1.11
(-1.04)
-1.36
(-1.23)
-0.79
(-0.75)
-1.65
(-1.14)
-0.20
(-0.87)
12 months after randomization 0.19
(0.18)
0.56
(0.47)
-0.11
(-0.10)
0.47
(0.35)
0.17
(0.70)
Number of Visits by Visiting Informal Caregivers
6 months after randomization -0.20
(-0.63)
0.05
(0.15)
-0.21
(-0.65)
0.31
(0.72)
0.39
(1.76)
12 months after randomization 0.15
(0.49)
0.33
(0.98)
-0.47
(1.56)
-0.19
(0.49)
0.28
(1.22)
NOTE: See Table V.4 for notes.

View full report

Preview
Download

"atritn.pdf" (pdf, 4.74Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®