Final Report on the Effects of Sample Attrition on Estimates of Channeling's Impacts. NOTES

  1. "Attrition" in this report is defined as the loss of sample members from the analysis sample during the demonstration evaluation. This definition contrasts with that of “channeling dropouts,” which refers to treatment group members who do not participate in channeling (i.e., those who decline, those determined at baseline to be ineligible, and those terminated from the demonstration).

  2. See Carcagno et al. (forthcoming) for a more complete discussion of the structure and organization of the channeling demonstration.

  3. See Phillips et al. (forthcoming) for complete documentation of interview data collection procedures.

  4. In addition to these surveys of sample members, there were also surveys of the primary caregivers of a subset of the sample members. Data from these surveys are used primarily in the evaluation of the effects of channeling on caregivers and therefore are of relatively minor importance for this study. The effects of attrition on estimates of caregiver impacts is examined in Christianson (forthcoming).

  5. These 14 cases were omitted from all subsequent analyses.

  6. See Brown and Mossell (1984) for an assessment of how this difference affected the comparability of the baseline data for the two groups.

  7. See Phillips et al. (1985) for a discussion of the 18-month cohort and interview.

  8. For a random 20 percent subsample of the research sample, records were also collected from other types of service providers (e.g., home health agencies) that were named in followup interviews by sample members. See Phillips et al. (forthcoming) for a detailed description of the provider records data.

  9. For some analyses, sample members who were in a hospital or nursing home or were decreased at the 6, 12, and 18 month “anniversary” date were included since their receipt of formal and informal care during the reference week was known to be zero. Because there were no channeling impacts on hospitalization, institutionalization, or mortality, exclusion of these cases does not affect the conclusions of the analysis. The use of in-community sample produces more meaningful estimates of service usage.

  10. This is not a strict hierarchy because a few members of the followup sample may have unknown Medicare status and therefore are excluded from the Medicare and nursing home samples. There are very few such cases.

  11. See Phillips et al. (forthcoming) for tabulation of the reasons for refusing the baseline interview.

  12. Among the 11 percent of controls who refuse to respond at baseline, a substantial fraction (24 percent in basic sites, 34 percent in financial control sites) do so because they were upset at being randomly assigned to the control group. For the remainder of the group refusing to complete the baseline, “too much bother” was the most frequent reason given for refusal. See Phillips et al. (forthcoming) for more information on reasons for refusals.

  13. In the financial control model, the treatment group has a somewhat higher percentage noncomplete due to death than does the control group at 12 and 18 months. However, this is due solely to the fact that interviews are attempted for a much higher proportion of treatment groups because of higher response rates at the baseline. Among sample members who completed the baseline, the percent deceased is very similar for treatment and control groups (28.4 and 28.6 percent, respectively, at 12 months and 38.2 and 40.6 percent at 18 months). It is important to bear in mind that because a substantial fraction of the deceased are included in the “no interview attempted” category, treatment/control differences in percent noncomplete due to death is not interpretable as an impact of channeling on mortality.

  14. The regression equation included a constant term, 2 binary variables for treatment status (the first equal to one for treatments in the basic model, the second equal to one for treatments in the financial control model), and 9 binary site variables. Coefficients on the two treatment status variables are the treatment/control differences for the respective models, controlling for the unequal distribution of the two groups across sites. The estimates can be shown to be exactly equal to weighted averages of the treatment/control differences at the 5 sites implementing each model.

  15. The number of responders in nursing homes is equal to the proportion of the full sample (of treatments or of controls) in nursing homes times the response rate for this group of individuals times the full sample size (N).

  16. The assumptions necessary for unbiasedness are that the disturbance term u1 have an expected value of zero conditional on the values of the regressions Z, and be uncorrelated with Z.

  17. Although the normality assumption (or any other distributional assumption) imposed on u2 is arbitrary, Amemiya (1981) offers a justification for the normal distribution, based on an argument that many unknown and additive factors determine whether the threshold for responding to an interview is exceeded.

  18. Throughout this discussion, X1 is treated as being fixed. The same results can be obtained for random X1 variables by making all expectations conditional upon X1.

  19. The probit model (Finney, 1964) is used to predict a binary response (Y2 = 1 or 0) as a function of explanatory variables X2:

    Prob(Y2 = 1) = F(X2b2),

    where F is the cumulative distribution function of the standard normal distribution.

  20. The standard errors from the least squares regression with the correction term are not correct due to heteroskedasticity introduced by the M-term. We have corrected the standard errors using methods based on Heckman (1979) and Greene (1981).

  21. It can easily be shown that evaluating the expression in equation (9) yields estimates of the bias that are identical to those obtained by computing the difference between the coefficients obtained from the adjusted and unadjusted regressions.

  22. The auxiliary regression coefficient on a variable in Z obtained from the regression of M on Z will tend to have a sign which is opposite to the expected sign of the correlation between that variable and the likelihood that the sample member is available for analysis. Since treatment group members are more likely to respond, the latter correlation will be positive, and the auxiliary regression coefficient will be negative.

  23. If only Medicare expenditures for these services were used, however, it would be more likely that impacts would differ across samples since most Medicare claims for treatments in financial control sites were made by sample members who dropped out of channeling, and so attrited from the various samples. We therefore would suspect that attrition may affect Medicare home health outcomes in financial control sites, and so focus instead on the combined Medicare plus FCS variables. Similar combined expenditures were used to estimate total expenditures for community services as reported in the formal community services report (Corson et al., 1985).

  24. The actual impact of some variable Xi on the probability of response, obtained by taking the derivative of the expression for this probability with respect to Xi, is f(Xb)*bi, where f(Xb) is the standard normal density evaluated at the point Xb and bi is the probit coefficient on Xi. However, since this expression depends on the values chosen for all of the variables in X, a sensible choice for the value of Xb at which to evaluate this derivative is that value for which the predicted probability of response equals the observed response rate for the sample. For response rates ranging between .30 and .70, f(Xb) evaluated at this point will be approximately .40.

  25. The estimates of these probit models of attrition are presented later in this chapter (Section C).

  26. Baseline data were collected by channeling staff for the treatment group and by research interviewers for the control group. Due to concern that this could lead to noncomparable measurement of the baseline data for the two groups, which could in turn lead to biased estimates of channeling impacts, we have included only baseline variables for which we have no evidence of differential measurement. See Brown and Mossel (1984) for a discussion of this issue and assessment of data comparability.

  27. The Chi-square statistic tests whether all of the coefficients in the model (except the constant) variables can be expected to yield significantly better predictions of whether individuals are likely to be in the analysis sample than is obtainable without any such data.

  28. This additional analysis was conducted for nursing home outcomes because of the central importance of these outcomes to the overall goals of the demonstration, and because it was felt that sample members who die within a period, who are often lost to the nursing home analysis, may have been above average users of nursing homes.

View full report


"atritn.pdf" (pdf, 4.74Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®