Final Report on the Effects of Sample Attrition on Estimates of Channeling's Impacts. A. How Medicare Data Were Used to Examine Attrition Bias


As discussed in the last section, the analysis samples used to estimate channeling's impacts on various outcomes were restricted to those sample members for whom the necessary data were available. These restrictions in data availability were due to nonresponse at the baseline or followup interviews, lack of information about Medicare or Medicaid eligibility, and in the cases of the nursing home samples and followup samples, lack of data due to death. Although various sources of data are available for only certain subsamples of the full screen sample for these reasons, Medicare data were obtained for about 97.5 percent of the full sample. (Medicare eligibility was indeterminate or inadvertent errors were made in claims requests for 2.5 percent of the screen sample members.) Thus, we are able to use these Medicare data to compare estimates of channeling's impacts on Medicare-covered services obtained on the full sample with impacts on these outcomes estimated on the more restrictived analysis samples. Major differences in impact estimates between the full sample and the analysis samples on these variables would suggest that impact estimates for other outcome variables that are obtained on these analysis samples also are likely to be biased because of attrition. On the other hand, if estimates of impacts on these Medicare-covered outcomes were similar for the full and analysis samples, we would be more confident (though not certain) that there is no bias due to attrition in estimates of impacts on other outcomes (specifically, those that can be estimated only on the various analysis samples).

In this analysis we compare impact estimates between the screen sample and the analysis samples on 11 outcome variables. The following 6 variables were constructed solely from Medicare claims data:

  • Number of days spent in hospital under Medicare
  • Number of days spent in nursing home under Medicare
  • Hospital expenditures reimbursed by Medicare
  • Nursing home expenditures reimbursed by Medicare
  • Physician Medicare reimbursements
  • Medicare reimbursements for other medical services (e.g., outpatient services, lab tests, x-rays)

In addition, 5 other variables were examined that required adding data from the Financial Control System (FCS) on use of certain services to that reflected in the Medicare data. The FCS data contain claims for services used by treatment group members in the financial control model, and are necessary for this analysis because, due to the pooling of Medicare and channeling funds for the provision of certain services covered by Medicare, Medicare claims were not filed for these services when arranged for by channeling in financial control sites. These additional variables are:

  • Number of skilled nursing visits under Medicare or channeling
  • Number of home health aide visits under Medicare or channeling
  • Skilled nursing reimbursements under Medicare or channeling
  • Home health aide reimbursements under Medicare or channeling
  • Total reimbursements by Medicare or channeling for skilled nursing, therapy, home health aides, special equipment and supplies

Because of the need to use FCS data, these variables are defined slightly differently for treatments in financial control sites than for other sample members. However, there is no reason to expect that inclusion of the FCS data should affect estimated impacts for the analysis samples differently from the way it affects estimates for the full sample.23 All outcomes are defined for months 1 to 6, 7 to 12, and, for the 18-month cohort, months 13 to 18.

Because they are restricted to what is available from Medicare claims, these 11 variables each capture only a portion of the total use of that service, which is the primary focus of the evaluation. Nonetheless, since the variables are available for attriters as well as nonattriters, they can provide valuable clues as to whether attrition is likely to be a problem for the analysis of total use of these services, and for other outcomes as well. This is especially true for hospital use, nearly all of which is covered by Medicare for the sample members in the channeling evaluation. We estimate that over 98 percent of total hospital days used by the sample are reflected in the Medicare claims. Thus, the comparisons presented below between impacts on Medicare-covered hospital days and expenditures estimated on the full sample to those estimated on the Medicare samples provide very good evidence of whether impacts on total hospital use are biased by restricting the analysis to the Medicare sample.

For other outcomes, the Medicare/FCS data offer less complete coverage of total use. The nursing home use and expenditures covered by Medicare is only a small fraction of total nursing home use and the variables on use of skilled nursing and home health aides do not cover all formal services used by sample members. Furthermore, there are no variables obtainable from Medicare records (and therefore available for the full sample) that pertain to sample member well being, receipt of case management, and receipt of informal services. For these outcomes, as well as for others obtained from the followup interviews, we will also rely on the statistical procedures identified in Chapter III to determine whether attrition bias exists. Similar impacts for the full and analysis samples on the Medicare outcomes presented here, however, would provide some evidence that attrition bias is not a problem in these samples for any outcomes.

Since not all members of the full screen sample completed a baseline interview, the standard control variables used in all the final impact analyses have been replaced by a parallel set of variables based on the screen interview only. These variables, essentially the ones examined in Table II.4a and Table II.4b are described in more detail in Chapter V of this report.

View full report


"atritn.pdf" (pdf, 4.74Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®