Methodological Issues in the Evaluation of the National Long Term Care Demonstration. A. Random Assignment

04/01/1986

The single most important feature of the evaluation was the use of an experimental design. Persons referred or otherwise applying to channeling were given a screening interview (usually by phone) to determine their eligibility for the program. Applicants had to be at least age 65, have at least a moderate level of disability in performing certain activities, and have needs for two or more services that were expected to remain unmet for the next six months.1 Eligible individuals were then randomly assigned by survey research staff in Princeton to either the treatment group, which was offered the opportunity to participate in channeling, or to the control group, which was denied that opportunity for the course of the demonstration.2 Both groups were then given a baseline interview, and followups were attempted at six, twelve, and (for half the sample) 18 months thereafter to gather data on outcomes that channeling was intended to affect.

TABLE II.1. Channeling Sites and Sample Sizes
Site   Treatment  
Group
  Control  
Group
Full
  Sample  
Basic Case Management Model
   Baltimore 471 271 688
   Eastern Kentucky 246 242 488
   Houston 401 273 674
   Middlesex County   451 299 750
   Southern Maine 264 260 524
   Total 1,779 1,345 3,124
Financial Control Model
   Cleveland 388 191 579
   Greater Lynn 309 308 617
   Miami 450 297 747
   Philadelphia 581 288 869
   Rensselaer County   195 195 390
   Total 1,923 1,279 3,202
All Sites 3,702 2,624 6,326
NOTE: These sample sizes are the numbers of research sample members with completed screening interviews that were available for analysis. A total of 6,341 individuals completed screens and were randomly assigned to the treatment or control groups, but 14 screen interviews were lost in the mail and one sample member’s treatment status was misrecorded. These 15 cases were excluded from all analyses.

Given the random assignment, the control group should closely resemble the treatment group in the aggregate on both observable and unobservable characteristics. Thus, it should provide as good an indication as possible of what would have happened during the followup period to treatment group members in that site in the absence of the demonstration. The impacts of channeling are estimated by comparing the post-randomization experience of the treatment and control groups, using the estimation procedures described in Chapter III. The random assignment ensures that estimated impacts will be unbiased.

It should be kept in mind that not all members of the treatment group actually participated in channeling (some refused, some died, other were terminated after being permanently institutionalized). Furthermore, not all treatment group members who did participate remained in the program for the duration of the analysis period. However, to ensure unbiasedness of estimates it was essential that the full treatment group (i.e., everyone offered the opportunity to participate) be compared to the full control group. While channeling obviously has no impact on treatment group members who do not participate, deleting such observations would destroy the equivalence of the two groups being compared if, as one expects, treatment group members who drop out differ from those who participate.3 Without this equivalence, comparison of the two groups would no longer yield unbiased estimates of channeling impacts. Thus, treatment group members was ignored, and the treatment/control comparisons yield unbiased estimates of the effects of having the opportunity to participate in channeling.

View full report

Preview
Download

"methodes.pdf" (pdf, 2.16Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®