We employed a difference-in-difference approach, which controls for preexisting differences between treatment and comparison areas, to estimate the impact of the voucher program on the probability that eligible individuals would make transitions from nursing facilities to the community. The probability of transition was modeled as a function of whether an individual resided in a site with a voucher program (Ti), whether the observation was from the time period after the voucher program was implemented (Pi), whether the individual resided in a treatment area during the implementation period (TiPi), and beneficiary-level control variables (Xi),34 using a logit model:
where the coefficient of the interaction between the voucher site indicator and the implementation period indicator, β3, captured the effect of the voucher program. The impact analysis was conducted separately for each treatment area (that is, Baltimore, Cincinnati, and Washington) and for the pooled samples (all sites and Baltimore combined with Cincinnati). We produced pooled estimates for Baltimore and Cincinnati without Washington because their baseline characteristics--particularly their pre-period transition rates--were much more similar to each other than to Washington's baseline characteristics. Weights developed via propensity score matching were included in all models.
The estimate of the coefficient, β3, is an estimate of the impact of the availability of vouchers on the unobserved index variable, yi, and is difficult to interpret directly. Hence, we converted each estimate to an estimate of the marginal impact of the availability of the vouchers on transitions in the relevant treatment sample.35 The results were estimates of the impact of the vouchers on the percentage of those in the analytical treatment sample who made transitions during the sample period. For each estimate, we tested the hypothesis of zero impact on this percentage, and we also tested the hypothesis that the true impact was equal to the maximum possible effect--that is, that 100 percent of those who used vouchers would not have made transitions without the benefit of the vouchers. The estimated marginal impact and test results from the logit model appear in Table III.7.36
|TABLE III.7. Estimated Impact of NED2 Vouchers on Community Transition Rates|
|Treatment Area|| Estimated Impact
| Maximum Potential Impact
| p-Value for Test
That Impact is Zero
| p-Value for Test That Impact
Equals the Maximum Potential Impact
|Pooled sample (all sites)||3.68||9.29||0.286||0.000*|
|Pooled Baltimore & Cincinnati||8.70||10.60||0.060*||0.681|
|SOURCE: HUD administrative data linked to MDS. Sample sizes are: pooled sample of all sites, 3,283; pooled sample of Baltimore and Cincinnati, 1,731; Baltimore, 963; Cincinnati, 768; Washington, 1,552.
NOTES: Effects represent the percentage point impact of voucher availability on rates of community transitions. Control variables vary across sites based on sample exclusions and may include gender, age, age squared, marital status, race, sensory ability (vision, hearing), functional characteristics (up to four different levels of functioning in the following categories: ability to make self understood, bed mobility, transfer ability, walk in room, walk in corridor, locomotion on unit, locomotion off unit, dressing, eating, toilet use, personal hygiene, and bathing), presence of health conditions (heart-related diseases, infection, metabolic conditions, musculoskeletal conditions, cognitive conditions, diseases affecting motor skills, neurological conditions, and psychological conditions), and institutional characteristics (days in nursing facility, days in nursing facility squared, Medicare and Medicaid coverage, type of residence before current stay, and intent to make transition), and binary site indicators. All models include an indicator for treatment area and an indicator for intervention period.
*Indicates estimated impact is statistically different from zero (column 3) or the maximum potential impact (column 4) at the 10% level.
Findings. The results indicated that NED2 vouchers had a positive, statistically significant impact on community transition rates in Cincinnati and in the pooled Baltimore and Cincinnati samples, but not in Baltimore by itself, in Washington, or in the full pooled sample. The estimated impact for the pooled Baltimore and Cincinnati samples (8.7 percentage points) was not significantly different than the maximum possible impact (10.6 percentage points), and analogous statements apply to the separate estimates for the two areas. The estimated impact for Washington was very small (less than one-half of one percentage point) and not statistically different from zero; the difference from the maximum potential impact was also not statistically significant at the 10 percent level, although it was close (the p-value was 0.106). The full pooled estimate was significantly different from the maximum potential impact, which likely reflected the implicit averaging of a very small effect in one site with much larger effects in the other two.
The estimated impacts for the pooled Baltimore and Cincinnati sites and for both sites independently are large relative to the pre-intervention period transition rates (Table III.8). The estimated impacts for these two sites represent, respectively, a 51 percent and 57 percent increase over the corresponding pre-intervention period transition rates in the treatment area. In contrast, the estimated impacts for the pooled sample of all sites and Washington are not only significantly insignificant from zero but are also small relative to baseline transition rates.
|TABLE III.8. Estimated Impact of NED2 Vouchers Relative to Baseline and Comparison Area Community Transition Rates|
|Treatment Area||Comparison Area
|Pooled sample (all sites)||46.94||50.02||48.13||54.89||3.68|
|Pooled Baltimore & Cincinnati||18.92||20.71||18.97||29.46||8.70|
|SOURCE: HUD administrative data linked to MDS.
NOTES: The first three columns include weighted averages of community transition rates. The fourth column shows the predicted value of community transitions in treatment areas during the intervention period, based on the weighted averages in the first three columns and the estimated impact in the last column. The difference-in-difference estimates represent the percentage point impact of voucher availability on rates of community transitions.
As a sensitivity test, we estimated a model with a smaller sample for Washington, based on the sample selection criteria used for Baltimore and Cincinnati. We included only individuals who were Medicaid eligible and had resided in nursing facilities for 90 or more days.37 Although the marginal impact estimate for Washington was slightly larger (2.5 percentage points), it remained statistically insignificant.
Our ability to identify impacts was limited by the low number of vouchers used during our analysis period. The likelihood of detecting impacts would be increased if we were able to extend our analysis into early 2012, when the bulk of the remaining vouchers were used. The effect of doing so would increase the maximum potential impact for each site in proportion to the increase in the number of vouchers leased. If the point estimates above represent real impacts, then we would also expect them to increase proportionately. If so, the resulting estimates for all samples would likely be more significant. It also seems likely that the estimate for the Washington sample, or for the three samples combined, would remain insignificantly different from zero.
We were not able to determine why the results for Washington were less consistent with a substantial impact than those for Baltimore and Cincinnati. The differences might, however, be attributed to important differences between Washington and the other sites. Most notably, the pre-intervention transition rates in the Washington treatment area were much higher than those in the other two areas (74 percent in Snohomish and 63 percent in Tacoma, compared with 18 percent and 24 percent, respectively, in Baltimore and Cincinnati). Hence, compared to those in Baltimore and Cincinnati, NED2 voucher users in Washington may have been much more likely to make transitions to the community even without the vouchers. Another difference was that essentially all of the voucher users in Baltimore and Cincinnati met MFP eligibility criteria, whereas many of those in Washington did not.