We found little evidence to suggest that Medicaid-covered CSHCN attributed to NCQA PCMH-recognized providers had utilization patterns consistently suggestive of higher quality care in Louisiana or Texas. In Louisiana, there were a few statistically significant differences between treatment and comparison groups in "late recognition" non-matched comparison group analyses. In matched comparison group analyses in Louisiana, findings typically suggested worse outcomes for the treatment group. In Texas, we found evidence of better outcomes related to well-child visits and follow-up after ED visits for treatment group children when compared to the "late recognition" comparison group, but comparisons of the treatment group to the matched comparison group showed no treatment-comparison differences for any outcomes. Findings in New Hampshire appeared more favorable for CSHCN attributed to NCQA-recognized providers for several measures. In "late recognition" and matched analyses, treatment group children were more likely to have follow-up after ED visits. Treatment group CSHCN were significantly more likely to receive any well-child visits in matched analyses; the magnitude and direction of the well-child visit odds ratio was similar, although only borderline-significant in "late recognition" analyses. In both sets of analyses, the odds ratios for ED visits and potentially avoidable ED visits for treatment group were less than one in all base case and sensitivity analyses, although not statistically significant. However, treatment children in New Hampshire were more likely to have any inpatient admissions in "late recognition" analyses.
The lack of consistent positive findings across all three states -- and especially the few significant adverse findings in Louisiana -- was unexpected, based on literature that suggests positive impacts of medical homes for pediatric patients in general (Aysola, 2012; Arauz Boudreau, 2012; Romaire, 2012a) and CSHCN specifically (Homer, 2008; Arauz Boudreau, 2012; Hamilton, 2012). Similar to our findings, one recent study found few meaningful differences in health services utilization when comparing CSHCN with and without a medical home (Romaire, 2012b). However, these studies measure the medical home based on parent-reported survey measures, which differs fundamentally from the NCQA practice recognition measure used in this study. Little is known about the relationship between parent-reported survey measures of the medical home and practice-based PCMH-recognition. Given that practice-based PCMH-recognition, especially the NCQA program, is currently the predominant approach used in programs and policies promoting the medical home model, it is important to begin linking such recognition to objective outcome measures.
There may be multiple explanations for our findings. In New Orleans, efforts to rebuild the primary care infrastructure after Hurricane Katrina included financial incentives for practices to transform into PCMHs and bonus payments for obtaining NCQA PCMH-recognition. However, a recent study found that practice-reported performance on various medical home processes declined as grant funding dried up in 2010 (Rittenhouse et al., 2012). The decline in performance among NCQA-recognized safety-net providers in New Orleans could explain our negative findings in Louisiana. In contrast, approximately two-thirds of treatment group providers in New Hampshire were affiliated with Dartmouth-Hitchcock medical center, a major medical center that has resources to invest in its primary care clinics. Primary care providers affiliated or owned by major medical centers may also have more opportunity to learn from each other through system-wide quality-improvement efforts. The affiliation with a major medical center may also explain the increased likelihood of hospitalization among treatment children in New Hampshire.
Another potential explanation for our lack of consistent findings across states is that NCQA-recognized providers might implement practice services that are more likely to be effective for the commercially-insured or relatively healthy pediatric patients that make up the majority of children in most primary care practices, and that similar analyses on commercially-insured populations in all three states might find consistently positive results. This phenomenon would suggest providers may need to improve activities surrounding patient-engagement and activation among Medicaid CSHCN populations and better understand family-related barriers to improved health care utilization and outcomes (Zickafoose, 2011; Coker, 2009). A related explanation for more positive findings in New Hampshire may be a function of the relative homogeneity of the population in terms of race and ethnicity, whereby NCQA-recognized providers are not hampered by lack of cultural competency that might be present in practices that serve more heterogeneous populations (Betancourt et al., 2003; Brach & Fraserirector, 2000). It also is possible that practices seeking NCQA-recognition do so because they are aware of deficiencies in their internal practices or especially challenging patient populations that lead to poor outcomes and can use the recognition process as an external impetus to improve the quality of care delivered to their patients. Our analyses also relied on 2010 data, and providers in our treatment group included those obtaining recognition during 2010. It may be that NCQA-recognized providers need more time to demonstrate improvements. However, a companion study of providers at NCQA-recognized practices found recognition was formal acknowledgement of most processes that were already in place (Petersen et al., 2013).
Lastly, it is possible that NCQA PCMH-recognition is an inadequate indicator for primary care practices with the services and processes that might be effective in improving care, as measured by the outcomes used in this study, for CSHCN in Medicaid programs. The companion study of NCQA-recognized providers noted that CSHCN are a diverse population with varying needs that may not conform to standard protocols (Petersen et al., 2013). Another qualitative study of an early NCQA-recognized PCMH practice found that transformation into a patient-centered practice and NCQA PCMH-recognition were distinct concepts (Dohan et al., 2013).
There are some important limitations to our study. NCQA-recognition is based on a wide range of practice services and processes, and we had no data on the relative strengths and weaknesses of the providers in NCQA-recognized practices on the specific NCQA dimensions. For example, children attributed to providers in recognized practices that scored highly on after-hours access might have different health care utilization patterns compared to children attributed to providers whose medical homes focused more on implementation of electronic health records. However, most providers in NCQA-recognized practices in Louisiana had Level 1 recognition compared to providers in New Hampshire and Texas who primarily had Level 3 recognition, yet we better outcomes in New Hampshire, but not Texas.
We measured only a few claims-based outcomes related to PCMHs. It is possible that NCQA-recognized providers score higher on other outcomes measures, including those related to patient satisfaction and experience and other measures of clinical quality of care. In both "late recognition" and matched comparison group analyses in Louisiana and Texas, it is possible that there are unobserved differences, or selection bias, between treatment and comparison groups that drive our results. For example, we had no reliable data on race and ethnicity and previous studies show racial differences in ED use and other services (Flores, 2010; Raphael, 2001; Stewart et al., 2010). The relatively small number of providers in NCQA-recognized practices in the states reduced our power to detect effects, particularly in Texas. It also is unclear how generalizable these findings are to other states. Additional studies evaluating the association between NCQA-recognition and outcomes are warranted.
This study used both program eligibility criteria and the CDPS to identify CSHCN. Studies that used other methods might have different results. The CDPS casts a wide net of conditions, but may include children who might not be classified as having special health care needs with other methods. This may also explain the higher rate of CSHCN in our study compared to recent survey-based estimates (Bethell et al., 2008). For example, it may include children who receive a rule-out diagnosis. Finally, we wanted to evaluate the association between NCQA-recognition and outcomes among the subset of CSHCN with disabilities (Davis & Brosco, 2007), but these analyses would likely be underpowered.
This is the first study we are aware of to evaluate the association between NCQA-recognition and health care utilization among Medicaid-covered CSHCN. Within-state findings were generally robust to the use of two types of comparison groups and multiple sensitivity analyses. Our findings suggest that NCQA-recognition does not uniformly indicate higher quality for CSHCN covered by Medicaid. It is likely that other state-specific and provider-specific factors, such as payment rates, quality-improvement activities, and the socioeconomic composition of practice panels, are important determinants of quality, as measured in this study.