Evaluation of Parity in the Federal Employees Health Benefits (FEHB) Program: Final Report. Overview of the Evaluation Methodology

12/31/2004

The design of the evaluation was quasi-experimental. Plan data on nominal benefits (for all FEHB plans) and archival (claims) data on access, utilization, and cost (for nine selected plans) were studied before and after the implementation of parity.Changes in these measures were compared to changes in matched non-FEHB comparison group plans from the Medstat Group MarketScan® Benefit Plan Design database (Medstat).

For selected plans, the Parity Evaluation Research Team (PERT) prepared case studies based on a site visit to each selected plan. These studies were also quasi-experimental in design. They inquired retrospectively about conditions and experiences before and after the parity policy went into effect.

Including the non-FEHB comparison group plans (i.e., Medstat) allowed for assessing secular trends occurring over the same pre- and post-parity implementation period. In this way, it was possible to determine to what extent pre- to post-parity implementation changes could be attributed to the policy change versus changes in the health care system that might have occurred regardless of the parity policy.

The analysis of changes from pre-parity to post-parity relied primarily on the archival claims data and information on nominal MH/SA benefits, as these were the only data that were not reported retrospectively. The pre-parity to post-parity changes from the archival claims data were compared to changes in the matched comparison group set of claims data covering the same period. In addition, the study investigated FEHB plan benefits, policies, and procedures, including changes implemented within the first two years of parity.

Detailed data were obtained through site visits to eight health plans and more limited data were obtained for all FEHB plans (with over 500 enrollees) through a Parity Reporting Requirement (PRR) instituted by the OPM. Information on changes in FEHB health plan structures, policies, and procedures were obtained by site visits with the key plan personnel, resulting in selected plan case studies. The case studies yielded a rich context for understanding results from the archival claims data analysis.

Each of the five domains described in the logic model was examined by the indicated analytic methods. The PERT began its examination of each domain with descriptive analyses covering all data elements relevant to that domain. These analyses entailed both quantitative and qualitative methods, depending on the data element. The more complex research questions were addressed by sophisticated methods, such as statistical modeling, using the claims data and case study methodologies to examine differences in plans’ benefits.

The evaluation design incorporated multiple data collection approaches, including:

  • nominal plan benefits data from the OPM website,
  • parity implementation reports from all FEHB plans with 500 or more enrollees,
  • site visits to acquire detailed plan data from selected plans and OPM,
  • archival enrollment and claims/encounter data from selected FEHB plans and a matched set of comparison group plans, and
  • focus groups of providers.

Table II-2 summarizes the project’s data collection approaches and the evaluation domains addressed by each approach. It should be noted that several of the evaluation domains were addressed by the combination of multiple data collection approaches, which is indicated when a domain appears in more than one row of the table. In this chapter, each of these approaches is discussed in depth.

Table II-2. Overview of data collection approaches

Data collection approach Evaluation domains addressed Lead
Organization
Obtain nominal plan benefits data from OPM website
  • for all plans
  • for years 1999, 2000, 2001, and 2002
Benefits -- Changes in the design of MH/SA benefits Harvard
Obtain limited data at two points in time on plan policies and procedures
  • from all plans
  • for 2000, 2001, and 2003
  • PRR in 2002 and 2003
Benefits -- Changes in:
  • plan policies and procedures for MH/SA benefits
  • design of MH/SA benefits
  • structure and/or management of general medical benefits
RAND
Conduct site visits to 8 selected plans and the OPM to obtain in-depth plan data. Benefits -- Changes in:
  • the design of MH/SA benefits
  • plan policies and procedures regarding MH/SA benefits
  • structure or management of general medical benefits

Cost -- Additional expenses incurred by FEHB plans and the Federal Government as a result of the parity implementation

Quality -- Changes in the use of or adherence to guidelines, new quality assurance measures, and use of evidence-based guidelines for the treatment of MH/SA conditions

RAND
Obtain enrollment and claims/encounter archival data
  • from 9 selected plans* and comparison group plans
  • for years 1999, 2000, 2001, and 2002
Cost
  • Changes in aggregate and per-enrollee costs to plans
  • Changes in cost to beneficiaries using MH/SA services
  • Evidence of adverse or favorable risk selection among enrollees or those disenrolling after implementation

Access -- Patterns of access to MH/SA services both pre- and post-parity

Utilization -- Patterns of utilization both pre- and post-parity

Quality -- Adherence to proposed guidelines as reflected by patterns of care for MH/SA conditions

Harvard

RAND

Focus groups of providers in the networks of selected plans in the West, Mid-Atlantic, and Northeast regions Provider awareness -- Providers’ awareness of the parity benefit implementation and its implications for clients’ care HS/Westat
*As explained in the text, eight plans were selected initially and were site visited. The PERT was unable to obtain comparable archival data on utilization and costs from one of the visited plans, but the PERT was able to obtain archival claims data from two additional plans.

 

View full report

Preview
Download

"parity.pdf" (pdf, 5.07Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"parityA.pdf" (pdf, 61.31Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"parityB.pdf" (pdf, 194.85Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"parityC.pdf" (pdf, 134.89Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®

View full report

Preview
Download

"parityD.pdf" (pdf, 147.52Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®