An NAIC survey of state insurance regulators undertaken in 1999 found that only 26 states affirmatively published complaint information “in either an annual report, consumer brochure or on the Department’s web site”. Readers are reminded that the four states with published reports (California, Oregon, New York and Texas) discussed below represent a sample of the higher- performing states with respect to complaints report cards. They were selected for inclusion in this study based on a web site review of the quality of published complaints reports across all states, with these four states appearing to produce among the most comprehensive reports.
Table 10.3 identifies the major features of the report cards published in the four states. Neither Maryland nor Vermont agencies published comparative health insurance complaints data, similar to the other four states. In Maryland both the Maryland Insurance Administration and the Health Education & Advocacy Unit have published reports analyzing the impact of the new appeals and grievances legislation, but do not routinely publish data on all health insurance complaints. It should also be noted that an alternative source of complaints information to published report cards is health insurance plan-specific complaints information. In all of the six states studied, regulators or ombudsman staff made available plan-specific information either through published reports (e.g. Oregon, Attachment 2), through their websites (e.g. Texas) or verbally in response to telephone calls from the public (e.g. Maryland). Staff at the Maryland Health Education & Advocacy Unit noted that consumer requests for plan-specific complaints information were more common during open enrollment periods. However the major focus of the following discussion is the status of comparative complaints reports.
In considering the value of complaints report cards for consumers, the starting assumption is that consumers will be able to exercise some choice among health insurance plans. To the extent that consumers have no or limited choices, report cards – whether they compare measures of quality, health prevention, consumer satisfaction or consumer complaints – are unlikely to be actively sought by consumers. The consumer market for report cards is therefore limited to a subset of the population, with the attention span probably limited to enrollment periods, rather than to more continuous interest in monitoring plans’ performance.
One key feature of complaint reports cards is the measure that they use to compare the relative performance across health insurance plans. Focus groups undertaken by the Pacific Business Group on Health (Schauffler and Rodriguez, 1996) support consumers’ unease with quantitative data. Consumers reported preferring formats based on grades (A, B, C, etc) over rankings such as “average” or “below average”, which, in turn, were favored over proportions such as 80% or 85%. In another focus group study, Jewett and Hibbard (1996) found that 43% of low comprehension problems of health quality report cards were due to gaps in understanding aggregate or quantitative concepts (e.g. mammogram rates were often confused with ratings or fees).
Table 10.3: Consumer Report Cards on Complaints
|Source||Department of Corporations||Health Rights Hotline||Department of Consumer & Business Services||Department of Insurance||Office of Public Insurance Counsel||NA|
|Includes consumer and/or provider complaints||Not stated||Consumer complaints only||Consumer complaints only||Reports total complaints||Separate tables for consumer, provider, total combined complaints||Recognizes vital role of both consumer and provider complaints|
|Distinguishes indemnity and managed care complaints||"Health care service plans" only - i.e. managed care. Separate tables for full service, dental, vision, psychological, other plans||Major breakdown is: commercial HMOs, commercial PPOs, Medicare, Medi- Cal; also separately reports complaints by health plans and medical groups||Separate tables for health insurance and "Health Care Service Contractors" - HMOs and other managed care organizations||Separate tables for HMOs, commercial health insurers and nonprofit indemnity insurers||Basic Service HMOs only with separate tables for large and small HMOs||No guidance|
|Distinguishes justified complaints||No, includes all 'requests for assistance" received||No, includes all complaints received||No, includes all closed complaints||Yes, includes total complaints and upheld complaints||No, includes all closed complaints||No guidance|
|Complaint index or ratio determination||Complaint ratio based on enrollees. Ratio of 1 means 1 complaint received per 10,000 enrollees||Complaint ratio based on enrollees. Ratio of 1 means 1 complaint received per 10,000 enrollees||Complaint index based on premiums. Index of 1 means share of complaints = share of premiums||Complaint ratio using upheld complaints and premium. Ratio of 1 means 1 upheld complaint for $1 million premium||Complaint ratio based on enrollees. Ratio of 1 means 1 complaint per 10,000 enrollees||Prefers "complaint index ratio" based on policies in force, not premium volume. Index of 1 means share of complaints = share of premiums|
|Complaint ranking||No||No||Yes||Yes||No||No guidance|
|Complaint categories||Uses 32 issues categories||Uses 8 issues categories||No breakdown||Total and prompt pay complaints only||No breakdown||Database includes 74 reasons for complaint|
|Presentation format||Numerical tables||Colored circle and arrow tables indicating average, higher or lower than average; bar charts with complaints rates, consumer anecdotes||Numerical tables||Numerical tables, barcharts with complaints ratios||Numerical tables, graphical barcharts ordered from highest to lowest complaints ratios||NA|
When assessed against these findings, the standard of the five complaints report cards examined in this study leaves considerable room for improvement. The two complaints report cards produced by the California Department of Corporations and the Texas Office of Public Insurance Counsel are based solely on complaints rates related to the number of plan enrollees. Both reports provide data on the average complaint rate across plans, allowing consumers who understand the concept of rates to assess the performance of individual plans against the average. Of the two, the Texas OPIC report is more user-friendly in presenting the complaints rate data in graphical format, as well as tabular format, which is helpful to consumers who are less at ease with quantitative data.
The Oregon and New York reports improve upon the Texas and California Department of Corporations reports by including “rankings” of complaints (e.g. 1, 2, 3, 4 etc,) in addition to using complaints rates. This grading system is more likely to be understood by consumers, although the use of the open-ended ranking is problematic. Hence, for example, of the 72 Oregon health insurance plans ranked 1 to 72, the first 11 plans recorded zero complaints, but are still ranked from 1 to 11 on the basis of their premium volume. It may have been more helpful for regulators to use their expertise to group plans into a limited number of grades (A to E), given that consumers are likely to have difficulty understanding whether there are substantive differences in complaints performance using an open-ended ranking.
Finally, of the five reports, it is only the privately funded Center for Health Care Rights in California that moves beyond complaint rates or rankings to also include a relatively simple “average” ranking. The Center’s report includes the following 5-point graphical scale
- - much higher than average;
- - higher than average;
- - lower than average;
- - much lower than average; and
- O - not statistically significant to allow users to visually compare the complaints performance of plans and more readily identify plans with significantly higher or lower complaints rates.
The Center for Health Care Rights report also uses a range of other presentation formats to explain complaints data including bar charts and consumer anecdotes. Hence, rather than simply listing the complaints rates for different types of problems (e.g. inappropriate care, customer service problems), the Center includes an example of each problem through including the story of a consumer with this problem, the action taken by the Hotline to resolve the problem and the system problem identified as a result of individual consumer problems.
Another criterion for consideration in assessing complaints report cards is the extent to which they offer guidance to users.
Hibbard, Slovic and Jewett (1997) suggest that some factors which may be useful in report card design generally include:
- Provision of global ratings by experts – to help reduce the information-processing burden; and
- Use of a decision support method that leads consumers step by step through a rational process, including framing the issues through providing contextual information.
Gormley and Weimer (1999) also support including “expert” opinion such as the use of benchmarks or objective standards against which readers can assess performance. In considering complaints reports cards specifically, Gormley and Weimer (1999) argue strongly that the data presented should only include “justified” complaints, implying that a regulator or expert has made an adjudication as to whether the complaints were found proven.
Once again, the results of the five complaints reports against the factors that comprise this criterion can best be described as patchy. Only the New York report provides advice on complaints and complaints rates using “upheld” or justified complaints. The Center for Health Care Rights includes all complaints received in its reports and argues convincingly that as an independent consumer assistance program, its role is not to adjudicate consumer problems but to provide assistance to all consumers including understanding their rights. However if this argument is accepted, the other two regulatory agencies (the California Department of Corporations and the Oregon Department of Consumer & Business Services) are deficient in not including justified complaints data in their reports. Regulators at the California Department of Insurance noted that there had been significant resistance, including legal challenges from the insurance industry generally (not limited to health insurance) to the publication of complaints data. As a consequence of this industry “push-back”, the California Department of Insurance now establishes proof of justified complaints and notifies insurance plans by letter when a complaint is found to be justified. Perhaps in response to this situation, the California Department of Insurance does not publish any comparative health insurance complaints rates.
The five complaints reports also demonstrate fairly limited performance when assessed against other aspects of the guidance criterion (e.g. use of expert global ratings, decision-support methods, provision of contextual information). If it is assumed that complaints data should be presented in the context of other health insurance plan performance measures to allow consumers to develop a multi-dimensional view, the New York and Texas reports are superior to those of the other states. These reports include a range of other relevant data such as HEDIS measures, NCQA accreditation status and CAPHS measures, in addition to complaints data. Some of the reports include other background information that consumers may wish to consider in making plan selection decisions such as:
- Advice on choosing a financially healthy insurance company and how to manage the cost of insurance (Oregon); and
- Consumer legal rights and protections, and information about different types of health plans and how they operate (Texas and New York).
Staff at the Center for Health Care rights indicated that their report was not intended primarily as a guide to consumers shopping for insurance, but is focused on providing consumer information which is more explanatory of rights and is also directed at regulators and employers for purposes including policy advocacy.
All the complaints reports struggle with providing expert global ratings or decision support methods to help users understand and interpret complaints data. In part, this probably reflects both the embryonic status of complaints reports and the lack of consensus on objective standards. In contrast, health quality report cards have a slightly longer history and some expert consensus around measures such as HEDIS and CAPHS (for example, immunization rates can be compared against performance benchmarks set in Healthy 2000 or state health documents). At interview, state regulators invariably mentioned the difficulty in interpreting “high” and “low” complaint rates, advising that one factor affecting comparative complaints rates across plans was the extent to which plans actively publicized the ability to make complaints, including providing contact information for the state insurance regulator. However, only the California Department of Corporations report provides any advice on this issue, stating that the variations in complaints across plans can be influenced by factors including “the effectiveness of the health plan’s internal grievance procedures, and the quality of the health plan’s services”, together with the “degree to which the health plan discloses to enrollees the right to file” complaints with the regulator. (Note that the New York report provides similar advice on grievances and utilization review appeals, but not in the section of the report containing information on complaints made to the regulator). Admittedly, the inclusion of such advice is likely to provoke a frustrated response by readers of “So what does this mean, should I discount all complaints data?” Unfortunately, this reflects the general lack of agreement on the validity and interpretation of complaints data.
Policy Implications and Recommendations
The above discussion suggests that existing complaints reports suffer from a range of problems which reduce their effectiveness as a tool for consumers. In assessing the role of report cards in bottom-up accountability to consumers, Gormley and Weimer (1999) propose that factors critical to their success include:
- The existence of meaningful variation in the reported measures;
- The ability of consumers to exercise choice; and
- The extent of consumer fees (for example, large consumer payments may diminish the salience of other measures such as quality or complaints data).
On this basis, complaints reports cards are likely to be less relevant to consumers as assessed against the second and third criteria, with significant work required to explain the meaning of variation in complaints across plans. In summary, in their current state of development, complaints reports are more likely to be useful for top-down accountability to policy-makers, legislators and regulators, than for bottom-up accountability to consumers.
It is recommended that comparative complaints data be publicly available to enhance the accountability of health insurance plans to consumers, employers, purchasers, policy- makers, legislators and regulators.
It is further recommended in developing or revising complaints report cards that consideration be given to the following features:
- Complaints data should be presented as part of a suite of performance measures of health insurance plans, rather than as the sole measure of plan performance. For example, the New York report (see New York, Attachment 2) which includes multiple measures such as HEDIS scores, NCQA accreditation and complaints data is preferable to the Oregon report (see Oregon, Attachment 1) which contains complaints data (but no other performance measures) for health insurance, auto insurance, home insurance, life insurance and annuities. In comparing these two reports, it is interesting to note their genesis – the New York report involved input from both the Departments of Insurance and Health, while the Oregon report is produced solely by the insurance regulatory agency. The desirability of including multiple performance measures lends support to earlier recommendations for the need to improve communication and coordination across multiple agencies with involvement in health insurance regulation and health services quality.
- Complaints report cards should include contextual information on the health insurance market to educate consumers about their health insurance choices.
- Complaints report cards should incorporate consumer-friendly performance measures and presentation formats, in accordance with the findings of research (e.g. the use of grades or averages, simple graphical formats).
- Complaints report cards produced by regulatory agencies should be based on justified complaints, in order to incorporate the expert adjudication of regulators and place plans on a level playing field.
- Complaints report cards should include decision-support methods or expert global advice, as such tools are developed. One example outside the complaints report cards arena is the 1998 HMO quality report produced by the Maryland Health Care Cost Commission. This report provides guidance to readers by asking them a series of questions to help structure their decision making and includes a worksheet for users to include the performance measures relevant to their needs. It is more normative and less neutral than the five complaints report cards examined in this study.