Unlike report cards that use clinical measures, The Consumer Assessment of Healthcare Providers and Systems (CAHPS) reports patient ratings of their experience with care.  The CAHPS surveys were designed to provider consumers of health care with more information and to enable the consumer to make better choices about health care coverage.  Many resources have gone into the development and administration of CAHPS surveys, and as a result, many reports and studies have been conducted on some of the effects of CAHPS.  This report seeks to look at the literature surrounding the CAHPS program and address beneficiary issues, including whether the CAHPS survey accurately reflects patient experiences and whether the CAHPS survey is useful and beneficial for consumers making healthcare decisions. 

Overview of CAHPS

In 1995, the Agency for Healthcare Research and Quality initiated CAHPS in order to develop survey and reporting products to give consumers feedback on health plans and provider performance.[1]  AHRQ, through the CAHPS program, seeks to support the assessments of consumer experiences with healthcare.[2]  CAHPS was originally launched in response to concerns that there was little information available about health plan enrollees' own perspective on the quality of their plans.  The role of CAHPS has been expanded, and now seeks to address needs of health care consumers, purchasers, health plans, providers, and policymakers.  The current goals of the CAHPS program are to develop an effective patient questionnaire for comparing data, and to generate tools and resources for sponsors to produce understandable comparative information that is useful for consumers and health care providers.[3]

CAHPS continues to evolve, and is now in its third stage:[4] 

CAHPS I:  1996-2001

• CAHPS began when AHRQ gave 3 cooperative agreements to research organizations that collaborated with AHRQ, CMS and others to develop test instruments for health plans. 
• During this period, CAHPS becomes the de facto standard for measuring consumer experience with health plans. 
• At the end of this period, CAHPS products achieved broad use.  CMS, the National Committee for Quality Assurance, and most state Medicaid programs adopted the CAHPS health plan survey.

CAHPS II:  2002-2007

• The second phase of CAHPS addressed the growing demand for new surveys in other health care sectors.  CMS requested collaboration on surveys for hospitals, nursing homes, in-center hemodialysis facilities, Clinician/Group Surveys, and others. 
• Researches also sought to develop tools for survey providers to help in quality improvements.
• Many stakeholders realize the value of CAHPS as a quality improvement tool.
• During this phase, CAHPS was translated into Spanish and research was conducted on the cultural competency of the survey instruments. 

CAHPS III:  2007-

• The emphasis of CAHPS is no longer simply on developing new surveys.  The goals have shifted to refining and supporting CAHPS surveys, and increasing the use of CAHPS surveys.[5]

Does the CAHPS Survey Accurately Reflect Patient Experiences?

Use of Proxies

Some respondents are not able to complete the CAHPS surveys themselves.  These respondents may have vision problems or may not understand the language, or they may not be able to cognitively understand the survey questions.  For various reasons, respondents may require the assistance of a proxy.  When a proxy actually completes the survey for someone, they are called a proxy respondent, whereas proxy assistants help the respondent by either reading the survey, translating, or writing responses.[6] 

Proxy respondents have an effect on survey outcomes.[7]  Because of the potential for bias, AHRQ advises consumers to complete their own surveys.  "To ensure that the responses to a CAHPS survey reflect the respondent's own experiences with care, consumers should complete their own surveys. It is especially important that providers not assist their patients because of the possibility of bias."[8]  AHRQ recognizes that when respondents get assistance from providers, they tend to respond more favorably.  In order to enable data collection from people unable to read the survey or write the responses, AHRQ suggests administering the survey by telephone or providing translators.[9]  Information on the CAHPS website and in the studies makes the questionable assumption that the CAHPS surveys are administered fairly and without any influence from the providers acting as a proxy.

Medicare health plan surveys are the exception to the restrictions against proxy assistance.  In Medicare health plan surveys, family or friends are permitted to assist beneficiaries who are unable to provide responses independently.  In studies that have been completed on Medicare CAHPS, it has been found that proxy respondent differences are small for objective reporting items but larger for global ratings.[10]  Proxy assistance effects are small.  The relationship of the proxy to the intended respondent also affects the proxy responses.  Spouses' responses are found to be closer to the patients' intentions.  Proxy respondents (non-provider respondents) typically give less positive evaluations then both the self-reporting respondents and spouse proxies.  Data also suggests that proxies who assist the beneficiary may also influence responses, but more research is needed to see how the proxy influences global ratings.[11]

CAHPS surveys that use proxies use a case-mix variable to analyze the proxy assisted surveys with the independently completed surveys.[12]  Using proxy respondents is important to ensure that the experiences of the most vulnerable patient groups are not omitted.  However, CAHPS surveys need to be aware of the effect proxy respondents may have on the results.  Studies have found that using case mix adjustment and propensity score rating or matching can improve the accuracy of surveys using proxy respondents.[13]  Additionally, when surveying a population where a high proportion of respondents will need proxy respondents or assistance, a good survey would place a greater reliance on specific and objective reports rather than on subjective global ratings.[14] 

Self-Reported Data  

One report by the Health Research and Educational Trust looked at the self-reported data, and concluded that people tend to rate themselves more favorably than is the case.[15]  This report discussed biases that people have about their own health and self-identity, and the tendency of people to downplay negative experiences.  This study was not discussing the CAHPS survey specifically, and it also was not advocating a dismissal of self-reported data.  Rather, this study brought to light factors that affect the accuracy of self-reported data and proxy data.  This study was one of the few that discusses the psychological factors that can affect the outcomes of health surveys. 

AHRQ's response to those who question the value of patient experience data is that patients are the most knowledgeable informants about their experiences with care. AHRQ distinguishes between clinical data and CAHPS surveys.  "Until recently, patients' views of their care were given weight only to the extent that they correlated with clinical measures.  Now there is growing acceptance of patients' perspectives on care as a valid measure in itself."[16]  CAHPS is designed to measure patient experiences; it is not substitute for a clinical measure of quality.[17]  Indeed, research has shown that self-reported data does not always reflect the clinical measures of quality.[18]  Therefore, CAHPS data is best considered distinct from and different than quality measures.  When a consumer uses CAHPS data to make health care decisions, it will be important that the consumer is aware of the distinction between clinical measures and patient experiences. 

Cultural Competence of Surveys

CAHPS surveys are distributed to a myriad of people who have a variety of educational levels, ethnic backgrounds, ages, and medical conditions, and who speak a variety of languages.  Studies have used CAHPS data to find that racial, ethnic and linguistic minorities face barriers in access to care, and lower quality of care than non-minorities in the same health plans.[19]  These studies call for health plans to provide a more diverse workforce with training in cultural competency, extended clinic hours, and interpreter services.[20] 

Most studies discussing ethnicity and language focus on the results of the CAHPS surveys, but they also tend to conclude with a call to construct surveys that are sensitive to different cultures and languages.  A study by the Journal of General Internal Medicine states that "[c]ulturally and linguistically appropriate CAHPS survey reports can potentially help reduce the observed racial/ethnic differences in plan enrollment patterns."[21]  This study also warns that if white and English speakers are more likely to use the results of these surveys, it is possible that the results could reinforce the concentration of minorities in plans with lower assessments of care.   Few studies examine the response bias within racial/ethnic groups for CAHPS surveys, but the one study that was cited seems to say that the bias is small.[22]  However, studies cite that the Spanish version of CAHPS is significantly less reliable and valid than the English versions.  Cultural differences may influence response style in surveys and limit the ability to make comparisons between respondents of different racial/ethnic groups.[23] 

Efforts are being undertaken to increase the cultural competency of the CAHPS surveys.  The Commonwealth Fund recently awarded money to the University of Florida, Principal Investigator Robert Weech-Maldonado, to develop and test the Patient Assessments of Cultural Competency Survey.  "The CAHPS surveys have been used to assess racial, ethnic, and linguistic differences in patients' experiences with care.  There are concerns, however, that the surveys do not fully capture the aspects of the care experience that are particularly relevant to minority patients, such as access to language services and perceived discrimination.  The goal of this project is to test, validate, and disseminate a new survey – the Patient Assessments of Cultural Competency (PACC) – that addresses issues of cultural competency."  The project team will create a short version of this survey to serve as a supplemental module for the CAHPS instruments.  AHRQ and the National Committee for Quality Assurance intend to collaborate on the dissemination of the PACC survey.  This survey should be completed in February 2009.[24] 

In addition to linguistic and cultural concerns, CAHPS surveys need to be readable and understandable for low-literacy groups.  Currently, the CAHPS surveys range from a sixth to eighth grade reading level.[25]  If reading level is a problem, CAHPS survey administrators can conduct a survey over the phone.  In order to increase readability, some studies are looking into creating an illustrated version of the CAHPS survey.  However, some say that the telephone method may be easier and cheaper to administer than an illustrated CAHPS.

Is the CAHPS Survey Useful and Beneficial for Consumers of Healthcare?

CAHPS is designed to be utilized by consumers as a tool for making health care decisions.  However, consumers may not be aware that this information exists.  In a 2005 Mathematica report, the authors state that the literature indicates that most consumers are unaware of the publicly available quality information.[26]  Efforts are being made to make consumers aware of CAHPS information, but current consumers still make decisions based primarily on information from family members, friends, or their physician and have a lower level of trust for published data.[27] 

CAHPS data is said to be easier for consumers to understand and relate to than report cards based on clinical measures.  Studies report that consumers value information and demand the ability to compare health plan choices.[28]  Focus group evidence indicates that consumers want patient perspectives on care ratings for hospitals.[29]  However, while many consumers may report that this information is valuable, many studies conclude that the valued CAHPS results do not affect choices.  Studies of Iowa and New Jersey Medicaid beneficiaries concluded that CAHPS did not affect health plan choices.[30]  The Mathematica literature review reports a mixed finding on the effects of CAHPS on actual consumer decision-making.  Some studies show that more educated and informed consumers are more likely to use this information.[31] 

While the effect of CAHPS data on consumer actions is hard to see, the effects of this data on health providers is apparent.  CAHPS literature suggests that the stakeholder role is becoming increasingly important.[32]  During the first stage of CAHPS, health plan providers became very interested in using CAHPS as a tool for quality improvement.  Many studies report that health plans are using CAHPS to identify areas of needed improvement and as a rating mechanism to compare their plans with their competitors. The majority of the literature on CAHPS effectiveness focuses on how health plans are using the CAHPS survey data and how best to use the CAHPS survey data for quality improvement.[33]  

Conclusion

The underlying premise behind the usefulness of CAHPS for beneficiaries is that people will be better off if they have more information about their health choices.  CAHPS seeks to provide a different set of information; not information about quality measures, but about a patient's perspective of global ratings, and experiences.  In order to effectively capture accurate information from patients, CAHPS survey administrators need to be aware of the effects that proxies have on data, the issues with self-reported data, and ensure that the surveys are culturally competent. CAHPS data must address these concerns in order to be reliable for patients.  Research shows that patients value and understand CAHPS ratings more than they understand complicated clinical measures.  However, patients are not generally utilizing CAHPS data to help them make health care choices.  Much research is being conducted on how to design an accurate survey tool, and many studies are being undertaken to examine how to increase the utility of CAHPS for health plans and health providers, however there seems to be less focus on creating a CAHPS report that is utilized by patients.  The effectiveness of CAHPS data on patient choices has yet to be fully realized.