APPENDIX B

Enhancing Emergency Preparedness, Response, and Recovery Management for Vulnerable Populations: Compendium of Risk Communications

Andrew M. Parker, Stefanie A. Stern, Lisa S. Meredith, Lisa R. Shugarman, Ellen Burke Beckjord, Anita Chandra and Stephanie L. Taylor

RAND Health

PDF Version: http://aspe.hhs.gov/daltcp/reports/2008/emergfrB.pdf (72 PDF pages)

TABLE OF CONTENTS

BACKGROUND

METHODS

Compendium of Risk Communication Materials for Vulnerable Populations

Compendium Search Methods

Compendium Data Extraction

Compendium Sample

RESULTS

Identify Promising Risk Communication Strategies

Identification of Gaps and Commonalities

CONCLUSIONS

Limitations

Summary

REFERENCES

NOTES

APPENDICES

APPENDIX B1: Compendium of Emergency Risk Communications for Vulnerable Populations

APPENDIX B2: Organizations Included in Compendium Sample

APPENDIX B3: Compendium of Unavailable Resources

APPENDIX B4: All-Star Score Sheet

LIST OF FIGURES

FIGURE 1: The Intersection of Public Health Emergency Preparedness, Vulnerable Populations, and Risk Communication

FIGURE 2: Compendium Sampling Flow

LIST OF TABLES

TABLE 1: Proportion of All-Stars Rated Highest on Each Dimension

TABLE 2: Compendium Resources by Source, Medium, Audience, Message, and Key Behavior

TABLE 3: Number of Resources Targeting Each Vulnerable Population by Emergency Type

BACKGROUND

Risk communication is a critical component of public health emergency preparedness (PHEP), response, and recovery efforts,^[1] as it provides an all-important link for the public to make informed decisions about what actions to take. According to the U.S. House of Representatives,^[2] public health emergency communication should be non-discriminatory, with equal access and utility to all individuals. As described in the literature review in Task 3 of this project,^[3] vulnerable populations face specific challenges in their ability to access, process, and act on risk communication and may have particular needs before, during, and after emergencies.

To assist planning and response efforts for these more vulnerable populations, we created a compendium to obtain and inventory relevant communication, outreach, and education materials, related to emergency preparedness, response, and recovery, that are intended for vulnerable populations. Specifically, the purposes of this compendium are:

1. To provide a list of resources to public health emergency planners and those working to deliver risk communications to vulnerable populations.

2. To identify promising risk communication strategies.

3. To identify gaps and commonalities in available resources.

Risk communication has been defined as an interactive process of exchange of information and opinion among individuals, groups, and institutions. It involves multiple messages about the nature of risk and other messages, not strictly about risk, that express concerns, opinions, or reactions to risk messages or to legal and institutional arrangements for risk management.^[4] For purposes of this task, and in keeping with other discussions of risk communication,^{[5, 6]} we focus on communications regarding risks that specifically include actionable information. That is, the information does not simply describe the nature or consequences of a risk, but rather provides information on how to prepare for, protect against, respond to, or recover from the risk. A great number of risk communication resources exist for general audiences. Yet, the literature review (Task 3) and recent events such as Hurricanes Katrina and Rita highlight the specific challenges and shortcomings of risk communications for vulnerable populations. In addition to being actionable, the public health emergency risk communications considered here must address the specific needs that vulnerable populations will likely have before, during, or after a public health emergency.

Vulnerable populations include individuals who have disabilities, are institutionalized, are elderly, are from diverse cultures, have limited English proficiency or are non-English speaking, are children, are transportation disadvantaged, are pregnant, have chronic medical disorders, or have pharmacological dependency (i.e., chemical dependency/addiction). The definition used here has been adopted by the Department of Health and Human Services and was derived from recommendations of the Interagency Coordinating Council on Emergency Preparedness and Individuals with Disabilities, the draft implementation plan for the Pandemic and All-Hazards Preparedness Act, and the draft revisions to the National Response Plan.

The compendium is built on the foundation provided by the literature review of promising risk communication approaches and communication strategies (Task 3). In conjunction with the literature review, this compendium informs the case studies of state or local region risk communication practices as they relate to vulnerable populations (Task 5).

METHODS

Compendium of Risk Communication Materials for Vulnerable Populations

As described in more detail below, risk communication materials for vulnerable populations were identified by searching publicly available websites. Websites were scanned and reviewed for communication materials that were at the intersection of three domains. To be included in the compendium, items needed to focus on each of three criteria:

1. PHEP, including preparedness for natural disasters, man-made disasters, terrorist events, and contagious public health exposures.

2. Vulnerable populations, as those at-risk.

3. Risk communication that both characterized the risk and provided information that was actionable.

Figure 1 depicts the intersection of these three criteria, including examples of resources that fit in this intersection. Many available resources address some but not all of these domains. Figure 1 also provides examples of resources that would not be in this intersection and hence are not included in the compendium.

Compiled materials have a significant focus on the needs or special circumstances of one or more vulnerable populations (e.g., those with disabilities, children, and pregnant women, etc.), targeting members of those vulnerable populations, their caregivers, and/or the provider communities that serve these populations. Some resources were targeted at service providers and not the vulnerable population specifically. In this case, we only include resources that provided actionable recommendations for communicating with the vulnerable populations and not merely general advice or considerations. Items could also address any stage of the preparedness cycle, including preparedness, response, and/or mitigation and recovery. Finally, many resources have been translated into a variety of languages (including American Sign Language). Resources that were mere translations of general population resources, without specific attention to the broader issues affecting limited English or non-English speakers, were not included in the compendium. However, where resources that met inclusion criteria are translated, these other languages are noted (in Appendix B1, final column).

FIGURE 1. The Intersection of Public Health Emergency Preparedness, Vulnerable Populations, and Risk Communication

Compendium Search Methods

We focused the compendium search on material that was both widely and readily available (from websites of major national organizations) through a snowball-sampling strategy that began with the identification of government and national organizations whose focus was on public health and emergency preparedness (e.g., American Red Cross), vulnerable populations (e.g., National Organization on Disability), or both (e.g., Disability Preparedness Center). Specifically, team members and other area experts at RAND identified organizations targeting each of these areas. The project team searched each of these organizations’ websites. Additionally, team members followed links from these to other websites one or two “clicks” deep and eligible items were cataloged. If any links led to sites that were themselves rich sources of information, we added those organizations to our existing list of organizations and returned to them later for thorough searches. The list of all organizations included in this compendium, as well as their website addresses, is provided in Appendix B2. Resources that were referenced on websites, but not immediately available electronically (either because no electronic versions were posted or the materials needed to be special ordered) were catalogued but not included in the final compendium. This small number of unavailable resources is discussed below and provided in Appendix B3.

Compendium Data Extraction

The construction of the compendium involved three progressive phases of review. Phase 1 review focused on the identification of candidate resources with the goal of growing the compendium, as well as initial cataloging of key dimensions, which flow from the conceptual framework included in the proposal. This framework draws on theories of persuasive communication and includes five aspects of communication:^{[7, 8]}

1. The source of the message, including whether the organization is a government or non-government organization.

2. The medium by which the message is conveyed, including whether the risk communication was text-based, audio/visual, or interactive (electronically, as in an on-line quiz, or personally, as in a discussion group). Because there is great diversity even within these categories, a specific type variable was also entered, which described the resource qualitatively (e.g., booklet, brochure, etc.).

3. The targeted recipient, which includes whether the risk communication targets members of the vulnerable population, caregivers, or providers. This also includes which vulnerable population is targeted.

4. The content of the message itself includes emergency type, which was cataloged as natural disaster, man-made disaster, terrorist threat or incident, infectious disease outbreak, infectious disease pandemic, other public health emergency, other emergency, or unspecified. It also includes stage of preparedness, which was catalogued as preparedness, response, or recovery.

5. The target behavior, which includes five functional areas--maintaining independence, communication, transportation, supervision, medical care, and other.

Generally, the categories within each variable were allowed to overlap, with a single resource potentially getting multiple assignments (e.g., targeting both vulnerable individuals and caregivers).

The compendium is a database in an Excel spreadsheet. Each resource is catalogued in a single row, with columns for document number, hyperlink to an electronic version of the resource, citation, source, source type, medium type, specific medium type, vulnerable population, target audience, emergency type, stage of preparedness, functional area, and whether the resource is also available in other languages. A legend of these variables is presented in Appendix B1.

During Phase 2 review, each resource was reviewed by a randomly assigned team member, and catalogued data were double-checked. Reviewers were also instructed to identify exceptional resources (all-stars), which would be reviewed carefully in Phase 3 to identify key messages and strategies. Specifically, Phase 2 reviewers were instructed to flag resources they viewed as exemplary, in terms of anticipated benefit to their intended audience. Anticipated benefit was defined as the conveyance of actionable information that is appropriate for the intended audience. Furthermore, reviewers were informed that the aim was to capture approximately the best 10 percent. The goal was not to systematically review each resource, but instead to identify key exemplars that could be the focus of a more intensive review.

For Phase 3 review, four team members divided up resources flagged as all-stars. Each all-star resource was then reviewed carefully, rated on six dimensions, including the extent to which the resource:

• Clearly stated and addressed objectives;
• Clearly stated and addressed risks associated with the public health emergency;
• Reasonably covered issues salient to the specified vulnerable population(s);
• Provided specific guidance on how to act on the advice given;
• Was clear and easy to understand; and
• Was engaging.

Reviewers made notes on the motivation behind their ratings and also noted key messages delivered by the resource. The all-star score sheet is provided in Appendix B4.

Compendium Sample

Figure 2 illustrates the compendium sampling flow. Phase 1 review identified 309 resources from 73 different organizations (as well as websites linked from those organizations’ websites). As shown in Appendix B2, 28 of these 73 organizations were within the Federal Government, 20 were associations (e.g., National Association of City and County Health Officials [NACCHO]), 15 were service organizations (e.g., the American Red Cross), four were research organizations (e.g., RAND), and four did not fit any of these classifications but were referenced sufficiently to warrant adding to our list of organizations. Upon the closer inspection of the Phase 2 review, it was determined that 32 resources failed to satisfy our definitions of PHEP, vulnerable populations, or risk communication. Eight resources were risk communications directed at the safety of responders themselves, rather than vulnerable populations. As these were outside the scope of the compendium, they were removed. 27 of the remaining 269 resources were unavailable for download, and hence were not immediately available to audiences. Because of this, they were separated from the main compendium and are not included in the following analyses. The result is 242 resources in the final compendium.¹ Forty-one (16.9 percent) of these were identified by Phase 2 reviewers as all-stars.²

FIGURE 2. Compendium Sampling Flow

RESULTS

Our results are presented in two sections. The first provides a detailed, qualitative analysis of promising emergency risk communication strategies for vulnerable populations among the risk communications included in the compendium. The second section takes a more quantitative approach, identifying gaps and commonalities in the coverage of materials across vulnerable populations, emergency types, and other key characteristics.

Identify Promising Risk Communication Strategies

The final Phase of review (Phase 3) consisted of carefully reviewing each resource labeled an all-star in Phase 2, with the goals of identifying promising risk communication strategies and providing examples of these strategies drawn from this set of resources. At this stage, the goal was not to quantitatively evaluate each and every resource. Each all-star resource was rated on six dimensions using a scale ranging from poor to truly extraordinary, with relatively finer distinctions made at the high end of the scale, reflecting that these resources had already been identified as exemplary. Subsequent analysis focused on the dimensions upon which specific all-star resources were seen to excel. Table 1 displays the proportion of the 41 all-stars rated as truly extraordinary (the highest rating) on the six score sheet dimensions. The all-star score sheet is included in Appendix B4.

Clarity and Understandability. The most often-cited point of excellence among the all-stars was clarity and understandability. Within this, three themes arose. First of all, providing concrete examples can dramatically increase clarity. Videos by Ready America (Documents #8 and #9 in the compendium) showed members of the vulnerable population and their caregiver acting out recommended actions, providing concrete models for recipients. A webcast (#77) by the Substance Abuse and Mental Health Services Administration (SAMHSA) ends with a mock counseling session, helping recipients to ground their expectations. An American Red Cross coloring book (#62) includes pictures of what children should be doing during an emergency. And a resource from the American Red Cross (#63) includes a letter written to seniors by seniors, conveying compelling, relevant experiences.

A second theme was that understanding may be facilitated by simply defining terminology and spelling out assumptions. Emergency preparedness communities often use language that may not be well-known to vulnerable populations, and hence need to be defined. For example, the Ready, Set, Prepare! children's activity book (#59), also from the American Red Cross, has a Words to Know section defining important words. Similarly, SAMHSAs Developing Cultural Competence in Disaster Mental Health Programs (#113) begins with a set of guiding assumptions which help set the stage for the rest of the resource.

Third, tailoring the format to the audience and situation can greatly impact clarity and understanding. Audio/visual and pictorial displays can have great impact. A simple but dramatic example is provided by the City of Los Angeles guide, Emergency Preparedness for People with Disabilities (#183), which contains a drawing of a dog holding a sign that says dont forget me. But whereas flashy can be good, simple can be better for specific audiences or situations. A set of tip cards for first responders from the Center for Development and Disability at the University of New Mexico (#4) consists of simple, bulleted information broken down by disability type. The effectiveness comes from the ability to quickly digest the key points in dynamic response environments. A very different example of formatting that facilitates use by different audiences is the Centers for Disease Control and Prevention Website, Index of Printable Hurricane and Flood Materials (#289), which provides risk communications in multiple languages in an easy-to-navigate matrix design. Another example of tailoring is the Masters of Disasters suite of material for earthquakes (#39). This set of materials offers lessons and examples tailored to multiple developmental stages, for use with children in different age groups. Other important aspects of tailoring the format include using appropriate reading levels and ensuring that availability matches the audience (e.g., because of modality or technology limitations).

Comprehensiveness. Coverage of issues relevant to vulnerable populations was also frequently noted in the review of these documents. Resources that were careful to acknowledge diversity within vulnerable populations stood out. For example, a factsheet from the U.S. Department of Veterans Affairs (#267) noted that children of different age ranges will likely have different reactions to terrorist attacks, and hence will require different caregiving strategies. The previously mentioned SAMHSA guide on cultural competency (#113) explicitly emphasizes this point.

Comprehensive coverage, however, does not mean uniform coverage. In particular, risk communicators may have the instinct to tell people what they should know, not recognizing that some of that information is obvious or widely known. Dwelling on such information could potentially damage credibility of the source. Instead, including and emphasizing crucial information that is less likely to be known can have a greater impact.^[9] In particular, such a strategy can lead to ah-ha moments with recipients, enhancing the perceived value of the resource. A straightforward example of this was noted in a brochure by the National Organization on Disability for animal owners (#180), noting that different resources (e.g., access to shelters) may be available to service animals than for pets in general. A brochure from the Community Emergency Preparedness Information Network on hurricane preparedness for the hearing- impaired (#143) suggested placing important papers in waterproof containers and putting refrigerators/freezers on their coldest settings to prepare for a potential power outage. These specific examples may not be news to some individuals, but were to the reviewers, and may also be to other individuals.

Action Orientation. Also frequently cited by reviewers was the action orientation of many all-star resources. An American Red Cross activity book (#59) encourages children to take an active role in preparing for emergencies, and a ready.gov scavenger hunt (#242) sends children in search of emergency kit items. The second section of the guide to cultural competency (#113) provides a framework for action. Additionally, checklists and self-assessments can help recipients tailor resources to their own needs. The booklet, Listen, Protect, and Connect: Psychological First Aid for Children and Parents, (#250) from the U.S. Department of Homeland Security (DHS), includes places to write in details, which helps to ground the risk communication in an individuals experiences. Finally, the credibility of calls to action may be enhanced by recognizing barriers to success. The NACCHO Advanced Practice Center Toolbox (#18) does this quite effectively, citing cost of preparedness (e.g., first-aid kit and battery-powered radio) and denial of risk as examples of barriers.

Ability to Engage the Audience. Another prominent theme was the ability of resources to engage their intended audience. Multi-modal approaches were noted by the reviewers as breaking monotony, grabbing attention, and providing different angles on a common message. For example, SAMHSAs substance use and trauma-related webcast (#77) combines an interview video with parallel PowerPoint slides (with the option of enabling closed captioning). Such approaches acknowledge both different learning styles and different communications needs. Embedding checklists, self-assessments, and other active segments can help to achieve some of the benefits of shorter resources, but within longer, more comprehensive resource packages. The Family Readiness Kit from the American Academy of Pediatrics (#125) was a particularly striking example of this, including checklists, childrens activities, and fact sheets within a larger kit. Similarly, in longer resources, and particularly those targeting providers, appendices can add crucial details without breaking up the overall flow of the main resource.

Clearly Stating and Addressing Objectives and Risks. Finally, reviewers also noted that stating the purpose of a resource early and clearly was helpful in grounding resources. For example, a preparedness manual from the American Red Cross (#1) for people with disabilities (PWD) was at the same time very tangible and very detailed--a combination facilitated by clearly spelling out objectives. A single-page SAMHSA handout entitled Alcohol, Medication, and Drug Use After Disaster (#92) very clearly and concisely spells out substance abuse risks after a disaster and provides strategies for managing these risks. Similarly, motivating recommended actions helps make them more compelling and relevant to the decisions that need to be made by the recipient. The tip cards for early responders (#4), in one specific instance, point out that every person and disability is unique, as a means of motivating responders to ask a person before attempting to assist them. Such motivations can serve to demystify recommendations by suggesting contextual considerations.

Identification of Gaps and Commonalities

In addition to the qualitative review of those resources labeled all-stars, we also analyzed the compendium itself to provide a general analysis of gaps and commonalities among those resources identified as emergency risk communications for vulnerable populations. Table 2 presents breakdowns of the 242 resources by key variables in the compendium. For ease of comprehension, categories have been ordered in descending frequency, except for stage of preparedness, which has a natural ordering.

The resources came almost equally from government and non-government sources. Text was the medium of choice, with relatively few audio/visual or interactive resources. Equal number of resources targeted members of vulnerable populations and their caregivers, with somewhat fewer resources targeting providers. Some of this latter effect may have been due to limiting the provider resources to just those specifically giving guidance on risk communication.

Among the resources that we found, there was great variation in the number of resources targeting different vulnerable populations. Many resources were directed at the disabled and children, with smaller but still sizeable numbers targeting the elderly and those with chronic medical conditions. Relatively few resources were found for the other vulnerable populations. It should be noted, however, that there was often considerable overlap among these categories, particularly between disabled and chronic medical conditions and between elderly and chronic medical conditions. Those resources labeled “other” for vulnerable population include those with mental or cognitive impairments, service animals, those with environmental illnesses or chemical sensitivities, and those dependent on medical devices.

Most of the resources were not specific to one type of emergency or took an all hazards approach. When they did specify an emergency type, it was most likely a natural disaster (an emergency type particularly common in resources targeting children). The remaining few focused on terrorist threats or incidents, infectious disease outbreaks, other emergencies (primarily fires), and man-made disasters. Virtually no resources specifically targeted pandemic diseases, which may differ from other public health emergencies in time horizon (two or three waves of 6-8 weeks duration versus days), greater immunologic risks for certain vulnerable populations, and greater need for personal resilience.

As might be expected, given that we are normally looking toward future emergencies, most resources addressed a preparation stage of preparedness. However, a moderate amount also addressed response, mitigation, and recovery.

As might also be expected, given the focus on risk communication, most resources targeted communication issues. Many also dealt with medical care, with somewhat less of a focus on maintaining independence and transportation issues. Relatively few dealt with supervision. The bulk of those falling in the “other” functional area focused on stress, coping, and mental health.

Finally, of those resources compiled, only 24 (10 percent) were also found to be translated into other languages, with Spanish being the most common. It should be noted, however, that this compendium was not designed to capture all translated materials, but rather we flagged when materials that met other inclusion criteria were also translated. Nevertheless, surprisingly few resources targeting vulnerable populations are also available in other languages (or easily identified as such).

Table 3 presents a cross-tabulation of vulnerable population by emergency type. Within both natural disasters and unspecified emergencies, the two most common emergency types, the frequency of different vulnerable population resources parallels the total (presented in the final column), with disabled being the most common vulnerable population targeted. However, for terrorist threats, infectious disease outbreaks, and man-made disasters, children (or those caring for them) are much more likely to be the intended recipient, rather than the disabled. The sparseness of coverage for many vulnerable populations (noted above and in Table 2) is demonstrated here across different emergency types.

CONCLUSIONS

This task centered on the compilation of emergency-related risk communications designed to address the needs of vulnerable populations. An extensive search of web-based resources from 73 Federal Government agencies and national organizations uncovered 242 available risk communications. The identified resources most commonly addressed issues relevant to PWD, children, the elderly, and those with chronic medical disorders. Relatively few resources were found for those who are institutionalized, pharmacologically dependent, from diverse cultures, transportation disadvantaged, pregnant, or who have limited English skills or are non-English speakers.³ Most of the resources identified did not specify emergency type. Where specified, emergencies were most commonly natural disasters.

Upon detailed review (Phase 2), 41 of these resources were flagged as exceptional (so-called “all-stars”) and subjected to more in-depth review (Phase 3). This review, in turn, uncovered a number of promising risk communication strategies that were effectively implemented by these resources. Most common were themes related to the clarity and understandability of the resources, closely followed by comprehensiveness and having an action orientation. Less often cited, but still highlighted, were strategies to make resources more engaging, as well as providing clear statements of objectives and risks.

The compendium reaffirmed several findings of the literature review (Task 3 of this project). The literature review revealed that some vulnerable populations are especially underrepresented in the peer-reviewed literature, such as institutionalized individuals, individuals with pharmacological dependency, and pregnant women. These three groups are also underrepresented with respect to publicly available resources (Table 2). The compendium resources also parallel the results of the literature review with respect to functional areas addressed. Resources addressing communication and medical care were found most frequently, whereas resources addressing supervision were least common. The literature also suggested that emergency preparedness risk communication should be offered in multiple languages; though the compendium was not designed to capture all translated materials, surprisingly few resources were available in languages other than English.⁴ Finally, the literature review emphasized the need for risk communication tailored to the developmental abilities of children. Here, the compendium found that resources specifically designed for children were prevalent, and addressed a wide variety of types of public health emergencies.

Limitations

The compendium targets resources that are widely available (e.g., through national organizations’) and easily accessible (i.e., on the web). Given the wide-ranging set of possible sources, a snowball-sampling strategy was necessary. This strategy may have limited the search, in turn unintentionally excluding some resources, such as those from state or local sources or those not available on the web. Notably, this compendium is not intended to be a census of risk communications, as such a database would not be cost-effective to create and would be quickly outdated. Hence, caution should be used when making generalizations from the compendium or this summary.

The compendium itself is based on relatively easy-to-identify categories within multiple variables. Each of these pieces of data, in turn, was checked by at least two people. Identification of all-stars, however, was a more subjective process--and one designed to identify exemplars, rather than to provide a detailed evaluation of each resource (although inter-rater agreement was high). Phase 3 review was more qualitative, although structure was provided through the use of a standardized score sheet. Still, the subjective nature of these reviews should be acknowledged, and conclusions taken as suggestive. Furthermore, the materials included in the compendium were not reviewed by members of the target audience, and evaluations are not available assessing their application and usefulness to these specific populations.

Summary

In closing, an impressive variety of risk communications were identified through a targeted search and compiled into the accompanying compendium. Promising risk communication strategies, reflected in our all-star exemplars, cut across many superficial features such as length, medium, and production value. Despite the range of formats, they each highlight the importance of clarity of presentation, careful vetting of information, and the ability to act on the information provided.

REFERENCES

1. Glik, D.C. (2007). Risk communication for public health emergencies. Annual Review of Public Health, 28, 33-54.

2. U.S. House of Representatives. (2006). Chapter 68 (Disaster Relief). U.S. Code Title 42: The Public Health and Welfare.

3. Beckjord, E.B., Stern, S., Meredith, L.M., Shugarman, L., Chandra, A., Tanielian, T., Taylor, S., & Parker, A.M. (2008). Enhancing Emergency Preparedness, Response, and Recovery Management for Vulnerable Populations. Task 3: Literature review. Available at http://aspe.hhs.gov/daltcp/reports/2008/emergfr-appendA.htm.

4. Commission on Risk Perception and Communication, Commission on Behavioral and Social Sciences and Education, Commission on Physical Sciences Mathematics and Resources & National Research Council. (1989). Improving Risk Communication. Washington, DC: National Academy Press.

5. Atman, C.J., Bostrom, A., Fischhoff, B., & Morgan, M.G. (1994). Designing risk communications: Completing and correcting mental models of hazardous processes, Part I. Risk Analysis, 14, 779-88.

6. Covello, V.T., & Allen, F.W. (1988). Seven Cardinal Rules of Risk Communication. Washington, DC: Environmental Protection Agency, Office of Policy Analysis.

7. McGuire, W.J. (1985). Attitudes and attitude change. In G. Lindzey & E. Aronson (Eds.), The Handbook of Social Psychology (Vol. 2, pp. 238-241). New York, NY: Random House.

8. Roberts, D.F., & Maccoby, N. (1985). Effects of mass communication. In G. Lindzey & E. Aronson (Eds.), Handbook of Social Psychology (Vol. 2, pp. 539-598). New York, NY: Random House.

9. Bruine de Bruin, W., Downs, J.S., & Fischhoff, B. (2007). Adolescents thinking about the risks of sexual behaviors. In M.C. Lovett & P. Shah (Eds.), Thinking with Data. New York, NY: Erlbaum.

APPENDIX B1. COMPENDIUM OF EMERGENCY RISK COMMUNICATIONS FOR VULNERABLE POPULATIONS

The spreadsheet legend is presented below, followed by the Compendium itself.