ACTION's 1991-92 VISTA Marketing Campaign: An Analysis

Bob Clark and Michele Adler

Office of Family, Community, and Long-Term Care Policy/ASPE/HHS

This report was prepared under interagency agreement #92-046-1032 between U.S. Department of Health and Human Services (HHS), Office of Family, Community and Long-Term Care Policy (now DALTCP) and ACTION. For additional information about this subject, you can visit the DALTCP home page at http://aspe.hhs.gov/_/office_specific/daltcp.cfm or contact the office at HHS/ASPE/DALTCP, Room 424E, H.H. Humphrey Building, 200 Independence Avenue, S.W., Washington, D.C. 20201. The e-mail address is: webmaster.DALTCP@hhs.gov. The Project Officer was Robert Clark

BACKGROUND

The Volunteers in Service to America or VISTA program was authorized by Congress in 1964 to provide constructive opportunities for Americans to volunteer on a full-time basis with locally sponsored projects designed to increase the capability of low-income people to improve the condition of their lives. VISTA volunteers are assigned to live and work in urban or rural sites located within any of the United States, Puerto Rico, the Virgin Islands, or Guam. Volunteers serve for at least one year and receive a basic subsistence allowance and coverage for health insurance. An additional $90 per month is paid as a stipend upon completion of service.

The types of services provided include drug abuse prevention, education, housing, economic development, community design, weatherization, community development, and other programs to address the needs of low-income communities. At the beginning of the program, most VISTA volunteers were recruited from college campuses. Most VISTA volunteers are recruited directly from the community. In September 1992, 22 percent of all VISTA volunteers were recent college graduates compared to only 13 percent in 1989.

During the 1989 VISTA Reauthorization Hearings, Congress expressed interest in raising the proportion of VISTA volunteers with college degrees. As part of this effort, ACTION endeavored to make college students more aware of the VISTA program and increase applications from college campuses through a targeted marketing campaign. This campaign took place during the 1991/92 school year. A contract was awarded to study the effectiveness of the marketing campaign. The purpose of this report is to analyze data from that contract.

This analysis was done outside of ACTION to avoid the perception of bias. The Department of Health and Human Services staff did the analyses at the request of Jane Kenny, Director of ACTION.

STUDY DESIGN

The VISTA marketing campaign consisted of two major components: a recruitment effort and an advertising campaign. The VISTA marketing campaign came out of ACTION headquarters in Washington, DC. Other VISTA marketing activities were occurring at the same time as part of normal activities from the state ACTION agencies and local sponsors.

Forty-five (45) college campuses and universities were selected for the marketing effort--five campuses from each of the nine ACTION Regions. The recruitment effort began in September 1991, when VISTA recruiters started full-time activities. These activities included talking to students either individually or in groups, putting up posters, and distributing brochures. The advertising campaign, which began in January 1992, primarily focused on print media, such as magazines and campus newspapers.

A quasi-experimental design was developed to evaluate the effectiveness of the marketing strategy. In each ACTION Region, one Test (or experimental) campus was selected from the five marketing campuses. Each Test campus was matched to a similar Comparison campus selected from the non-marketing campuses. Matching was done on the basis of public-private status, enrollment size and geographic location. The idea is that unique features of a Test Campus are likely to be found on its counterpart Comparison Campus, so that differential effects of marketing are not likely to be due to these features.

Student Surveys were conducted on the nine pairs of Test and Comparison campuses. The Student Surveys were conducted by telephone at two points in time. The marketing plan called for the main thrust of the recruitment campaign on the Test campuses to be conducted in the Fall of 1991 and the Spring of 1992. The campaign began, after some delays, in January 1992. The first Student Survey was to be conducted in both the Test and Comparison campuses in January 1992 as a pre-campaign interview with the second Student Survey to be conducted during April and May 1992 on these same campuses as a post-campaign survey. The Student Surveys were timed to occur one week before final exams.

One-hundred (100) randomly selected Juniors and Seniors on each of the nine Test campuses and the nine Comparison campuses participated in the first or pre-campaign Student Survey. A different sample of 100 randomly selected Juniors and Seniors was chosen to participate in the second or post-campaign Student Survey. Thus, altogether there were approximately 1,800 precampaign Student Surveys (900 each in the Test and Comparison campuses) and 1,800 post-campaign Student Surveys (900 each In the Test and Comparison campuses).

As in any sample survey, there are potential sources of error, principally sampling error and measurement error. In this study, sampling errors may be attributed to differences in the surveyed and non-surveyed campuses and in the students selected from these campuses. While we are dealing with matched test-comparison campuses, unobserved differences rather than the marketing intervention could affect the results.

Measurement errors may arise from factors like: (a) unreliability of answers from respondents due to faulty memory, phrasing of questions or unobserved processes, and (b) differences in the manner of survey administration from campus to campus and time period to time period. Specifically, in some Test campuses, the marketing campaign began before the pre-campaign Student Survey was conducted. In particular, VISTA recruiters began their activities earlier than anticipated. Hence, any pre-post comparison may understate the effect of the campaign in those cases. Furthermore, since the VISTA marketing campaign was only part of overall VISTA recruitment activities, since there were differences among recruiters in their specific recruitment activities, and since some overall recruitment activity could have occurred on Comparison campuses, this was not a true experimental design in the classic sense.

Finally, data from one Region were excluded from pair-wise comparisons, because a Comparison campus in that Region was found later to have been the site of test marketing efforts. Data from the corresponding Test campus in that Region were also excluded.

Information was also used from the VISTA Applicant Survey, a mail survey of VISTA applicants who had either recently graduated from college or were upperclassmen. In all, 504 VISTA applicants responded to the survey. In addition, a VISTA Recruiter Survey was administered by telephone to nine recruiters on Test campuses.

EVALUATION QUESTIONS

ACTION specified the following evaluation questions to be addressed from the Student and VISTA Applicant Survey data.

1. Did awareness of VISTA increase on the test market campuses when compared to the campuses not in the test market?

2. If yes, can the increase be attributed to the campaign?

3. How did college students learn about VISTA?

4a. What do they know about VISTA service and how do they react to VISTA?

4b. Who are VISTA applicants and what do they know about VISTA service?

5a. What features of VISTA service are most salient to those students who are aware of the program?

5b. Why do applicants to VISTA want to volunteer?

6. What has been the impact of VISTA's marketing, including the message and the means?

7. What has been the impact of VISTA's recruiters?

• Is there a relationship between campus visits and awareness?
• Is there a relationship between campus visits and applications?

RESULTS

1. Did Awareness of VISTA Increase on the Test Market Campuses When Compared to the Campuses Not in the Test Market?

As shown in Table 1, the proportion of students who recognized¹ the VISTA program increased by 5.3 percent on Test campuses from 41.1 percent reported in the pre-campaign Student Survey to 46.4 percent in the post-campaign Survey. No change occurred on the Comparison campuses.

2. If Yes, Can the Increase be Attributed to the Campaign?

To determine whether the increase on Test campuses was due to the campaign, the following pair-wise comparison was conducted. A description of the methodology is shown in Figure 1.²

If the matched campuses are truly alike and the VISTA campaign had no effect, one would expect the net differences to be zero. For example on hypothetical Test campus A, VISTA recognition might have gone from 4 percent to 8 percent and on Comparison campus B, from 3 percent to 7 percent. In both cases, the change over the two time periods is 4 percent. The net difference between Test campus A and Comparison campus B is zero and is obtained by subtracting the two values: 4 percent for B from the 4 percent for A.

Since the survey covered only a sample of students on each campus, a value of zero is unlikely; some variation from zero will probably be found. If the difference between Test and Comparison campuses is in fact quite large, then there are grounds to reject "no real difference" null hypothesis and accept the idea that the VISTA campaign made a difference.

A t-test was employed to determine if the differences in the change over the two time periods for the Test and Comparison campuses were statistically significant. Table 2 shows the result of the t-test.

The marketing campaign was instrumental in modestly raising student awareness of the VISTA program, although statistical significance was not achieved. The t-value was 1.77, which approached but did not reach statistical significance at the .10 level. Looking at individual pairs of campuses, the largest percentage difference was between Princeton and Vassar at 16.3 percent. Princeton had five visits from recruiters during the market campaign. This was the highest number of visits for any campus. All other campuses received only two visits. However, it cannot be assumed from this one instance that VISTA recognition would increase with the number of recruiter visits, since there is no information on the type, extent, or any other indication of the quality of visits made. Finally, the overall significance level for VISTA recognition was higher than that obtained in similar paired comparisons for Habitat for Humanity and Peace Corps recall.

3. How Did College Students Learn About VISTA?

There were four possible sources from which students could have learned about VISTA through the campaign: (a) publications (b) posters, (c) brochures, and (d) talking with someone. The campaign involved putting up posters, distributing brochures, and directly recruiting students. Multiple responses were possible. Finally, although radio and TV ads were not part of this campaign, students may have recalled VISTA from such ads conducted in the past.

To determine how students learned about VISTA, comparisons were made between responses to the post-campaign Student Survey responses from the Test campuses and the Comparison campuses. On the Test campuses, 421 or 46 percent of all students surveyed recalled VISTA compared to 293 or 36 percent of those from the Comparison campuses. (See Table 3.)

Over a third (34.4 percent) of students aware of VISTA on Test campuses had learned of the program from publications, compared to a slightly higher 37.2 percent on the Comparison campuses. (However, for campus newspapers in particular, the situation is reversed, as shown on Table 4 below.) A significant difference³ was found between the proportion of students who learned of VISTA from Radio/TV on the Test campuses (11.6 percent) compared to those on the Comparison campuses (17.8 percent). Students on Comparison campuses were more likely to have learned of VISTA from Radio/TV. (As already noted, Radio/TV were not planned elements of the 1991-92 marketing campaign.)

Higher proportions of Test campus students who knew of VISTA learned from posters, brochures and talking with someone. Significant differences were shown for those students who had learned of VISTA through brochures.⁴

Within each of these media categories, there were a number of specific possibilities. For example, under publications, the possibilities included a campus newspaper, a local newspaper, Newsweek, People, and Time magazines. Since students could list more than one source of information, adding the raw numbers does not necessarily give an unduplicated count.

As shown on Table 4, a higher percentage of students on the Test campuses relative to the Comparison campuses had heard of VISTA from:

• the campus newspaper (7.8 percent versus 4.8 percent)
• television advertisement (6.4 percent versus 8.5 percent) (Note: In this case, the comparison campus percentage was higher.)
• posters in the placement office (4.8 percent versus 2.0 percent)
• talking with a friend (7.8 percent versus 5.4 percent) and
• talking with a VISTA recruiter (3.1 percent versus 1.0 percent).

Based on these data, students on the Test campuses who had learned of VISTA from publications were most likely to have learned about VISTA from the campus newspaper; those who had talked with someone most often had learned about VISTA from talking with a friend.

In other cases, we conclude that students had either heard of VISTA before the marketing campaign began, learned about it from a variety of sources other than those cited above, or could not recall their source of information.

4a. What Do Students Know About VISTA Service and How Do They React to VISTA?

The student questionnaire asked about the respondents' knowledge of six selected facts about VISTA (Questions 23-28). Respondents were queried on whether they knew that VISTA volunteers--

• receive a stipend of $7000 (Q23)
• make only a one year commitment (Q24)
• have opportunities with local sponsors (Q25)
• have opportunities throughout the country (Q26)
• can defer some student loan payments (Q27)
• receive training for their assignment (Q28).

Similar to the analysis for Question 1 above, each Test campus was matched with its counterpart Comparison campus, yielding eight pairs. For each campus, the proportion of students who answered YES to whether they knew the VISTA fact was calculated using results from the post-campaign Student Survey.

Based on the means for the Test and Comparison campuses, as shown in Table 5, students were most likely to know that VISTA service involved assignments throughout the country and training for volunteers. They were least likely to know about the $7,000 stipend and opportunities with local sponsors.

To determine extent of knowledge, differences were calculated by comparing pairs of Test and Comparison campuses and then calculating a mean for these eight differences. The result was the mean difference in the "knowledge-of-VISTA-facts" between Test and Comparison campuses. A t-test was employed to determine if the difference between Test and Comparison campuses was statistically significant.

Table 6 shows the result of the t-test. In general, a higher percentage of respondents on Test campuses showed knowledge of the six VISTA facts, but no results were statistically significant. Based on the 0.14 significance level, Test campus respondents appeared most likely to know that VISTA volunteers made only a one year commitment relative to Comparison campus students.

The only students asked about their reaction to VISTA were those who could specifically recall a specific slogan, catch phrase, concept, or other message content. The number of these students was small; only 35 students or 8.3 percent on Test campuses and 27 students or 9.2 percent of those on Comparison campuses. However, a very high percentage of those who did recall any content reacted positively: 88.6 percent had either a positive or very positive reaction to the message on Test campuses and 96.3 percent on Comparison campuses.

4b. Who Are VISTA Applicants and What Do They Know About VISTA Service?

According to the VISTA Applicant Survey, the overwhelming majority of VISTA applicants who have either graduated or are expected to graduate from college, are women (77 percent), young (their median age is 22.9 years), White (88.7 percent), and have either graduated from college (50 percent) or are college seniors (49.2 percent). Nearly one in four (24.6 percent) are aged 25 or over and one in ten (10.2 percent) are aged 30 or over. While most VISTA applicants are White, minorities also apply: 4.6 percent are African-American, 3.4 percent are Asian, 0.6 percent are American Indians or Alaska Natives, and 2.8 percent are from other races. In addition, 3.2 percent of applicants indicated that they were of Hispanic origin.

Not surprisingly, VISTA applicants are much more knowledgeable about the VISTA program than students. (See Table 6.) Almost everyone knows that assignments are available throughout the country (98.4 percent), that at least a one-year commitment is involved (96.6 percent), that student loans can be deferred (94 percent), and that training is available (90.5 percent). A lower, but still sizable proportion of applicants knew about local sponsors (82.9 percent) and that there was a $7,000 stipend (75 percent).

5a. What Features of VISTA Service are Most Salient to Those Students Who Are Aware of the Program?

For this analysis, we compared responses from the Test campuses to those from Comparison campuses on a series of six statements with which respondents were asked to signify their level of agreement.

Following is a condensed version of the six statements, which began with:

"Becoming a VISTA volunteer would..."

• help with important professional contacts (Q17)
• help with my career decision (Q18)
• help me gain self-confidence (Q19)
• help me make an important community contribution (Q20)
• help me make an important contribution to my country (Q21)
• [be] for someone else, not me (Q22).

There were only 43 respondents from 43 Test campuses and 22 from Comparison campuses. (See Table 7.) The categories STRONGLY DISAGREE and DISAGREE and AGREE and STRONGLY AGREE were combined. Even so, many cells had counts of five or fewer. Hence we did not perform statistical tests, but instead looked for response patterns.

As shown in Table 7, for the first five statements, Test campus respondents are invariably more likely to agree with the statement than the Comparison campus respondents-in some cases (e.g. "important professional contacts" or "help with career decision") by a margin of 15 percent or more. Consistent with this pattern, the students on Test campuses were more likely to disagree with the last statement, thus indicating that the Test campuses developed a more positive attitude toward the benefits of VISTA service than Comparison campuses.

5b. Why Do Apllicants to VISTA Want to Volunteer?

Typically, VISTA is the not the first volunteer experience that applicants have had. Most (74.8 percent) VISTA applicants indicated that they performed volunteer work in college. This becomes apparent in looking at the reasons why people applied to become VISTA volunteers. (More than one reason could be given.) According to the VISTA Applicant Survey, the most popular reason (89.7 percent) for applying for VISTA service was the opportunity to help others. This was followed at 72.4 percent by "my interest in volunteering". The third most popular reason was "the help that VISTA would give my career" at 67.9 percent, followed by "personal private reasons" at 39.9 percent. Less Important were "financial benefits" at 15.3 percent, "the specific Volunteer assignment offered to me" at 14.7 percent, and "health benefits" at 13.7 percent.

6. What Has Been the Impact of VISTA'S Marketing, Including the Message and the Means?

The VISTA marketing campaign appears to have had some effect in increasing the awareness of VISTA on Test market campuses. Posters, brochures and talking with someone reached more students percentage-wise on the test campuses relative to the Comparison campuses. Test campus respondents showed a pattern of more knowledge about selected VISTA facts than the Comparison campuses. Similarly they indicated overall a more positive assessment of expected benefits from VISTA service.

The number of respondents for many of these areas is small; hence we have again relied in some cases on apparent patterns rather than tests of significance.

The likelihood of becoming a VISTA volunteer is slightly greater on the Comparison campuses, but the difference based on a Chi-square test is not significant. (See Table 9.)

7. What Has Been the Impact of VISTA'S Recruiters?

• IS THERE A RELATIONSHIP BETWEEN CAMPUS VISITS AND AWARENESS?
• IS THERE A RELATIONSHIP BETWEEN CAMPUS VISITS AND APPLICATIONS?

The numbers to support an analysis of these questions on the student file are too small: 13 students on Test campuses indicated they had learned of VISTA through a recruiter compared to 3 on Comparison campuses. These differences are further attenuated since only the Test campuses were to have been part of the marketing effort; yet 3 Comparison campus students Indicated they had spoken to a VISTA recruiter.

Unlike students at large, a high proportion of VISTA applicants learned of the program through personal contact. Nearly two-thirds (62 percent) of VISTA applicants talked with a VISTA recruiter. By contrast 51 percent had talked with a friend, 23 percent with a family member and 24 percent with a former VISTA volunteer. Regarding other media, 47 percent had seen information in a publication, 11 percent had heard of VISTA on radio or TV, 41 percent had seen a poster and 81 percent had seen a brochure.

SUMMARY AND CONCLUSIONS

The results of the analysis can be summarized as follows:

• Student recognition of VISTA increased over time on the Test campuses relative to the Comparison campuses. The 5.4 percent different in the two groups approached but did not reach statistical significance.
• Students on Test campuses were significantly more likely to have learned of VISTA from brochures (which were a key part of the marketing effort). Among publications, the campus newspaper was an important source of information on the Test campuses. Students on Comparison campuses were significantly more likely to have learned of VISTA from radio and TV. Such ads were not part of the campaign and could only have been viewed or heard previously.
• Student knowledge of the VISTA program was strong in some areas, weak in others. Students on both the Test and Comparison campuses were more likely to know that VISTA service involved assignments throughout the country and included training. They were less likely to know about the $7,000 stipend and the opportunities with local sponsors. On the other hand, applicants to VISTA very extremely knowledgeable on almost every aspect of VISTA.
• Few students on the Test or Comparison campuses recalled the specific content of marketing messages; those who did reacted very favorably to VISTA.
• Test campus students were consistently more likely to agree with statements about the benefits of VISTA service, e.g., as a means to make important professional contacts, helpful in making a career decision, etc.
• A high percentage of recent applicants (62 percent) indicated that they had talked with a VISTA recruiter, indicating that this was an effective means of attracting applicants.
• VISTA applicants are for the most part female (77 percent), young (median age of 22.9 years), and have performed volunteer work during their college years (74.8 percent). The two most popular reasons for applying to VISTA was "the opportunity to help others" at 89.7 percent and "my interest in volunteering" at 72.4 percent.

Overall, the VISTA marketing campaign was a modest success. The specific marketing activities performed by VISTA recruiters (i.e., posters, brochures, campus newspaper ads, and personal visits) were major sources of program recognition. Future marketing efforts should consider targeting on the areas where knowledge about VISTA was weakest. Repeating the marketing campaign over time could have a cumulative effect greater than the one-time effort analyzed in this report. Finally, although samples are small, perceptions of the VISTA program among those who responded were overwhelmingly positive.

ACKNOWLEDGMENTS

The authors gratefully acknowledge the invaluable assistance and support of Charles Halla and Ruth Fu, ASPE's Computer Support Division, who did all the computer runs, and Brenda Veazey, ASPE/FCLCP, who produced the tables and graphics for this report. Gary Kowalczyk, David Rymph, and Chuck Helfer of ACTION supplied the data files and provided helpful information and comments on earlier drafts of this report.

NOTES

1. Recognition of the VISTA program came an unprompted and a prompted response to the Student Survey. An unprompted response was recorded if a response of was recorded for the following question:

   Q1. "Do you know the names of any programs that offer full-time volunteer service for recent college graduates?"

   A prompted response was recorded for students who answered "No" to question 1, but who (when read a list of volunteer programs which they recognized) answered positively upon hearing the name "VISTA" read.

2. Note that this methodology was the basis of other pair-wise comparisons in this report.
3. Chi-square -- .05 level.
4. Chi-square -- .01 level.

ANALYSES OF VISTA MARKETING AND RECRUITMENT ACTIVITIES REPORT AVAILABLE

ACTION's 1991-92 VISTA Marketing Campaign: An Analysis (January 1993)

Full HTML Version   http://aspe.hhs.gov/daltcp/reports/1993/action.htm

Full PDF Version   http://aspe.hhs.gov/daltcp/reports/1993/action.pdf

INSTRUMENT AVAILABLE

VISTA Recruitment: A Survey of Recent Applicants

Full HTML Version   http://aspe.hhs.gov/daltcp/instruments/VISTArec.htm

Full PDF Version   http://aspe.hhs.gov/daltcp/instruments/VISTArec.pdf

To obtain a printed copy of this report, send the full report title and your mailing information to:

U.S. Department of Health and Human Services
Office of Disability, Aging and Long-Term Care Policy
Room 424E, H.H. Humphrey Building
200 Independence Avenue, S.W.
Washington, D.C. 20201
FAX:  202-401-7733
Email:  webmaster.DALTCP@hhs.gov