Feasibility Study for Demonstration of Supported Education to Promote Educational Attainment and Employment among Individuals with Serious Mental Illness: Final Report

Heather Ringelsen, PhD, and Amy Ryder-Burge, MS
RTI International

Marsha Langer Ellison, PhD, Kathleen Biebel, PhD, and Shums Alikhan, BA
University of Massachusetts Medical School

ABSTRACT
The project focused on answering a series of research questions about Supported Education program composition, implementation, service context, the experiences of individuals involved in Supported Education programs, available Supported Education data sources and ongoing evaluations, Supported Education policies, financing, and gaps in the Supported Education knowledge base. This final project report includes chapters describing the results from each task, as well as a final synthesis chapter that identifies future Supported Education needs and opportunities.

DISCLAIMER: The opinions and views expressed in this report are those of the authors. They do not necessarily reflect the views of the Department of Health and Human Services, the contractor or any other funding organization.

"

Acknowledgments

RTI International and the University of Massachusetts Medical School prepared this report under contract to the Office of the Assistant Secretary for Planning and Evaluation (ASPE), U.S. Department of Health and Human Services (HHS) (Contract No. HHSP23320095651WC). The authors sincerely appreciate the guidance of Emily Jones, Kristina West, and Kirsten Beronio from ASPE during the implementation of this project; each provided helpful feedback on our procedures and specifically on this report. The report also incorporates information received during telephone discussions with several individuals who graciously provided their time: Gary Bond (Dartmouth Psychiatric Research Center); Leslie Caplan (National Institute on Disability and Rehabilitation Research); Jean Close, Kathryn Poisal, Margherita Sciulli, and David Shillcutt (Centers for Medicare and Medicaid Services); Denise Juliano-Bult (National Institute of Mental Health); Paul Margolies, Gary Scannevin, and Liza Watkins (OnTrackNY); Sandra Miller (Delaware Division of Vocational Rehabilitation); Lisa Mueller (Edith Nourse Rogers Memorial Veterans Hospital, Massachusetts); Kim Mueser (Boston University); Michelle Mullen (Rutgers University); Tamara Sale (Early Assessment and Support Alliance [EASA], Oregon); Cara Sams (Transition-Age Youth Programs, LifeWorks Northwest); Jo-Anne Sharac (Quinsigamond Community College, Massachusetts); Luana Turner (University of California, Los Angeles); and Karen Unger (Rehabilitation Through Education). We would also like to acknowledge the program administrators, providers, other staff, and young adults from the following locations who offered their time and thoughts during our three site visits: the EASA program in Oregon, the Learning Enhancement and Resource Network in Northern New Jersey, and the University of Minnesota. In addition, at RTI International, Susan Beck, Michelle Bogus, Debbie Bond, Cathy Boykin, Mikayla Craig, Janice Handler, Loraine Monroe, Margaret Smith, and Roxanne Snaauw provided report production and editorial assistance to help enhance the quality of this final product.

The views and opinions expressed in this report are those of the authors and do not necessarily reflect the views, opinions, or policies of ASPE, HHS, RTI International, the University of Massachusetts Medical School, or the environmental scan or site visit participants. The authors are solely responsible for any errors.

Acronyms

AA	Associate of Arts degree
ADA	Americans with Disabilities Act
ASPE	HHS Office of the Assistant Secretary for Planning and Evaluation
BA	Bachelor of Arts degree
BCT	Behavioral Consultation Team
CHIP	Children's Health Insurance Program
CMCS	Center for Medicaid and CHIP Services
CMS	HHS Centers for Medicare and Medicaid Services
CVLT	California Verbal Learning Test
DMHAS	New Jersey Division of Mental Health and Addition Services
DRC	Disability Resource Center
EASA	Early Assessment and Support Alliance
EDIPPP	Early Detection and Intervention for the Prevention of Psychosis Program
ETP	Early Treatment Program
FAFSA	Free Application for Federal Study Aid
FIPSE	Fund for the Improvement of Postsecondary Education grant
FTE	Full-Time Equivalent
FY	Fiscal Year
GED	General Educational Development
GI	Government Issue
GPA	Grade Point Average
HHS	U.S. Department of Health and Human Services
ID	Identification
IDEA	Individuals with Disabilities Education Improvement Act
IEP	Individualized Education Plan
IPS	Individual Placement and Support
ISSS	International Student and Scholar Services
LAC	Learning Abroad Center
LEARN	Learning Enhancement and Resource Network
MA	Master of Arts degree
MSERP	Michigan-Supported Education Research Project
NIDRR	National Institute on Disability and Rehabilitation Research
NIMH	HHS National Institute of Mental Health
NREPP	National Registry of Evidence-based Programs and Practices
PANSS	Positive and Negative Syndrome Scale
PAWS	Pet Away Worry and Stress
PIER	Portland Identification and Early Referral program
PTSD	Post-Traumatic Stress Disorder
RAISE	Recovery After an Initial Schizophrenia Episode
RCT	Randomized Controlled Trail
RFP	Request for Proposal
RTC	Research and Training Center
RTE	Redirection Through Education program
SAMHSA	HHS Substance Abuse and Mental Health Services Administration
SCS	Student Counseling Services
SE	Supported Employment
SEd	Supported Education
SEER	Supported Education Enhancing Rehabilitation
SMI	Serious Mental Illness
SSDI	Social Security Disability Insurance
SSI	Supplemental Security Income
UCLA	University of California, Los Angeles
VA	U.S. Department of Veterans Affairs
VHA	Veterans Health Administration
VR	Vocational Rehabilitation
WIOA	Workforce Innovation and Opportunity Act

1. Executive Summary

Individuals with serious mental illness (SMI) face considerable challenges in educational attainment and employment. Supported education (SEd) presents a promising approach to address the educational goals of individuals with mental illness. The goals of SEd are for individuals with SMI to successfully be able to: (1) set and achieve an educational goal (e.g., training certificate or degree); (2) improve educational competencies (literacy, study skills, time management); (3) navigate the educational environment (e.g., applications, financial assistance); and (4) improve educational attainment.

The current project was designed to characterize the current state of knowledge about SEd as a way to assess the feasibility of conducting a demonstration of SEd for individuals with SMI. This project sought to identify key considerations in planning and preparing for a larger-scale demonstration of SEd by compiling evidence on SEd programs, identifying gaps in the knowledge base about SEd, and describing possible approaches for addressing unanswered questions about SEd. The project focused on answering a series of research questions about SEd program composition, implementation, service context, the experiences of individuals involved in SEd programs, available SEd data sources and ongoing evaluations, SEd policies, financing, and gaps in the SEd knowledge base.

Three key tasks were associated with this project: (1) a literature review; (2) an environmental scan of SEd researchers, program managers, and other key informants; and (3) site visits to three programs implementing SEd service delivery models. This final project report includes chapters describing the results from each task, as well as a final synthesis chapter that identifies future SEd needs and opportunities.

The current policy and practice landscape makes a focus on SEd interventions and supporting evidence particularly timely. The negative functional impact of SMI, particularly among young adults, is receiving increased public attention. There are several recent SEd program development and evaluation efforts, especially for individuals with first-episode psychosis. Institutions of higher education have also noted a burgeoning student population with mental health conditions. In addition, two recent policy and practice opportunities provide new possibilities for SEd program funding: the Workforce Innovation and Opportunity Act (WIOA) and the early intervention for SMI set-aside in the Substance Abuse and Mental Health Services Administration Block Grants. These policy and funding opportunities for SEd are complemented by the increased experimentation with SEd practices in the field. Consequently, findings from the current project suggest that SEd is on the cusp of widespread and sustained implementation.

1.1. Supported Education Program Model Development

Our findings suggest that the practice of SEd includes common core strategies to support individuals with mental health conditions to choose, keep, and obtain an educational goal. The literature review, environmental scan, and site visits shed light on principal issues concerning development of a model of service to support the educational goals of individuals living with mental health conditions. Findings include recognizing that variability among SEd program models is largely due to differences in service context. SEd program service settings can range from specialty mental health settings (e.g., hospitals, clubhouses, community mental health centers) to primary and post-secondary education settings and to state vocational rehabilitation (VR) agencies. Despite differences in settings and specific program characteristics, a shared set of core SEd service array components is also present across SEd efforts. Environmental scan and site visit findings demonstrate that widely different settings can successfully practice the core elements of providing educational supports. Findings also show that SEd is often integrated and delivered in tandem with supported employment (SE) services, but this integration can be beneficial and disadvantageous. Post-secondary campus settings can offer particularly unique opportunities, distinct from traditional mental health system-focused SEd services, to support students with mental health conditions directly in a college environment.

Results of the literature review, environmental scan, and site visits indicate the following needs and opportunities for the development of a fully specified, replicable, and testable model for SEd:

• Specifying SEd Core Components: Specification around the core components of SEd should be increased. This could include matching specific components to SEd activities and to measureable outcomes. A first step could include examining existing program-specific SEd manuals and various SEd efforts being implemented across the country to further operationalize components and activities.

• Identifying SEd Staffing Requirements: Staffing requirements should be further elucidated to reflect the range of education, disciplines, and training that contribute to skilled SEd staff. This should include an emphasis on specified skill sets and the capacity to support individuals with educational goals. Staffing requirements should also include developing and routinizing training supports coupled with ongoing coaching and mentoring.

• Defining SEd Specialist Tasks: The role of an education specialist needs to be clearly defined, not only specific to a standalone SEd program, but also when integrated with SE. This includes defining discrete tasks and activities associated with supporting educational goals, while also emphasizing inter-personal and relational skills that facilitate the strong relationships that are the foundation of the work between a SEd participant and a SEd specialist.

• Operationalizing SED/Individual Placement and Support (IPS) SE Integration: Strategies on how to integrate SEd and IPS SE need to be further defined and operationalized. This should include defining measurable goals and outcomes specific to educational goals and milestones, as well as strategies for staff on how to balance and integrate education and employment goals.

• Operationalizing Campus Best Practices Supporting Student Mental Health: A set of best practice guidelines should be developed to highlight successful strategies for improving campus-based supports for students with mental health conditions. Specifics should include how to secure administrative and leadership buy-in and how to partner with key campus departments (e.g., disability services), as well as more ancillary departments (e.g., travel abroad) to address student mental health. Additionally, strategies to normalize mental illness and decrease stigma on campus should be considered.

1.2. Supported Education Funding

Funding challenges to support SEd program services were commonly described within the published literature and by environmental scan and site visit participants. Environmental scan participants, in particular, noted the lack of one centralized funding strategy for SEd services. Consequently, SEd programs relied on different funding vehicles (municipal, federal, state, collegiate, and/or private corporations) that varied in terms of their stability and ultimate sustainability. Creative braiding of funding will likely be the solution to the absence of a clear funding stream, and guidance on how to properly utilize funding opportunities to meet individual client needs will aid provider organizations.

This project identified several needs and opportunities that could help sustain funding for SEd programming:

• Braided Funding Case Studies: Those working in the SEd field need to better understand how various programs across the country have and are currently braiding funding to support their SEd program activities. Published case studies that demonstrate successful braided funding strategies in support of SEd services could be widely used to help program administrators circumvent the funding challenges noted in stakeholder discussions across this project.

• Medicaid Billing Code: The availability of an SE Medicaid billing code has helped to disseminate and sustain SE approaches for individuals with psychiatric disabilities across the United States. A similar Medicaid billing option could support and extend the availability of SEd services to complement employment supports. The availability of this type of billing option would directly benefit young adults with mental illness who are highly likely to have both educational and employment goals.

• Guidelines for SE/SEd Medicaid Billing: Programs described using the SE Medicaid billing code to support the activities of SE/SEd specialists' time; however, procedures for billing joint SE/SEd program activities vary. SE/SEd program administrators could benefit from guidelines that describe how to bill SEd activities that occur as part of IPS or other SE services.

• Increased Clarity Around Medicaid Waiver Option Processes: Program administrators implementing SEd programs could benefit from enhanced clarity around the availability of Medicaid funding to support education services through the 1915(c) Home and Community-Based Services waiver option. This guidance could come in the forms of a state Medicaid director letter, program guide, frequently asked questions document, or fact sheet.

• WIOA Expansion: The recent WIOA expansion offers an opportunity for SEd program implementation and support through VR. The expanded emphasis on WIOA to address career needs of 15-21-year-olds will certainly involve supporting their education attempts. VR dollars, with their high federal match for state dollars, can incentivize SEd services for this population. There is also an opportunity to braid the dollars associated with WIOA with Medicaid to provide the rehabilitation services that are concomitantly needed.

1.3. Supported Education Research and Evaluation

The published literature on SEd research demonstrates the promise of SEd interventions to affect education enrollment among individuals with mental illness. Suggestive evidence from noncontrolled studies also indicates that individuals improve their employment and educational attainment after participating in a SEd program. However, there is a lack of controlled comparative evidence to suggest that participating in a SEd program explicitly leads to gains in educational attainment. SEd intervention research and evaluation are limited by the predominance of nonexperimental study designs, small sample sizes, and few long-term follow-up assessments of program participants. A rigorous and comparative demonstration project to determine the impact of SEd programs on educational enrollment, attainment, and ultimately employment is clearly needed.

Project results indicate the following research and evaluation needs and opportunities:

• Rigorous Evaluation and Research Designs: SEd programs demonstrate a strong ability to support evaluation studies and data collection efforts; however, existing evaluation efforts are not systematic. Rigorous evaluation and research designs are needed that capitalize on the existing SEd program infrastructure and data collection readiness.

• Randomized Controlled Trials (RCTs): Rigorous research designed to understand the impact of SEd on core outcomes of interest is needed. In particular, a well-designed RCT could help establish the evidence base necessary to move SEd from a "promising" to an "evidence-based" practice.

• Follow-Up Data Collection for 3-5 Years (minimum): Any future SEd research or evaluation trial must be designed with follow-up data collection that extends a minimum of 3 years and ideally 5 or more years from baseline to adequately capture longer-term educational degree attainment and ultimately job sustainability outcomes. Most SEd studies are limited by 1-2-year follow-ups (or less), which is an insufficient amount of time for most individuals to complete a full degree requirement.

• Large Sample Size: Larger sample sizes in SEd outcome studies are needed to analyze differences in outcomes by demographic characteristics and mental illness/symptomology. Larger sample sizes are also needed to allow sufficient power to disentangle the additional benefit of SEd to IPS approaches, separate from their impact on employment outcomes. This would not be feasible in a multisite design.

1.4. Supported Education Demonstration Project

A central goal of this study was to determine the feasibility of a SEd demonstration project. Results from our literature review, environmental scan, and site visits clearly suggest that a SEd demonstration project is needed and that the SEd field would be ready to support such a project. The implementation of many different SEd programs are described in the published literature. The environmental scan and site visit results demonstrate that several provider organizations are well poised to conduct systematic data collection on SEd processes and outcomes. Work is under way to develop SEd intervention model fidelity tools, program training manuals, and implementation guides. Also, there is general consensus in the field about what outcomes are important to measure in order to best assess SEd program impact.

Given the methodological limitations of existing SEd research, an optimally designed demonstration project would have two sequential and complementary stages. Stage 1 (6-12 months) would focus on refining and testing existing fidelity and implementation guides to support a high-quality process evaluation. Stage 2 (3-5 years or more) would include conducting a multisite RCT with long-term follow-up of program participants for 3 or more years. The field could benefit from a demonstration that is explicitly focused on the impact of a SEd-specific intervention, separate from an intervention that emphasizes employment supports. Such a demonstration project would provide the platform necessary to generate the type of evidence needed to move SEd programs from a promising practice to an evidence-based practice, thus encouraging future funding and widespread adoption.

2. Background

2.1. Educational Attainment and Employment among Individuals with Serious Mental Illness

Individuals with serious mental illness (SMI) face considerable challenges in educational attainment and employment. SMI is accompanied by a myriad of cognitive, emotional, and social difficulties that negatively affect educational performance (Souma, Rickerson, & Burgstahler, 2006). High school completion rates for youth with SMI are dismal. More than 50% of students with a mental disorder (aged 14 years or older) drop out of high school (Armstrong, Dedrick, & Greenbaum, 2003; Marder, 1992)--the highest dropout rate of any disability group (U.S. Department of Education, 2004). Students with a mental health condition also have the poorest rates of school attendance, lowest grade point averages (GPAs), and highest course failure and expulsion/suspension rates of any students with disabilities. In contrast to students in other disability groups, whose post-secondary employment rates increased significantly since the early 1990s, employment rates of students with psychiatric conditions have not improved (Wagner, 2005). Results from the U.S. Department of Education's most recent National Longitudinal Transition Study indicate that students with psychiatric conditions have a post-high school employment rate of only 50% (Wagner & Newman, 2012).

Individuals with SMI experience educational difficulties that extend to the college setting. Even when they attend college, they experience longer delays in entering college (Newman et al., 2011), and exhibit extremely high dropout rates (Salzer, Wick, & Rogers, 2008). Colleges and universities, including graduate and professional schools, are seeing a dramatic increase in the number of students with psychiatric conditions (Sharpe, Bruininks, Blacklock, Benson, & Johnson, 2004). A survey of five institutions in the Big Ten Conference revealed a 30%-100% increase in student services addressing psychiatric conditions over a 1-year period (Sharpe & Bruininks, 2003). A national survey of more than 95,700 college students across 139 institutions revealed the extent of the problems. The findings were alarming: 75% experienced a traumatic event within the previous 12 months, more than half reported that they had more than average or tremendous amounts of stress, 56% felt very lonely, 61% were very sad, and 46% felt hopeless. Roughly 35,400 students (37%) had been diagnosed with a psychiatric condition in the past year that warranted community mental health services (Collins & Mowbray, 2005).

Educational attainment is strongly linked with critical employment outcomes, such as unemployment and wage earnings (U.S. Department of Labor, 2010) and consistently predicts later employment among adults with mental illness (Burke-Miller et al., 2006; Ellison, Russinova, Lyass, & Rogers, 2008; Rogers, Anthong, Lyass, & Penk, 2006). Supported employment (SE) interventions have a long history of trying to promote the engagement of individuals with SMI in the workforce. These interventions have a strong evidence base and are well suited to provide job placement and employment support to individuals with SMI. However, even studies of SE have found that participants tend to work only part time with relatively low earnings and that job retention rates vary dramatically even after successful job placement (Becker, Whitley, Bailey, & Drake, 2007; Bond, Drake, & Becker, 2008; Campbell, Bond, & Drake, 2011; Mueser et al., 2005; Salyers, Becker, Drake, Torrey, & Wyzik, 2004). Consequently, low levels of education, vocational training, and job skills/readiness may continue to be important factors in increasing sustainable employment and promoting long-term self-sufficiency.

Promising programs to address these educational and employment challenges are emerging; however, despite some strategies with preliminary evidence, evaluation data are extremely limited.

2.2. Supported Education Interventions

Supported education (SEd) interventions focus on individuals with SMI who face challenges in achieving educational goals due to their impairment. SEd has been defined as supports "to assist people with psychiatric disabilities to take advantage of skill, career, educational and inter-personal development opportunities within post-secondary educational environments" (Collins, Bybee, & Mowbray, 1998). The goals of SEd are for individuals with SMI to successfully be able to: (1) set and achieve an educational goal (e.g., training certificate or degree); (2) improve educational competencies (literacy, study skills, time management); (3) navigate the educational environment (e.g., applications, financial assistance); and (4) improve attitude and motivation.

SEd presents a particularly promising approach and is the focus of this report. There is preliminary evidence for the effectiveness of SEd to assist individuals with identifying educational goals, to link to needed resources, and to cope with barriers to educational attainment (e.g., Cook & Solomon, 1993; Hoffmann & Mastrianni, 1993; Mowbray, Collins, & Bybee, 1999; Robson, Waghorn, Sherring, & Morris, 2010; Thompson, 2013; Unger, 1993; Unger, Pardee, & Shafer, 2000). Unfortunately, this general evidence base is limited. Two systematic reviews of SEd approaches have been published relatively recently: Leonard & Bruer (2007) and Rogers, Kash-MacDonald, Bruker, & Maru (2010). Both reviews focused on outcome studies and specifically prioritized studies that operated under controlled situations. In addition, several other articles or reports have been published that more generally summarize the state of the SEd literature (e.g., Chandler, 2008; Ellison, Rogers, & Costa, 2013; Manthey, Goscha, & Rapp, 2014; Mueser & Cook, 2012; Parrish, 2009; Unger, 2011). Generally, these reviews conclude that SEd helps individuals progress toward educational goals and increase their self-esteem and positive self-perceptions and that individuals are satisfied with services. However, across these literature summaries, authors ask for caution in interpreting results. Although many studies of SEd interventions have been published, most do not include rigorous designs and include only minimal evaluation data. Few well-controlled studies exist (Rogers et al., 2010).

2.3. Current Project and Research Questions

The current project was designed to characterize the current state of knowledge about SEd as a way to assess the feasibility of conducting a demonstration of SEd for individuals with SMI. This project sought to identify key considerations in planning and preparing for a larger-scale demonstration of SEd by compiling evidence on SEd programs, identifying gaps in the knowledge base about SEd and describing possible approaches for addressing unanswered questions about SEd.

There were three key tasks associated with this project: (1) a literature review (summarized in Chapter 3); (2) an environmental scan of SEd researchers, program managers and other key informants (described in Chapter 4); and (3) site visits to three programs implementing SEd service delivery models (summarized in Chapter 5). Findings from across these tasks, as well as the identification of future needs and opportunities for SEd programs and research are described in Chapter 6.

The project focused on answering a series of research questions about SEd program composition, implementation, service context, the experiences of individuals involved in SEd programs, available SEd data sources and ongoing evaluations, SEd policies, financing, and gaps in the SEd knowledge base. Specific research questions are described in Table 2-1 along with the approach (literature review, environmental scan, or case study) used to address each question.

TABLE 2-1. Research Questions and Proposed Analytic Approach
Research Questions	Analytic Approaches to Address
	Literature Review	Environmental Scan	Case Study
SEd Program Composition
What services/supports are included in typical SEd interventions and related programs, and how are they combined?	X	X	X
How do SEd and related programs recruit participants? How do SEd programs continue to keep participants engaged in SEd interventions?		X	X
What are the demographic characteristics of clients served by SEd programs? What are client needs and goals regarding educational attainment?	X	X	X
How are SEd and related programs staffed and managed?		X	X
What are key challenges for individuals' accessing SEd, attaining educational goals, and transitioning to employment?		X	X
SEd Program Implementation
Who are the primary partners involved with the SEd program implementation process? Which partners are most critical to running SEd programs? Are certain partners missing in SEd program implementation processes that would be helpful?		X	X
Do SEd programs engage service users in planning and developing programming?		X	X
What are the main challenges in implementing SEd and related programs? How have these challenges been overcome?		X	X
How do SEd programs measure implementation success? What metrics are important to SEd program funders?		X	X
SEd Program Service Setting and Context
How do SEd approaches differ depending on the service setting (mental health agency, VR, VA system, community college), and what are the policy implications of these differences?	X	X	X
Can SEd and related programs be disseminated through integration with SE programs available for people with SMI?		X	X
Can SEd and related programs be disseminated through integration with other interventions for people with SMI (medical homes, substance abuse treatment, state VR programs, etc.)?		X	X
What plans have been made across SEd programs to maximize the potential for program sustainability?		X	X
Experiences of Individuals Involved in SEd Programs
How did participants in SEd programs learn about the program? What do participants say keep them engaged in the program?			X
What services do participants receive through the SEd program? Are these different from education services that these individuals have received before? If yes, how?			X
Do individuals who receive SEd program services identify specific goals? Who from the SEd program supports individuals served by SEd programs? What do these individuals do?			X
Do individuals who are receiving SEd program services feel that something has changed (improved or gotten worse) since they've been in the program? What do they think facilitated this change?			X
What do individuals served by SEd programs feel has been most useful? What do they think would make things even better? What is missing?			X
Available SEd Data Sources and Ongoing Evaluations
What data sources are available to assess the impact of SEd and related programs (education, employment, program participation, health--service use and outcomes, influence of contextual factors on program impact)?	X	X	X
Can current studies be modified to address unanswered questions, or is a new demonstration recommended?	X	X
What specific outcome measures should SEd studies examine (program implementation and educational, employment, and health outcomes)?		X	X
What evaluations are ongoing and when will they end?		X
What are the key challenges to evaluating SEd and related programs?		X	X
Current SEd Evidence Base
What is the evidence on SEd program design implementation and financing?	X		X
What is the impact of SEd on client educational attainment, employment and health?	X		X
What other programs described in the literature (but not formally called "supported education") have similar objectives and designs to SEd programs?	X
Does the SEd literature identify different program impacts by psychiatric, demographic, or socioeconomic characteristics?	X
SEd Policies
What state/federal policies inform and guide SEd programs? Are there particular policies that support SEd program growth? Are there specific policies that restrict SEd program growth?		X	X
What state/federal agencies are engaged in SEd policies?		X	X
SEd Financing
How are SEd and related programs financed?		X	X
What existing and potential financing streams could be leveraged to fund expansion of SEd/related programs?		X	X
How can various funding sources be used to meet the needs of individuals, including needs for SEd services?		X	X
What are main challenges in financing SEd and related programs?		X	X
Gaps in the SEd Knowledge Base
What are gaps in the literature on SEd programs that are relevant to further program dissemination and scale-up?	X	X
What are gaps in the evidence base on SEd programs that prevent SEd from being considered an evidence-based practice?	X	X
What are important unanswered questions that are relevant to planning a SEd demonstration?	X	X
What are potential study designs to address important gaps in knowledge of SEd and related programs?	X	X

3. Literature Review

3.1. Introduction

The objective of this literature review is to complement and expand on prior reviews of the literature published around SEd interventions. Two systematic reviews of SEd approaches have been published relatively recently (Leonard & Bruer, 2007; Rogers, Kash-MacDonald, Bruker, & Maru, 2010) and several other articles summarize the state of SEd program implementation and research (e.g., Chandler, 2008; Ellison et al., 2013; Manthey et al., 2014; Mueser & Cook, 2012; Parrish, 2009; Unger, 2011). To compliment this prior work, the current literature review adds studies published from 2010 through 2014. Within these recent studies, we have placed particular emphasis on publications that include outcome-oriented trials. Second, this review includes publications before 2010 that were excluded by Rogers & colleagues (2010) and Leonard & Bruer (2007). These publications include studies that describe SEd program models (without reporting on program outcomes), process and implementation evaluations, and publications that summarize qualitative results exclusively. These publications are reviewed to help offer a description of SEd programs that are both currently being (and have historically been) used in the field. Finally, this review offers a slightly expanded definition of SEd interventions to include those that are education-focused without explicitly being referred to as "SEd."

More specifically, this report draws on the existing published literature to:

• Describe the characteristics of SEd interventions (service characteristics, populations served, financing strategies, and implementation challenges).

• Report on the impact of SEd interventions.

• Identify gaps in the published literature about SEd interventions, particularly those relevant to the feasibility and design of future demonstration activities.

3.2. Literature Review Method

This literature review was guided by the definition of SEd adopted by Collins & Mowbray (2005) in their survey of SEd programs--"a specific type of intervention that provides support and other assistance for persons with psychiatric disabilities for access, enrollment, retention, and success in post-secondary education."

Search terms for the preliminary literature search included SEd or supportive education, education OR school OR post-secondary education and (treatment or intervention), and employment and (treatment or intervention). All of these search terms were paired with mental illness, mental disorder, SMI, or psychiatric disability/disabilities. Search terms such as "education" and "employment" were also used to broaden the literature reviewed to potentially include interventions focused on post-secondary education support and intervention that may not have been labeled explicitly as "supported education." Search engines used included PubMed, the Web of Science (includes Science Citation Index Expanded and Social Sciences Citation Index), PsycINFO, and the Education Resources Information Center. Both peer-reviewed publications and gray literature (e.g., government or university-published reports) were included in the literature review. The search was limited to articles written in English and published from 1990 to November 2014. Articles published outside of the United States were included. In addition to these keyword searches, we examined citations contained in each article and citations from key SEd review articles to identify other potential articles to include in the review. Please note that we did not include unpublished articles in this review. Unpublished work, conference proceedings, or manuscripts in press will be reviewed within a subsequent report resulting from our environmental scan.

This keyword search and supplemental article review identified 150 abstracts for consideration. In reviewing these abstracts, we excluded 75 publications, because the study:

• Focused exclusively on SE or employment without an education component.

• Examined a traditional occupational therapy intervention, "wellness" education programs, or other psycho-educational programs (e.g., programs designed to help manage symptoms).

• Included only children.

• Included only a single case example (i.e., one consumer's story).

• Was only theoretical, without any emphasis on SEd program or evaluation data.

Acceptable SEd studies included descriptive program model summaries, original research (both outcome and process evaluations), and review articles. For original research publications, we included pre/post evaluations, correlational studies, experimental studies, and quasi-experimental studies.

After applying the exclusionary criteria, 75 publications were left for consideration. We did not carefully re-analyze the 13 SEd outcome studies published from 1989 to 2009 and included in the systematic review conducted by Rogers & colleagues (2010). Instead, a synthesis of the Rogers review is included within this report, along with a summary of some seminal studies. We did, however, review several articles published from 1989 to 2009 that were not included in the Rogers review, likely because of their focus on program model descriptions, process evaluations, or qualitative research. We also identified 31 articles that had been published on SEd since 2010; 16 of these were original research studies designed to examine the impact of a SEd intervention. We have placed particular emphasis on these 16 studies, not all of which included equally rigorous designs and study methods. Some of these publications describe preparations for a SEd trial or characteristics of participants currently involved in an ongoing trial. The strengths and weaknesses of these recent SEd studies will be discussed in this chapter.

3.3. Characteristics of Supported Education Programs and Their Participants

Several publications on SEd summarize the characteristics of the programs themselves and their participants. This report section describes the populations typically served by SEd programs; traditional SEd models, settings, services and staffing; financing; efforts to integrate SEd approaches with SE and related programs. In 2011, the U.S. Department of Health and Human Services (HHS) Substance Abuse and Mental Health Services Administration (SAMHSA) published a toolkit for SEd programs to structure and guide SEd program implementation. Appendix B of this toolkit provides a Supported Education Fidelity Scale developed by researchers at the University of Kansas, as well as a scoresheet for programs to examine adherence to key aspects of the SEd program model.

3.3.1. Participants in Supported Education Programs

Eligibility criteria for individuals served in the SEd programs described in the published literature vary slightly across programs. All programs require participants to have some history of psychiatric disability without any age restriction. Some programs go further to require specific a specific duration period (e.g., "for 12 months") for the mental illness while others target individuals experiencing a first-episode of mental illness or psychosis. Several programs describe that program participants were required to have an interest in pursuing post-secondary education, basic English fluency, and a willingness to utilize mental health services. Some programs also require that participants be actively enrolled in mental health treatment, even sometimes requiring adherence to a medication regimen (e.g., Gutman, Kerner, Zombek, Dulek, & Ramsey, 2009). All programs had some prior education eligibility criterion, but this criterion differed slightly across programs. Some programs required participants to have a high school diploma or General Educational Development (GED) (or to least have them near completion; e.g., Collins et al., 1998), whereas other programs did not have this requirement and described active work with participants to acquire GEDs (e.g., Hain & Gioia, 2004). Some programs explicitly stated that participants needed to show no evidence of a significant drug or alcohol problem (Gutman, 2008), no pre-morbid history of mental retardation or neurological disorder (e.g., Nuechterlein et al., 2008a). Another program excluded individuals with unstable housing or homelessness and those lacking a support system (e.g., Hutchinson, Anthony, Massaro, & Rogers, 2007). Another excluded individuals with a history of violence (Holter & Paul, 2004).

Looking across SEd programs that were operating at that time, Mowbray & colleagues (1996) noted that SEd participants tended to be younger, more educated, and higher functioning than individuals with SMI from more general non-SEd program samples. For example, many participants in the Michigan-Supported Education Research Project (MSERP) had significant problems with mental health symptoms, social skills deficits, and histories of substance abuse; however, these issues did not prohibit participants from being able to stay involved in the SEd program (Collins et al., 1998). Unfortunately, the SEd program outcome literature is too premature to conclude which types of individuals are best positioned to benefit from SEd approaches.

Some recent SEd approaches have adapted and tailored SEd programs to better fit special populations. For example, Shor & Aivhod (2011) describe the rehabilitation beit midrash adaptation of a SEd program that maintains the principles and practices of psychiatric rehabilitation while implementing the approach in a culturally oriented context. All program participants were men, 70% of whom lived in rehabilitative residential facilities and were Orthodox or strictly Orthodox Jews. This descriptive article discusses using Judaic program content and values as a method to advance the rehabilitation process and enhance program participants' sense of belonging and inclusion. As another example, Smith-Osborne (Smith-Osborne, 2012a, 2012b) describes the design, development, and adaptation of a SEd program specifically for veterans. Adaptations were made based on a participatory action research approach that worked to engage stakeholders in the community, U.S. Department of Veterans Affairs (VA), and higher education settings. Program components are modified to reflect the veteran student context. For example, veterans share a house (including students and nonstudents), rather than participate in a more traditional rehabilitation housing program. Budgeting includes VA disability pension instead of Supplemental Security Income (SSI) benefits. This program's impact is currently being tested in a randomized controlled trial (RCT).

3.3.2. Supported Education Program Models

There are several different approaches to SEd, each designed to help individuals with SMI succeed in the post-secondary education environment. These approaches vary according to their setting location, service array mix, and integration with the mainstream post-secondary education environment.

Since the early 1980s, post-secondary institutions and mental health providers have developed SEd programs. Historically, some of these models have been "owned" and developed via leadership within the college system, whereas others have their origins and leadership from the mental health specialty system. The earliest SEd models were classroom-based (Walsh, Sharac, Danley, & Unger, 1991); however, with federal grant funding, SEd models were expanded from 1989 to 1994 to be implemented in a variety of settings (e.g., hospitals, mental health agencies, clubhouses) (Unger, 1998). On-site and mobile support models have now been added to these traditional, self-contained, classroom-based models. Federal grant funding also promoted the use of clubhouses across the United States to disseminate SEd via a free-standing organization (separate from the education or mental health systems).

In one of the first classifications of SEd programs, Unger (1990) characterized three different types of SEd program models:

• Self-Contained Classroom Model: Students with psychiatric disabilities attend closed, self-contained SEd classes on-campus (but separate from mainstream post-secondary classes). Classes typically use a structured curriculum and are time limited. Students are not initially integrated into regular classes, but they may participate in the activities and use the institution's resources. However, support is available from program staff for students as they progress and move into regular classes. Education specialists may be from the sponsoring program or the academic institution.

• On-Site Model: These models are sponsored by a college or university at which SEd services are provided in an individual rather than a group setting. Students attend mainstream post-secondary education classes. Support services are typically made available to all students with disabilities and are enhanced by adding specialized mental health staff or a peer support group. The education specialist works exclusively at one site and typically has an office on the campus or program site from which he or she provides support services (see description of this position in Ellison et al., 2014).

• Mobile Support Model: Students attend mainstream post-secondary education classes of their choice, but SEd services are provided by an agency (typically a mental health agency) external to the education facility. The SEd education specialist office is at the mental health agency. SEd program staff provide support, assistance, and problem-solving in an individualized, flexible way wherever this support is needed. The SEd education specialist travels to meet the student in the mental health agency, community, or campus or education program site.

SEd program models continue to grow and expand over time. Consequently, these historical, individual classifications have become less and less useful. SEd programs are becoming more eclectic as discovered by Mowbray, Megivern, & Holter (2003b) in their survey of SEd programs being implemented across the United States. This survey found no SEd programs that operated with only a classroom model. Meanwhile, the majority (66%) of programs were offered through clubhouses. The clubhouse model is typically a support program designed for people with serious and persistent mental illnesses. Participants are considered "members" (as opposed to "patients" or "clients"), and activities are recovery oriented and strengths based. Because of the number of clubhouse-based SEd programs, Mowbray & colleagues (2003b) added some other classifications of SEd program models to those originally developed by Unger (1990):

• Clubhouse Full Model: These SEd programs are located at clubhouses and offer individual counseling (either by staff or peers). The full clubhouse model provided 0.5 full-time equivalent (FTE) or greater staff devoted exclusively to post-secondary education (excluding GED services), an educational unit in the clubhouse, and at least two services beyond individual counseling (e.g., mentors/tutors, educational software programs, group support, education liaisons, transportation services, recruitment/outreach). These services could be mobile.

• Clubhouse Partial Model: These SEd programs are located at clubhouses that focus on post-secondary education with fewer services than the full model (e.g., less than a 0.5 FTE staff person, only one service offered beyond individual counseling).

• Free-Standing Model: This model provides some component of its services on a college campus or provides mobile services but also includes services off-site at a central office. Free-standing programs offer two services beyond individual counseling with 0.5 FTE or greater staff focused on post-secondary education.

Some of the diversity represented by SEd programs described in the literature can be seen in Table 3-1. This is not an exhaustive list but offers a few examples of SEd program models discussed in the literature.

As shown in Table 3-1, some older models described in the literature strictly follow a traditional classroom-based model. For example, the Redirection Through Education (RTE) program established in 1973 in Toronto, Canada, offers self-contained for-credit and noncredit classes taught by program-hired faculty. Course completion leads to a program-specific graduation certificate. Meanwhile, other programs mix model approaches. However, these self-contained models are now rare in the United States. Consistent with Mowbray and colleagues (2003a), several SEd programs now integrate various model aspects into their program approaches (e.g., on-site and classroom-based). A few examples of integrated approaches include Laurel House, the Bridge Program, and Supported Education Enhancing Rehabilitation (SEER). Laurel House (http://www.laurelhouse.net/) is a clubhouse program written about in the late 1990s that offered social, vocational, and residential services to people with a history of psychiatric hospitalization. This model includes a mixture of the free-standing model (classes and support services were located in the clubhouse) but also included aspects of on-site support (service supports were also provided on-campus) (Dougherty et al., 1996). A more recently established program, the Bridge Program, offers 12 modules of self-contained classes on site at Columbia University. Students were then offered 6 weeks of on-site mentoring and support at Columbia University from occupational therapists to facilitate their integration into mainstream education courses or subsequent employment (Gutman, 2008). Meanwhile, the SEER program operated out of Spokane, Washington, described offering on-site classes at a community college, along with mobile support that follows enrolled students wherever they choose to pursue their education or employment goals (across the entire country, not tied to a specific post-secondary institution) (Hain & Gioia, 2004).

TABLE 3-1. Examples of SEd Program Models, Names, and Supporting Citations
SEd Program Model	Model Description	Sample Program Name, Setting, and Citation
Classroom model	Students attend closed, self-contained SEd classes on-campus (but separate from mainstream post-secondary classes).	RTE (Gilbert, Heximer, Jaxon, & Bellamy, 2004; Kidd et al., 2014)
On-site	Students attend mainstream post-secondary education classes sponsored by a college or university where SEd services are provided in an individual (not group) setting.	Houston Community College System (Housel & Hickey, 1993) California Community Colleges System (Jacobs & Glater, 1993) Mott Community College (Unger, 1990) Bridge Program, Columbia University (Gutman, 2008)
Mobile support model	Students attend mainstream post-secondary education classes, but SEd services are provided by an agency (typically a mental health agency) external to the education facility.	South Beach Psychiatric Center in New York (Lieberman, Goldberg, & Jed, 1993) Thresholds Community Scholars Program in Chicago (Cook & Solomon, 1993) Spruce Mountain Inn (Unger, 1990)
On-site and mobile support model	A combination of the on-site and mobile support models.	SEER community college program in Spokane, Washington (Hain & Gioia, 2004)
Free-standing model	Provides several services off-site at a central free-standing office.	Unnamed mental health clinic-based program in Quebec, Canada (Beguet, Fortier, & Gauvin, 2004)
Free-standing and on-site	Provides some service components on a college campus or provides mobile services but also includes services off-site at a central free-standing office.	Laurel House (clubhouse model) (Dougherty et al., 1996) On-campus services with county mental health agency support (Thompson, 2013)

3.3.3. Supported Education Program Settings

The behavioral health care system for individuals with SMI is complex and involves multiple sectors. Service sectors that provide support for educational and employment outcomes include specialty mental health, primary and post-secondary education, vocational rehabilitation (VR), and the Veterans Health Administration (VHA) service systems. Examples of SEd program approaches are used in all of these settings.

Mowbray & colleagues (2003a) conducted a national survey of known SEd programs across the United States. This survey found that the majority of SEd programs (66%) were offered through clubhouses; these clubhouse programs were extremely varied in terms of the amount and diversity of service approaches. Other than clubhouse-based approaches, the next most common setting was within a post-secondary institution. Mowbray & colleagues (2003a) did note a handful of SEd programs that were not located at either a clubhouse or university-based site (i.e., mental health agency/provider).

Because many SEd programs are often directly affiliated with a community college or university setting, Collins & Mowbray (2005) conducted another national survey. This time, they surveyed campus disability service directors and queried these post-secondary schools about the presence of SEd programs. According to the survey, most of the campus-affiliated SEd programs were located off campus (72%). Most of these SEd programs were managed by a mental health agency (68%), but some were operated by a clubhouse or vocational program (12%) or college or university (19%), or they were located in another setting (24%). The majority of these campus-affiliated programs focused on both post-secondary school enrollment and retention (58%), as opposed to solely enrollment (16%) or retention (26%). The average number of people enrolled at one point was 32 (standard deviation=50), with a median of 10.

More recently, efforts to integrate SEd programs with SE have been led out of the specialty mental health system (e.g., Killackey, Jackson, & McGorry, 2008; Nuechterlein et al., 2008a), with services often offered both on-site, in free-standing mental health agencies, or with mobile support functions. Moreover, a recent review by Smith-Osborne (2012a) described almost 15 different SEd programs providing education services and supports to veterans.

3.3.4. Supported Education Program Service Array

In practice, SEd program service features vary widely. In 2004, Waghorn & colleagues identified ten features of SEd programs. In our review of the SEd literature, we continue to find these core services offered within the context of SEd programs. In addition to these ten service components, some SEd program models now offer post-graduation employment transition support (e.g., Hutchinson et al., 2007) and work with family members to increase program engagement (e.g., Nuechterlein et al., 2008a). The ten features of SEd programs are as follows:

1. Service coordination with professionals outside of the SEd program.

2. Specialized career counseling, including vocational planning and exploration.

3. Specialized, program-trained staff with time allocated explicitly to SEd programs.

4. Financial assistance.

5. Skill building to facilitate integration into the academic environment, including stress and time management and academic or study skills training.

6. On-campus information about student rights and resources.

7. On-campus or off-campus mentoring and support, individual or group support, or peer support.

8. Coordination with post-secondary education institutions to facilitate course access or within-course assistance.

9. Access to tutoring, library assistance, and other forms of supplemental educational support.

10. General support (off-campus preferred) for the multiple individual barriers and life stressors that can lead to educational attrition.

A particularly common element for most SEd interventions is the presence of an individual whose job is to focus on educational goals, sometimes called an education specialist. This individual works with the program participants to identify educational goals, assist in enrolling appropriate courses, and follows up with participants to troubleshoot problems and offer supports over the course of their study (Ellison et al., 2013; SAMHSA, 2011). This education specialist-type service is most often paired with more general mentoring and support and skill building activities. Unfortunately, research on SEd interventions has not progressed to the point of being able to offer explicit guidance about what type of a SEd service array is most appropriate for clients with certain needs profiles and educational goals.

Some recently published studies have considered mechanisms and services to enhance the impact of SEd programs. One particularly interesting approach has been led by Kidd & colleagues (2012a, 2012b, 2014). This team has conducted a series of trials to examine the impact of supplementing a SEd program (RTE) with a cognitive remediation program for young adults. Cognitive remediation is a type of treatment intended to improve difficulties with attention, memory, information processing speed, problem-solving, organization, and planning. The SEd program includes remedial skills training in English fluency, study skills, and other noncredit courses. Counselors also assist students with learning difficulties and stress management. The cognitive remediation program lasts 10 weeks, with 20 computer-based, 45-minute cognitive exercise sessions held twice per week using the COGPACK program. COGPACK is a computerized remediation program targeting improved executive functioning, such as verbal learning and processing speed, among individuals with schizophrenia. COGPACK sessions cover attention, psychomotor speed, learning and memory, and executive functions. In the RCT comparing SEd alone versus SEd with cognitive remediation, there was no evidence that cognitive remediation facilitated improvement in cognition above and beyond gains in sustained attention and vigilance associated with SEd alone (Kidd et al., 2014).

Components of the Supported Education Fidelity Scale and scoresheet examine many key aspects of the potential SEd service array, including individualized post-secondary school enrollment supports, resources for students enrolled in academic institutions, knowledge-building activities, the establishment of an educational assessment and goal-setting process (see SAMHSA [2011] for more complete operationalized definitions of each component). The presence of this tool will allow future SEd evaluation and research protocols to better account for variation in SEd program service arrays.

3.3.5. Integration of Supported Education and Supported Employment Program Approaches

Researchers (e.g., Evans & Bond, 2008) have suggested that SE models may be appropriate service delivery mechanisms for providing SEd services. Attempts to integrate SEd and SE service models, particularly within mental health centers, represent a recent shift and emerging area of SEd research. Some recent publications include specific examples of integrating SEd principles and services into SE approaches. Example programs vary from basic training in Microsoft Office-type computer skills (Hutchinson et al., 2007) to a more fully integrated SEd/SE approach (Nuechterlein et al., 2008a). These models typically take place within a mental health agency, in the context of Individual Placement and Support (IPS), and with young adults with psychotic or related disorders (Rinaldi, Perkins, McNeil, Hickman, & Singh, 2010; Robson et al., 2010). The IPS model was designed as a standardized approach to SE for individuals with SMI (Drake, 1998). It consists of six evidence-based principles for SE or SEd, which are as follows: a goal of competitive employment (or educational attainment for SEd), rapid job search (or rapid enrollment in school for SEd), integration of rehabilitation and mental health, attention to consumer preferences, continuous and comprehensive assessment, and time-unlimited support (Bond, 1998). There are now two RCTs designed to examine the impact of a SEd program integrated or combined with SE (specifically IPS) compared with usual services (Killackey et al., 2008; Nuechterlein et al., 2008a). Only preliminary outcomes are available at this time; other results will be forthcoming.

Hutchinson & colleagues (2007) describe the Training for the Future program at Boston University's Center for Psychiatric Rehabilitation. This program offers a 10-month, classroom-based program that teaches computer skills. After completing the program, students participate in a 2-month unpaid internship program while taking a seminar focused on work skills. After the internship, students are provided with individual job development and employment support for as long as needed. In a repeated measures, time series pre/post evaluation design (with measurements at baseline and 3-month, 6-month, 12-month, and 18-month follow-up), this program approach demonstrated increases in participants working for pay or as volunteers from baseline to 18 months, increases in hours or work per week, and increases in mean earnings per month (among working participants). The program also found a significant linear decrease in program participants' report of mental health and rehabilitation services used over time (Hutchinson et al., 2007). Participants also reported positive gains over time in standardized measures of self-esteem and empowerment.

In a more comprehensive, integrated approach, Nuechterlein, Subotnik, Turner, & colleagues (2008a) describe an interesting model in which the IPS/SE model is being extended to include SEd for individuals with first-onset psychosis. Extending SE models to include SEd may be particularly critical for transition-age youth and young adults with first-onset mental illness. The next section of this report provides more information on extending SE models to include SEd for this very specific subpopulation of young adults.

Combined SEd and SE approaches may be more common than originally realized. Manthey, Holter, & colleagues (2012b) conducted a survey of IPS/SE programs to understand which elements of SEd services were perceived as valuable and what educational services were being provided by the programs. IPS program respondents most highly valued the provision of concrete educational services and services to minimize educational barriers for program participants. The majority of programs surveyed (approximately 57%) provided some type of educational service and support. The authors suggest that the number of SEd services provided by IPS/SE programs may have been underestimated by previous SEd-oriented surveys (e.g., Mowbray et al., 2003a) because these programs were not formally being called out explicitly as SEd programs. Integrated SEd and IPS/SE services may be feasible and may enhance the impact of either approach offered in isolation. Outcomes such as those that will be produced by the larger trial being conducted by Nuechterlein, Subotnik, Turner, & colleagues (2008a) will be helpful in understanding the impact of this combined approach.

Supported Education for First-Episode Psychosis

A combined SEd and SE model in first-episode psychosis cases may help to intervene more effectively and prevent chronic work disability status. Given that the first episode of schizophrenia typically occurs from the late teens to mid-20s, it is common that this episode will interrupt an ongoing educational experience. Nuechterlein, Subotnik, Turner, & colleagues (2008a) argue that it is a logical and developmentally appropriate step to resume an educational goal for participants who desire to do so. These desires can be seen in the Nuechterlein et al. (2008a) SEd/SE treatment group choices: 36% chose to pursue school alone, 31% chose to pursue jobs alone, and 33% chose to return to both school and jobs (most typically starting with school and adding a part-time job).

The combined IPS/SEd model tested by Nuechterlein et al. (2008a) includes a preliminary evaluation of the participant's employment or educational goals; a specialist who works to find placement either in an educational or employment setting; and support services during the participant's course of study to provide coordination with teachers, course planning, and study skills aid. These SEd services occur in tandem with more traditional SE activities. Nuechterlein, Subotnik, Turner, & colleagues (2008a) importantly note that traditional SE approaches have most often focused on chronically ill individuals. The program also encourages participants to tailor school and work to their preferences and abilities--20% choose GED credentialing programs, 60% chose community colleges, and 20% choose 4-year colleges. Preliminary findings suggest that 83% of people with recent-onset schizophrenia who received the intervention had returned to regular paid work or school during 6 months of intensive treatment as compared with 41% in the control group. Outcomes from this RCT are currently being analyzed, with results forthcoming. This approach represents a promising adaptation of SEd for first-episode mental illness.

Killackey, Jackson, & McGorry (2008) conducted a small RCT (sample of 41 individuals) that integrated some features of SEd into an SE program for people with first-episode psychosis. Program developers indicated that some integration of education components into the SE were merited due to the fact that many participants with first-episode psychosis had their educational experiences interrupted. Many program participants described having educational goals either separate from or in addition to employment goals. Killackey et al. (2008) found that this intervention approach led to greater employment and more class completion than usual care. Follow-up analyses to this study showed that no individual-level characteristics were associated with employment and education outcomes other than the program (SEd/SE vs. usual vocational and educational services) assignment (Baksheev, Allott, Jackson, McGorry, & Killackey, 2012). Education outcomes here were described as "studying or entering a course of education." Although the sample size was small, this study and those previously described provide suggestive evidence that an integrated SEd/SE approach may be helpful, particularly for those experiencing a first psychotic episode. Unfortunately, these programs do not describe the process of integrating SEd and SE interventions into one approach. Consequently, it is difficult to understand which SEd services components are specifically incorporated into SE intervention approaches and how these are implemented in the field.

3.3.6. Supported Education Program Financing

Historically, funding for SEd services has been from a mixture of federal, state, local, and foundation sources. Primary funding sources tend, in part, to be driven by the SEd program setting and owning organization. For example, in the survey by Mowbray & colleagues (2003a), most clubhouses received funding from the state or county mental health agency. Secondary funding sources from clubhouses were often VR dollars and foundation grant funding and were generated through independent fundraising. Meanwhile, on-site models in the Mowbray & colleagues (2003a) survey received funding from even more sources, including colleges or universities, state/county/city mental health agencies, VR, foundations, and United Way. All of the free-standing programs received largely mental health funding.

As detailed by Holter & Paul (2004), acquiring state education funding for SEd programs is particularly complicated. Education funding is typically divided between the U.S. Department of Education for kindergarten through 12th grade services (necessary if a SEd program provides GED service support) and the state Board of Regents (necessary if a SEd program provides adult education support). The U.S. Department of Education typically issues payments based on the headcount of students on a single day of the school year; meanwhile, the state Board of Regents may have a lengthy application that results in a calculated funding formula. Funding for special education can flow through both sources. Programs at locations such as clubhouses are not easily categorized into a secondary or post-secondary institution framework, so state funding is extremely difficult to access.

The complex funding strategies necessary to support SEd programs over time can be seen in a few published program histories. For instance, the MSERP was initially federally funded for 3 years and then moved to a combination of state and local mental health agency funding (Collins et al., 1998). In another example, Hain & Gioia (2004) describe the complicated funding history of the SEER program in Spokane, Washington. The State of Washington originally had a mandate indicating that SEER be dually maintained and funded by the Division of Vocational Rehabilitation and the local mental health community. However, original program funding was even broader based--provided by the community college system, the public mental health system, VR, and the state mental health division. Over time, however, many of these funding sources disappeared; at the time of the article's publication, 70% of program funds were from the community college system, and 30% were from the county public mental health system (Hain & Gioia, 2004). Even when SEd program publications do not describe funding sources in detail, authors often describe funding issues as an implementation and sustainability challenge.

In 2014, Manthey, Goscha, & colleagues described seven ways in which SEd programs strive to create service funding:

1. Reallocate resources from other programs to provide services.

2. Braid funding from municipal, federal, state, collegiate, and private corporations.

3. Secure grant funding for short-term support while deferring costs through cross-agency collaboration.

4. Perform general fundraising activities.

5. Defray SEd program costs by subsidizing SEd through SE funds.

6. Use fee-for-service schemes.

7. Fund the program through Community Mental Health Services Block Grants.

To facilitate funding for SEd services, Manthey, Goscha, & colleagues (2014) recommend that funders lift some key funding barriers to help ease SEd program implementation and dissemination: (1) remove caps on billable hours for SEd services; (2) create guidelines to allow specialty mental health centers to bill Medicaid for SEd; (3) create specific guidelines to allow SEd programs to be billed as part of SE services; and (4) encourage increased use of peer support-run SEd services while allowing SEd services to be billed through peer support channels. Sustainable and consistent funding sources continue to impede program growth and evaluation.

3.3.7. Other Programs Similar to Supported Education in the Literature

Most standalone post-secondary education interventions for individuals with psychiatric disabilities reference their approach specifically as "supported education." However, we did find some publications with interventions targeting secondary students (high school, transition-age youth) where the approach was not necessarily defined as "supported education," but where the intervention had a similar service array. For instance, the Portland Identification and Early Referral (PIER) program focuses on helping secondary school students with psychiatric disabilities and their family members better understand mental illness. The program describes the use of strategies to help students complete secondary education and enter post-secondary education settings or employment (Downing, 2006). Another secondary school approach administered occupational therapy in the public school system for children with emotional disturbances. That approach was designed to enhance learning and promote high school degree completion (Chandler, 2007). We did not extensively review these types of secondary school approaches; however, we wanted to note their presence in the literature.

Another way in which SEd approaches are noted in the literature, but not explicitly labeled as "SEd programs," was when these approaches were included in a very broad array of integrated education and employment support services. Many early intervention programs for individuals experiencing a first episode of psychosis include SEd components, without explicitly being named as SEd programs. For example, a multisite RCT is currently being conducted by McFarlane & colleagues (McFarlane et al., 2014) involving young adults at risk for schizophrenia and psychosis. This trial is designed to examine the impact of the Early Detection and Intervention for the Prevention of Psychosis Program (EDIPPP) (McFarlane et al., 2012), which examines the effectiveness of a PIER-based program across the United States. EDIPPP includes a SEd program that is bundled with an array of other family-based services and supports (McFarlane et al., 2014). The focus of this intervention is on the early identification, treatment, and prevention of psychosis among young adults (and not solely post-secondary education enrollment). Consequently, this type of trial does not explicitly examine SEd program outcomes, but represents the integration of SEd approaches into a broader mental health intervention.

3.3.8. Implementation Challenges

Issues related to program implementation are often described in published SEd research and evaluation studies. Client-level implementation issues and challenges include participation, attrition, and hardships facilitating professor-student relationships. Barriers to program participation were more commonly described by program participants with moderate participation rates than individuals with high program participation rates (as cited in Rogers, Kash-MacDonald et al., 2010). Participation rates vary related to participants' substance abuse behaviors, number of hours worked for pay, quality of life, and size of social network (as cited in Rogers, Kash-MacDonald et al., 2010). Positive client-level outcomes result when program staff are able to facilitate effective partnerships between students and their instructors (Cook & Solomon, 1993). The ability of the case worker to disclose as generally as possible about the student to the professor enhanced the chances of school success (Nuechterlein et al., 2008a). The number of staff providing mobile support per client may also need to be considered; as Cook & Solomon (1993) noted, more than one staff person is needed to provide adequate mobile support.

SEd program systems-level implementation issues and challenges are also described in research and program evaluation publications. These issues largely focus on developing a positive working relationship with the staff or faculty within the school community where services are provided and building up the capacity of mental health services at the educational institution, enabling students to have strong contact with their mental health services provider. Suggestions for building relationships with educational institutions include conducting in-service training and liaison activities with post-secondary faculty (Cook & Solomon, 1993); having a positive relationship with the representative of the community college, which is necessary to maintain the SEd program as a high priority (Mowbray, 2000); contacting the disabled student services office before the start of the first day of class, instead of waiting for a problem to arise (Nuechterlein et al., 2008a); and collaborating with consumer-run programs and regularly presenting about the SEd program and what it can do for its clients (Mowbray, 2000). Most educational institutions are ill equipped to provide the treatment and resources that students with SMI require, so mental health programs need to initiate SEd programs (as cited in Unger, 2011; Unger, Pfaltzgraph et al., 2010; Wagner & Newman, 2012) because students who are able to maintain contact with mental health services have a higher retention rate than those who are unable to maintain contact (as cited in Unger, 2011; Watkins, Hunt, & Eisenberg, 2012).

Ellison & colleagues (Ellison et al., 2014) describe program modifications that were made to add a SEd component to an IPS-SE model for implementation with an emerging adult population (17-20 years). In particular, early feasibility testing revealed the need to have a separate educational specialist position (in addition to the already existing employment specialist). The program offered both an education and employment-oriented program track; however, enrollment in the education track was below expectations. An education specialist was added to be a resource for education-related needs; SEd program participation increased. This program also used peer mentors but noted challenges in keeping peer mentors consistently employed. They eventually went with older peer mentors (ages 28 and 30) "who had lived experience, but were far enough along in their own development and recovery to maintain strong boundaries with participants" (Ellison et al., 2014).

3.4. Synthesis of Prior Review Findings on the Impact of Supported Education Interventions

Three particularly comprehensive reviews of SEd studies have been published. The first review was written by Mowbray & Collins (2002) and summarized publications up to 1996. The second review was published by Leonard & Bruer (2007), with a particular focus on implications for psychiatric hospitals and other mental health facilities. Most recently, Rogers, Kash-MacDonald, Bruker, & Maru (2010) conducted a systematic review of SEd publications from 1989 to 2009. Rogers &d colleagues (2010) focused on study designs intended to examine the impact of SEd programs. Interestingly, the Leonard & Bruer (2007) publication included no papers authored by Mowbray & colleagues. Meanwhile, Dr. Mowbray authored or co-authored seven of the 13 articles reviewed by Rogers & colleagues (2010). The Rogers review is the most recent and by far the most systematic; consequently, this review is summarized as follows.

3.4.1. Articles Reviewed by Rogers and Colleagues (2010)

Rogers & colleagues (2010) summarized the results of 13 articles published between 1989 and 2009. All articles were reviewed by three individuals and separately rated for the quality of their research methods. The review article individually summarizes each article: its findings and its methodological strengths and weaknesses. Seven of the 12 publications reviewed by Rogers & colleagues (2010) were conducted by Mowbray & colleagues using the MSERP study dataset. In fact, Rogers & colleagues (2010) note that the number of articles published on the MSERP dataset skews the findings toward one model and obscures "the number of alternative models which have not been adequately tested" (p. 8). The articles included in Rogers & colleagues' (2010) review and their associated study designs are described in Table 3-2.

TABLE 3-2. Study Designs for Publications Reviewed by Rogers & Colleagues (2010)
Study Design	Program Name	Citations
Experimental (RCT)	MSERP	Collins et al., 1998; Collins, Mowbray, & Bybee, 1999a, 1999b
Quasi-experimental (comparison group)	Program not named	Hoffmann & Mastrianni, 1993
Correlational	MSERP	Collins et al., 1999b; Collins, Mowbray, & Bybee, 2000; Mowbray, Bybee, & Collins, 2001; Mowbray et al., 1996
Pre/Post	Continuing Education Project, Thresholds Community Scholars Program, others not named	Best, Still, & Cameron, 2008; Cook & Solomon, 1993; Unger, Anthony, Sciarappa, & Rogers, 1991; Unger & Pardee, 2002; Unger et al., 2000

3.4.2. Seminal Experimental or Quasi-Experimental Studies Published before 2010

The systematic review conducted by Rogers & colleagues (2010) found only two SEd trials that the researchers considered "rigorous": one an experimental RCT (Collins et al., 1998) and the other a high-quality quasi-experimental trial (Hoffmann & Mastrianni, 1993). These studies continue to stand as seminal works in the field.

Collins, Bybee, & Mowbray (1998). The only RCT of SEd was published by Collins, Bybee, & Mowbray (1998). This study included 397 participants. Participants were recruited from the Detroit metropolitan area and primarily came from the public mental health system. Some came from self-help programs and advocacy services, and others were recruited by word of mouth. These participants were enrolled in one of two experimental conditions (a classroom intervention and a group support intervention) or a control condition (where individuals were given the name of a support person to contact with questions). Both the classroom and group model had meetings twice a week for 2.5-hour sessions (for 14 weeks). The classroom model had two instructors and a curriculum that covered managing the campus environment, career exploration, and managing stress. The group model had two facilitators; one was a mental health consumer. Groups were designed to explore career and education options and make meaningful, individualized decisions. All participants received an information packet covering assistance in obtaining VR services, facilitated access to special student services and advising, on-site mentorship, and access to contingency funds for assistance with short-term, school-related expenses.

Collins & colleagues (1998) found that participant satisfaction was significantly higher among those participating in the group model than among those in the control group. Participation did vary significantly across the three groups, with the highest participation rates in the group model condition. Authors examined participation rates and found that 35% of those with high participation rates in SEd programs enrolled in college or vocational services compared with 23% of those with no participation in SEd, a significant positive effect (Collins et al., 1998). This was the only significant finding that resulted from comparisons across the three groups. There were nonsignificant differences among the three conditions on having taken college or vocational education classes since baseline and on work status. In a long-term follow-up of this same sample, the percentage employed or enrolled in school increased significantly, from 24% to 39%, for those in a classroom SEd model (Mowbray et al., 1999).

Unfortunately, Mowbray & colleagues' (1999) study failed to use an intent-to-treat model and instead analyzed only the data available from those subjects who completed the post-test assessment. This is particularly problematic because the study had 26% of participants (104 individuals) drop out between baseline and post-test. The participants who dropped out during the course of the trial are also not separated by condition (two treatment conditions versus control). This methodological flaw makes the study's outcomes difficult to interpret.

Hoffman & Mastrianni (1993). The only quasi-experimental study of a SEd program published before 2010 examined a SEd intervention conducted within an inpatient psychiatric hospital. Hoffman & Mastrianni (1993) compared the outcomes of participants in this SEd program with those of patients from a matched psychiatric hospital with a more traditional approach to inpatient treatment. The SEd program integrated academic goals and opportunities into those typically available in regular treatment. Individuals in the SEd program also participated in special academic activities in partnership with a community college. The SEd intervention group had a higher rate of college enrollment (69%) than did the comparison group (47%). And, of those who enrolled in college, SEd participants (88%) were more likely than the comparison group (58%) to return to school full-time or progress from part-time to full-time in school. Unfortunately, this study had several methodological problems. First, subjects in this study were not randomized to treatment conditions, and there were some notable differences between groups. For example, 37% of the participants in the comparison group had primary Axis II diagnoses (i.e., personality disorders) compared with 0% in the experimental group. Analyses also focused on only post-test data, without controlling for baseline levels of the key outcome variables. Finally, subjects enrolled in both the intervention and comparison groups had particularly high levels of baseline education (average of 13 years for both groups), so it is unclear how these results would translate to a more typical inpatient psychiatric population.

3.4.3. Rogers & Colleagues' (2010) Conclusions

Reflecting on the state of the literature regarding the impact of SEd, study results suggested that SEd programs may help increase college enrollment and vocational outcomes (e.g., Mowbray et al., 1999; Unger et al., 1991), improve school retention rates (e.g., Unger et al., 2000), and possibly decrease psychiatric hospitalizations (Unger et al., 1991). More specifically, Rogers & colleagues (2010) drew the following positive conclusions about the impact of SEd programs:

• There is suggestive evidence (from noncontrolled studies) that individuals improve their employment and educational status after participating in a SEd program (Best et al., 2008; Cook & Solomon, 1993; Hoffmann & Mastrianni, 1993; Unger et al., 1991; Unger & Pardee, 2002; Unger et al., 2000).

• Individuals who stay engaged in SEd interventions appear to be able to finish courses and keep satisfactory grades (Best et al., 2008; Cook & Solomon, 1993; Unger & Pardee, 2002; Unger et al., 2000).

Unfortunately, Rogers & colleagues (2010) also came to the following conclusions:

• There is no comparative evidence that participation in a SEd program leads to gains in post-secondary educational enrollment or employment when compared with the outcomes of individuals not participating in a SEd program (Mowbray et al., 1999).

• No significant quantifiable changes in self-esteem or quality of life resulted after participation in a SEd program (Unger & Pardee, 2002; Unger et al., 2000).

• Effectiveness data in support of SEd programs are limited. This is due to the absence of well-controlled studies, the limited number of studies that examined key outcomes of interest (e.g., degree completion, employment), and the preponderance of short follow-up periods limiting the ability to examine longer-term participant outcomes.

3.5. Impact of Supported Education Interventions

When Rogers & colleagues (2010) conducted their systematic review of SEd programs, they identified 17 published outcome studies that included pre/post (n=4), experimental (n=3), quasi-experimental (n=1), correlational/survey/observational (n=9), or post-test only (n=4) designs. Seven of these 13 manuscripts were published by the same researcher (Mowbray). Our review of articles published prior to the fall of 2015 uncovered an additional 16 outcome studies published since Rogers & colleagues' (2010) review. These 16 studies included the following designs: pre/post (n=6), experimental (n=5), quasi-experimental (n=0), correlational/survey/observational (n=4), and post-test only (n=1) designs. The four experimental study publications represent three different RCTs, two of which are ongoing and do not yet have extensive published results reflecting comparative outcomes. The number of SEd program outcome studies accumulated from 1989 to 2009 almost doubled in the last 5 years (2010-2014). This represents marked growth in the literature. Perhaps more importantly, these publications also demonstrate the emergence of new scientists in the field of SEd research. These 16 recent publications also represent the work of 13 different first authors. A list of these 16 studies, their research designs, and types of outcomes reported in each publication can be found in Table 3-3. In addition to these 16 studies, we also found 15 publications that were not outcome-oriented trials: ten review articles (or calls for future research or opinion papers), three descriptive program summaries, and two other miscellaneous papers (a SEd guide and an environmental scan). Findings reported in these studies will also be included in this portion of the literature review report.

In this section, we will summarize both client-level and systems-level outcomes described in these original research and review publications. We also include comments about previous findings published before 2010. We did not systematically rate these studies by the merits of their analytic designs; however, in Section 3.6, we discuss methodological strengths and weaknesses of the research studies examined.

TABLE 3-3. Outcomes Examined in SEd Program Impact Studies Published since Rogers & Colleagues' (2010) Systematic Review
Articles	Article Type	Study Design	Educational Engagement	Educational Attainment	Employment	Health and Mental Health	Self- Perception	Service Access
Baksheev, Allott, Jackson, McGorry, & Killackey (2012)	Original research	RCT	X		X
Kidd, Kaur-Bajwa et al. (2012b)	Original research	Pre/post without comparison group	X		X
Ellison, Klodnick, Bond, Krzos, Kaiser, Fagan, & Davis (2014)	Original research	Pre/post without comparison group	X	X
Kidd, Kaur et al. (2014)	Original research	RCT				X	X
Kidd, Kaur-Bajwa et al. (2012a)	Original research	Pre/post without comparison group				X
Manthey, Holter et al. (2012b)	Original research	Survey				X		X
Nuechterlein, Subotnik, Turner et al. (2008a)	Program summary	RCT in progress (only outcomes measured described, not comparative data)	X		X
Nuechterlein, Subotnik, Ventura et al. (2008a)	Original research	RCT (conference presentation summary)	X		X
Rinaldi, Perkins et al. (2010)	Original research	Pre/post without comparison group	X		X
Robson, Waghorn et al. (2010)	Original research	Pre/post without comparison group	X	X
Schindler, & Sauerwald (2013)	Original research	Pre/post without comparison group	X		X	X	X
Smith-Osborne (2012a, 2012b)	Program summaries	RCT in progress (no comparative outcomes published to date)		X		X	X	X
Thompson (2013)	Original research	Post-test only					X
Wagner & Newman (2012)	Original research	Survey	X	X	X
Watkins, Hunt, & Eisenberg (2012)	Original research	Qualitative
Yahaya, Ramli et al. (2010)	Original research	Correlational

3.5.1. Client-Level Outcomes

Rogers & colleagues (2010) noted that typical processes and outcomes described in studies of SEd programs included:

• Educational Enrollment or Engagement: Enrollment in post-secondary education programs, classes, or courses

• Educational Attainment: Post-secondary courses completed, grades attained, certificates achieved, or diplomas or degrees.

• Employment: Full-time or part-time (paid or volunteer) work, hours worked, and wages earned.

• Self-Perception: Self-esteem, self-efficacy, quality of life, and adjustment.

• Number of psychiatric hospitalizations.

• Consumer satisfaction.

After examining the frequency of these outcomes across the 14 recently published studies, we added or modified a few outcomes of interest for our review:

• Health and Mental Health Status: Cognitive and executive functioning, general physical health, and specific psychiatric symptoms (e.g., post-traumatic stress disorder [PTSD], psychotic symptoms).

• Noneducation and Employment-Related Service Use: Use of and engagement in mental health services, including psychiatric hospitalizations.

Similar to the findings from Rogers & colleagues (2010), the most commonly reported outcome within studies published since 2010 was educational engagement and then employment. Only a few articles reported on any type of educational attainment outcome. None reported degree status achieved (beyond the receipt of a program certificate). Several articles reported on health/mental health status, as well as self-perception outcomes. Findings related to these outcomes from studies published both before and after 2010 are described as follows.

Educational Enrollment or Engagement

Secondary data analysis from the National Longitudinal Transition Studies (Wagner & Newman, 2012) has found that school completion rates for students with emotional disturbances who are enrolled in special education services have increased from 47% to 78% from 1990 to 2005. Furthermore, the percentage of students with SEd who enroll in post-secondary education has increased nationwide from 18% to 35%. This increase in high school educational attainment and post-secondary educational enrollment shows the need for SEd programming to meet the needs of these students as they increasingly progress into post-secondary education institutions (Kirsh et al., 2014). As discussed frequently in the literature, both educational enrollment and attainment continue to be critical outcomes to monitor within SEd programs.

Approximately half of the articles published since 2010 mentioned any type of educational enrollment or engagement outcome. The majority of articles mentioned educational enrollment outcomes related to SEd program participation or post-secondary course enrollment.

Earlier work using an RCT found nonsignificant differences between the two treatment and one control conditions on having taken college or vocational education classes since baseline (Collins et al., 1998). In this same study, greater SEd program participation was related to greater participation in college or vocational classes (Collins et al., 1998). More recently, there is suggestive evidence (where significance of the outcomes was not indicated) that SEd program participants have increased enrollment in post-secondary educational institutions and courses (Kidd et al., 2012b; Manthey et al., 2014; Mowbray, 2000). Furthermore, protective factors that help to retain students with SMI enrolled in post-secondary education have been described in the literature. These include active coping, peer support, counseling and psychosocial support, academic support, and academic accommodations (as cited in Hartley, 2010).

Many studies combined educational outcomes with employment outcomes, with many reporting significant increases in educational engagement or employment (Rinaldi et al., 2010; Schindler & Sauerwald, 2013; SAMHSA, 2011; Unger et al., 1991--as cited in Unger, 2011). Killackey et al. (2008) found that an integrated SEd/SE intervention approach led to greater employment and more class completion than usual care. With combined education/employment outcomes, it is impossible to discern the differential impact of the intervention on only education versus employment outcomes. Sometimes this choice by study authors may be due to an integrated SEd/SE approach where program goals were education or employment (but not both goals for all participants). However, some caution should be taken here in interpreting studies that combine educational enrollment and employment outcomes. Many studies have very small sample sizes and may be underpowered to detect group differences. Outcomes may have been combined due to neither outcome alone resulting in significant differences. For example, this was true for the study conducted by Schindler & Sauerwald (2013), where nonsignificant changes occurred in enrollment in higher education from pre-test to post-test.

In one of the few recent RCTs involving a SEd intervention component, Nuechterlein, Subotnik, Ventura, and colleagues (2008a) found that 83% of subjects in the intervention group (combined SEd and SE) returned to work or school, compared with 41% of those in the treatment-as-usual group (p<0.001) during the first 6 months of treatment. This pattern continued even at the end of the 18-month trial (72% versus 42%). Nuechterlein, Subotnik, Ventura, and colleagues' (2008a) study likely combined employment and education outcomes because both were targets of the combined intervention approach. Unfortunately, this combination does not allow the separate examination of the additive impact of the SEd program component within an SE approach. Studies are needed to examine SE alone plus SE with SEd to understand the unique and differential impact of each approach on client outcomes.

Educational Attainment

One-third of articles reviewed mentioned educational attainment as an outcome. Three of the 16 outcome-oriented studies published since 2010 reported on educational attainment. The variables used to asses educational attainment included course completion (as cited in Rogers, Kash-MacDonald et al., 2010; as cited in Unger, 2011; Cook & Solomon, 1993; as cited in Manthey, Goscha et al. 2014; as cited in Mueser & Cook, 2012; Robson, Waghorn et al., 2010), post-secondary degree/certificate completion (as cited in Unger, 2011; Morrison, Clift et al., 2010), high school degree completion (Wagner & Newman, 2012; as cited in Ellison, Rogers, & Costa, 2013; Ellison, Vorheis et al., 2014), satisfactory GPA (Smith-Osborne, 2012b; as cited in Rogers, Kash-MacDonald et al. 2010; as cited in Unger, 2011), and number of credit hours enrolled in and completed (Unger, Pfaltzgraph et al., 2010; as cited in Unger, 2011). Severity of illness is often mentioned as the first barrier to degree/certificate program completion; however, environmental supports have been proven to prevent educational attrition due to mental illness (as cited in Unger, 2011).

Studies generally reported increases in educational attainment from pre-intervention to post-intervention; however, no studies reported these gains in comparison to a group not involved in a SEd program. For example, Robson, Waghorn, & colleagues (2010), in a pre/post comparison of an IPS plus SEd program, reported that 70% of their program's participants had completed their course of study or were continuing with their studies at an 18-month follow-up. In another example, Smith-Osborne (2012a) reports increased GPAs post-SEd intervention among the veterans participating in an RCT; however, GPAs for the intervention group when compared with the control group have not yet been published.

Degree completion was the most rarely reported indicator of educational attainment. The small number of original research articles measuring educational attainment through degree or certificate completion is likely affected by relatively short follow-up data collection periods that do not extend far enough to capture degree/certificate completion. Limited research funding may prohibit the longer-term follow-up periods necessary to examine degree completion. The use of variables measuring course completion, credit completion, or GPA are useful for measuring short-term changes from pre-test to post-test that can indicate potential successful degree/certificate completion. However, studies that follow participants for several years post-intervention to understand the impact on degree completion are very much needed.

Employment

Approximately half of the articles reviewed mentioned some type of employment outcome. The majority of these (11) are from articles published since 2010 or later. The variables used to assess employment outcomes included currently employed (e.g., Collins, Bybee, & Mowbray, 1998; as cited in Krupa & Chen, 2013; as cited in Manthey, Goscha et al., 2014; as cited in Morrison, Clift et al., 2010; Mowbray, 2000; Rinaldi, Perkins et al., 2010; as cited in Rogers, Kash-MacDonald et al., 2010; Schindler & Sauerwald, 2013; Unger, Pfaltzgraph et al., 2010; Wagner & Newman; 2012), type of employment (Cook & Solomon, 1993; as cited in Unger, 2011), hours worked (Cook & Solomon, 1993), pay (Cook & Solomon, 1993; as cited in Manthey, Goscha et al., 2014), and job tenure (Kidd, Kaur-Bajwa et al., 2012a; as cited in Krupa & Chen, 2013). Some of the articles reported the combined educational and employment outcomes together (Baksheev, Allott, Jackson, McGorry, & Killackey, 2012; Mowbray, 2000; as cited in Rogers, Kash-MacDonald et al., 2010; Schindler & Sauerwald, 2013).

The majority of these articles reported positive, if not significant, employment gains in all of the variables used to measure this outcome. Two relatively recent studies are worth mentioning, specifically because of their relatively strong research designs. In a repeated measures, time series pre/post evaluation design (with measurements at baseline and 3-month, 6-month, 12-month and 18-month follow-up), Hutchinson & colleagues (2007) tested the impact of a SEd program (focused on computer skills training) and an SE program. This program approach demonstrated increases in participants working for pay or as volunteers from baseline to 18 months (18%-64%), increases in hours or work per week, and increases in mean earnings per month (among working participants) (Hutchinson et al., 2007). Notably, Hutchinson & colleagues (2007) carefully accounted for attrition over time in all analyses. Gutman, Kerner, & colleagues (2009) conducted a small sample RCT (n=38) that included a treatment-as-usual control group. The SEd program included a 12-week on-site classroom training program that also included academic mentoring and support. Gutman & colleagues (2009) found that 63% of participants at the 6-month follow-up were enrolled in some form of educational program or job training, had obtained employment, or were in the process of applying to a specific program in the next year. This outcome was true for only 6% of participants in the control group.

Some more recent studies have explored the impact of educational attainment on employment (as cited in Ennals, Fossey et al., 2014). This exploration is an important step in determining whether SEd outcomes not only lead to educational attainment but subsequently lead to better employment outcomes as well. There may be active disincentives for SEd program participants to seek employment. For example, Krupa & Chen (2013) reviewed research stating that a disincentive to employment for individuals in SEd programs can be the risk of losing government financial assistance. It is important to understand the perceived barriers to reaching educational or employment goals for program participants.

National estimates of employment for individuals with mental illness is at 48%, SMI at 37%, and schizophrenia and related disorders at 22% (as cited in Unger, 2011). These employment rates are significantly lower than that of the general population (Wagner & Newman, 2012).

Self-Perception

Approximately one-third of the articles reviewed mentioned some type of self-perception outcome. Variables with reported self-perception outcomes include school efficacy, self-esteem, coping, anxiety, empowerment, recovery/resilience, psychosocial wellness, quality of life, social adjustment, social support, and self-efficacy. All but a few articles mentioned increased or significant outcome measures on all variables measuring self-perception (as summarized by Manthey, Goscha et al., 2014; Thompson, 2013; Smith-Osborne, 2012). Of note, Hutchinson & colleagues (2007) used standardized measures of self-esteem (the Tennessee Self-Concept Scale) and empowerment (Empowerment Scale). Their repeated measures design demonstrated significant linear increases in ratings on these two scales over the 18-month course of the study. These results are confirmed via qualitative findings. In qualitative studies, SEd program participants consistently report an increased sense of control, empowerment, and socialization that they gain from program participation (e.g., Bellamy & Mowbray, 1998; Schindler & Sauerwald, 2013). Nonsignificant changes in self-esteem or quality of life were found in less recent studies without standardized measures (Cook et al., 2005a; Unger & Pardee, 2002; Unger et al., 2000). The use of standardized scales in more recently published studies examining changes in self-perception as a result of SEd interventions may help to explain why recent studies are more likely to demonstrate positive change.

Health and Mental Health

Less than one-quarter of the articles reviewed mentioned any type of health or mental health outcome. All of these were from articles published since 2010. Specific health and mental health outcomes reported to be associated with SEd program participation included increases in independent living (as cited in Manthey, Goscha et al., 2014) and decreased PTSD symptoms and increased health (as cited in Smith-Osborne, 2012). Gutman & colleagues (2009) found statistically significant differences between the experimental and control group on three different rating scales measuring social skills, school behavior, and attention skills. The recent RCT by Kidd & colleagues (2012b, 2014) has supplemented a classroom-based SEd program with cognitive remediation. This trial includes many standardized measures of executive functioning. The SEd program resulted in significant improvements from pre-intervention to post-intervention in the Trail Making Test B, verbal learning as indicated in the California Verbal Learning Test (CVLT), the time component of the Digit Vigilance Test, and on the general psychosis symptomatology measure (Positive and Negative Syndrome Scale [PANSS]). Of further note, significant improvement in sustained attention and vigilance was found in only the control group that received the standard SEd program (without the cognitive remediation component) (Kidd et al., 2012b; Kidd et al., 2014). The integration of standardized measures of health, mental health, and particularly executive functioning is a positive trend in studying SEd program outcomes.

Use of Other Types of Services

Only a few articles reviewed mentioned any type of outcome related to mental health service use, access, or engagement. Two articles are worth mentioning. First, Collins, Bybee, & Mowbray (1998) found a significant difference in involvement in rehabilitative services for participants with the highest level of participation in the group condition alone. Participants with lower participation rates or participants enrolled in the classroom condition or control group were significantly less likely to be involved in rehabilitative services than those who were high participants and in the group condition. Second, Hutchinson & colleagues (2007) found a significant linear decrease in program participants' report of mental health and rehabilitation services used over the course of 18 months. Ideally, SEd program participation enhances client functioning thereby reducing the need for intensive or restrictive psychiatric treatment. Findings particularly by Hutchinson et al. (2007) are promising, but understanding the impact of SEd programs on mental health and noneducation services merits further research attention.

Impact of Client Characteristics on Program Participation and Outcomes

SEd program effects may or may not differ based on various client characteristics. Importantly, the presence or absence of major psychiatric diagnoses does not appear to affect a SEd program participant's post-secondary education enrollment (Unger & Pardee, 2002; Unger et al., 2000). A client's prior work or school activity appears to be the strongest predictor of later involvement in work and school (Collins et al., 2000). Single, unmarried SEd program participants were less likely than married participants to be involved in post-intervention work and school activities (Collins et al., 2000). More frequent contact with a social network has been found to be associated with more post-intervention work and school activities (Collins et al., 2000). Meanwhile, less financial stability was associated with fewer post-intervention work and school activities (Collins et al., 2000). Gutman & colleagues (2009) noted several other factors associated with program success: adherence to a medication routine, stable residence, and motivation to attend the program regularly. Meanwhile, diagnosis, prior educational level, number of past 5-year hospitalizations, age of mental illness onset, and parental education had no relationship to program success (Gutman et al., 2009).

Individual client characteristics also appear to be associated with SEd program participation. More hours worked per day, higher ratings of residential quality of life, and a larger social network were all related to higher attendance in a MSERP SEd group (Bybee, Bellamy, & Mowbray, 2000). The presence of a substance abuse problem, on the other hand, was associated with lower attendance in a SEd program (Bybee et al., 2000).

A few other adaptations to the traditional SEd program design have been implemented to target students with cognitive difficulties who do less well in SEd programs alone (Kidd et al., 2012b), veterans with PTSD (Smith-Osborne, 2012a), and Orthodox Jews with SMI who have educational goals unique to their religious community (Shor & Avihod, 2011). Some of these adaptations have shown promising results (Kidd et al., 2012b), some outcomes have yet to be published (Smith-Osborne, 2012a), and others may not be published beyond program summaries because the population of interest is unique, and outcome goals are not easily generalizable (Shor & Avihod, 2011).

Summary of Supported Education Program Impact on Client-Level Outcomes

In reflecting on the impact of SEd programs on client-level outcomes, we note some changes in the field in the last 5-10 years. Recent publications have added to prior evidence from noncontrolled studies demonstrating that individuals appear to improve their educational enrollment after participating in a SEd program. Recent research also adds to the suggestive evidence (from noncontrolled studies) that individuals improve their employment and educational attainment after participating in a SEd program. Unfortunately, there continues to be a lack of comparative evidence that participation in a SEd program leads to gains in post-secondary educational enrollment and more importantly, educational attainment. Randomized trials currently in progress (Nuechterlein et al., 2008a; Smith-Osborne, 2012a, 2012b) may add to this comparative literature in forthcoming publications.

The comparative evidence on the impact of SEd on both education and employment outcomes is growing, with particularly noteworthy work by Hutchinson & colleagues (2007), Nuechterlein, Subotnik, Turner, & colleagues (2008a), Killackey & colleagues (2008), Baksheev & colleagues (2012), and Gutman & colleagues (2009). The rationale to combine education and employment-oriented program approaches is compelling, particularly as many young adults jointly pursue both education and employment goals. These experimental or quasi-experimental studies do demonstrate quantifiable impacts on employment outcomes. Unfortunately, for the purpose of understanding the specific impact of SEd interventions, these studies are not as helpful. Three of the four studies test a combined SEd/SE approach where the differential impact of SEd on employment beyond the impact of the SE approach cannot be determined. But, generally, there appears to be growing research that SEd approaches, particularly when combined with SE, increase participant employment.

At the time of Rogers & colleagues' (2010) systematic review, the authors concluded that SEd studies demonstrated no significant quantifiable changes in self-esteem or quality of life after participation in a SEd program (Unger & Pardee, 2002; Unger et al., 2000). Recent research provides evidence to contradict this conclusion (outcomes summarized by Manthey, Goscha et al., 2014; Thompson, 2013; Smith-Osborne, 2012; Hutchinson et al., 2007). Qualitative studies also support these positive changes in self-perception as a result of SEd program participation (e.g., Bellamy & Mowbray, 1998; Schindler & Sauerwald, 2013).

Finally, it is premature to conclude that SEd programs affect general health, mental health, or functional status. This is hindered by the general lack of longitudinal data examining these outcomes of interest. However, the use of standardized measures to assess self-perceptions, and health and mental health status or functioning is a noteworthy advance in this field. Ongoing trials may provide new evidence about health and mental health outcomes in the next 5-10 years.

3.6. Supported Education Research Methods Summary and Gaps in the Published Literature

This review has highlighted the substantial number and variety of efforts in SEd research. The review also highlights what is missing in the field and next steps needed for SEd research. This section first summarizes study methods and then discusses gaps in the literature and potential next steps.

3.6.1. Study Methods

Outcome Data and Measures

In studying the impact of SEd interventions on client outcomes, researchers primarily rely on primary, client-reported data. Data from program or education administrators or the use of secondary, administrative data are rarely described. Increasingly, however, standardized measures of health, self-perceptions, mental health, and executive functioning are being used in SEd program outcome studies.

Variables measured using standardized outcome measures include anxiety, using the Personality Assessment Inventory (Collins et al., 1998) and the Zung Self-Rating Anxiety Scale (Cook & Solomon, 1993); social adjustment, using the Social Adjustment Scale-Self-Report (Collins et al., 1998); symptomology, using the Symptom Checklist-10 (Collins et al., 1998) and the Brief Psychiatric Rating Scale (Robson et al., 2010); drug and alcohol use, using two scales from the Personality Assessment Inventory (Collins et al., 1998); self-perception/self-esteem, using Rosenberg's Self-Esteem Scale (Collins et al., 1998; Cook & Solomon, 1993; Kidd et al., 2012a; Kidd et al., 2014); empowerment using the scale developed by Rogers, Chamberlin, Ellison, & Crean (1997); quality of life using Lehman's Quality of Life Interview (Collins et al., 1998); coping, using the coping mastery scale (Cook & Solomon, 1993); mental health, using the PANSS (Kidd et al., 2012a; Kidd et al., 2014); cognitive measures, using the Wide Range Achievement Test 3, Trail Making Test A, the digit span subtest of the Wechsler Adult Intelligence Scale-III, CVLT, Trail Making Test B, Wisconsin Card Sorting Test, and the Digit Vigilance Test (Kidd et al., 2012a; Kidd et al., 2014); and SEd program fidelity, using the SAMHSA Supported Education Fidelity Scale (Manthey et al., 2012b). Increasing use of standardized measures to understand program impact provides an improved opportunity to demonstrate quantifiable changes in client outcomes as a result of SEd program participation.

Length of Follow-Up Periods

Of the few studies that collected post-intervention outcome data, most included only a post-program completion assessment (post-test). Times between baseline/pre-test and follow-up/post-test ranged from 3 months to 9 months. This range in data collection periods obviously creates difficulties understanding immediate post-program impact. Longer programs would have allowed program participants a longer time to enroll in education courses or seek and obtain employment. Consequently, post-program participation outcomes should not be compared directly across studies.

Preliminary studies exploring ways to adapt SEd programs often only reported findings at the conclusion of the program; for example, when studying the integration of cognitive remediation into a pre-existing SEd program (Kidd et al., 2012a; Kidd et al., 2012b). Publications focused on adaptations to traditional SEd programs are new to the research literature, and outcome data with longer follow-up data collection periods on these adaptations will require more time for these outcomes to be measured, analyzed, and published.

Only a handful of articles reviewed included follow-up data collection beyond an immediate post-program assessment. Excluding post-test only data collection, seven of the original research outcomes study publications that we reviewed collected follow-up data anywhere from 8 months to 3 years after baseline. A typical post-secondary degree/certificate program takes 2-4 years to complete. With the exception of one study with a follow-up data collection period of up to 3 years, none of the follow-up data collection periods would have been positioned to gather information on post-secondary degree completion or program certification attainment for any program participants still in the midst of their course of study. Data suggests that SEd programs help participants with SMI reach their educational goals (e.g., Collins et al., 1998; Rogers et al., 2010), yet current research lacks the minimum necessary follow-up periods to accurately assess degree/certificate completion for the majority of clients. Mueser & Cook (2012) and Manthey, Goscha, & Rapp (2014) noted that short-term funding periods may limit the ability of researchers to accurately assess the primary goals of SEd programs and instead lead researchers to focus on the short-term goals of educational enrollment, class or credit completion, and GPA (e.g., as cited in Rogers, Kash-MacDonald et al., 2010; Robson, Waghorn et al., 2010; as cited in Unger, 2011; Unger, Pfaltzgraph et al., 2010).

Strengths and Weaknesses of Study Designs

Ellison, Rogers, & Costa (2013) mentioned in their review of SEd literature for young adults that many of the articles reviewed are what they considered "pre-scientific" and consequently the impact of many SEd programs have yet to be measured systematically. Many research studies include small sample sizes, limited use of control groups, short follow-up periods, use of nonstandardized measures, and preliminary research analysis. These critiques could also be named as true for some more recent publications not included in the Ellison, Rogers, & Costa (2013) or Rogers, Kash-MacDonald, & colleagues (2010) reviews (e.g., Kidd et al., 2014; Manthey et al., 2014; Manthey et al., 2012b; Robson et al., 2010; Schindler & Sauerwald, 2013). Small sample sizes (e.g., Gutman et al., 2009; Kidd et al., 2014; Robson et al., 2010) and high attrition rates (e.g., Cook & Solomon, 1993; Manthey et al., 2014) limit analysis possibilities and the generalizability of findings. Small sample sizes could explain why outcome data from original research publications and review articles do not consistently mention the statistical significance of findings (e.g., Krupa & Chen, 2013; Robson et al., 2010; Thompson, 2013). A high number of review articles or program summaries continue to highlight the importance of SEd programs, but additional research is needed to produce evidence of the long-term outcomes of SEd programs.

Certain aspects of research findings were unclear or were not mentioned in the methods or results sections, limiting the reliability of the reported findings. Sample composition issues included high attrition rates not being broken down by the individual study intervention or control conditions (Collins et al., 1998). Sometimes sample demographic characteristics were unclear or not described (Manthey et al., 2012b; Yahaya et al., 2010), including studies that lacked a description of the number of sample members with SMI versus other conditions (Morrison et al., 2010; Thompson, 2013). Other studies failed to describe the number of respondents in the control group (Nuechterlein et al., 2008a) or the overarching demographics of the sample, especially in relation to how they could affect outcomes (Manthey et al., 2012b).

Outcome measures around educational engagement or attainment were often not reported separately and instead were reported in conjunction with employment outcomes (Rinaldi et al., 2010). Schindler & Sauerwald (2013) provide some insight into the possibility that educational outcomes alone may have failed to reach statistical significance, but without individual findings reported, it is impossible to understand fully these results. Sometimes key outcomes of interest were not adequately explained or measured even within the context of relatively rigorous designs. For example, Rinaldi, Perkins, & colleagues (2010) included an extensive follow-up data collection in their study--assessments at four time points, extending to 24 months post baseline. The primary outcome of interest in this study was the number of clients working or studying (without an operationalized definition of "studying"). This study missed the opportunity to capture explicit educational enrollment and attainment information. In another study, reported outcomes in the article text were confusing and not well-justified using sound methodological procedures (Yahaya et al., 2010). Fortunately, there is an emerging small body of research studies examining the impact of SEd programs using RCTs (Nuechterlein et al., 2008a; Smith-Osborne, 2012a); however, most comparative outcomes between treatment and control groups for educational and employment have not yet been published.

Adaptations to traditional SEd programs are an important step toward fully understanding the benefits of SEd programs and their services. Some examples of adaptations to traditional SEd programs are cognitive remediation (Kidd et al., 2012a; Kidd et al., 2012b; Kidd et al., 2014), understanding how SEd services are being provided at community agencies (Morrison et al., 2010), and SEd programs serving veterans (Smith-Osborne, 2012a) or Orthodox Jewish communities (Shor & Avihod, 2011). These early adaptations largely serve to provide information about how the program and its services are adapted to meet the specific needs of a target population or community. At this point, there is little generalizable outcome data about how these adaptations in SEd programming or services provide long-term benefits to these target populations.

3.6.2. Unanswered Research Questions

Findings from the literature review pointed to several remaining unanswered research questions. These questions are summarized in the following bullets:

• SEd/SE approaches have shown that participants have higher levels of "school activity," but this is not the same as demonstrating higher rates of degree attainment and, even more importantly, a change in life status as a result of these advanced degrees (improved standing in the labor market) (Mueser & Cook, 2012). Because so many studies are short-term and/or focus on only course completion, it is difficult to draw conclusions about impact on degree completion, job acquisition (as a result of new degree status), and ultimate employment. So, two critical largely unanswered research questions still remain:

  • Do SEd programs enable individuals to complete a course of study that successfully leads to increased educational attainment as represented by a post-secondary degree or certificate?
  • Can SEd programs enable individuals to successfully get and sustain jobs?

• How can services offered within SEd programs be tailored to best address individual functioning, skills, needs, preferences, and age cohort (Leonard & Bruer, 2007)? Are different SEd model variations needed for various client profiles (e.g., GED support versus vocational training versus traditional 4-year college)?

• What is the ideal participant profile for participation in a SEd approach? As discussed by Leonard & Bruer (2007), two particularly well-known and controlled studies examined included, as part of their selection criteria, students with above-average intelligence and students with a mean education of at least 13 years (Hoffmann & Mastrianni, 1993; Unger et al., 1991). Are SEd programs too challenging for clients whose language, reasoning, or other academic skills are inadequate? Should clients with poorer educational backgrounds be offered a separate type of training program more focused on adult basic education to build skills?

• How should SEd programs be best integrated with existing evidence-based practice models? For example, how can SEd and SE be woven together to best meet the needs of individuals with SMI? Are they both part of a service continuum? Which clients would benefit most from one approach over the other (or both)? Are some clients best suited to immediately receive SE services, whereas others could benefit from moving through a SEd model and then to SE?

• How do SEd programs recruit and engage participants?

• How do existing SEd adaptations need to be tailored to better address the needs of secondary and post-secondary students, both for those with long-standing psychiatric conditions or with first-episode psychosis?

• A cadre of recent trials has examined SEd/SE approaches with individuals experiencing first psychotic episodes. Would these integrated SEd/SE interventions be equally or more effective among individuals with trauma, mood disorders, or other symptom profiles?

3.6.3. Methodological Limitations

There are several methodological limitations to studies within the published SEd literature. These limitations hinder opportunities to better understand the impact of SEd programs on key outcomes of interest. Some methodological limitations include the following:

• Trials needs to be designed with follow-up data collection that extends 3 or more years from baseline to adequately capture longer-term educational attainment and job sustainability outcomes. Most SEd studies are limited by 1-2-year follow-ups (or less), which is an insufficient amount of time for most individuals to complete a full degree requirement.

• Larger sample sizes in SEd outcome studies are needed to analyze differences in outcomes by demographic characteristics and mental illness/symptomology.

• Research studies should more commonly use analytic techniques to account for high rates of attrition that occur in the context of the research study to minimize outcome biases.

• Experimental research needs to match comparison and intervention participants on key characteristics and level of functioning to really understand what works best for whom.

• Studies of SEd program replication are lacking (across program developers, sites, or communities). Without this type of evaluation and implementation research, it is difficult to know how easily transportable various SEd approaches may be.

3.6.4. Other Gaps in Knowledge that Prevent Supported Education Program Dissemination and Scale-Up

Methodological limitations not only weaken the SEd evidence base, but they also limit the possibility for broader SEd program dissemination. We identified two primary gaps in the SEd knowledge base that impede larger-scale-up of SEd programs.

• More information is needed on the ideal service context for SEd interventions. What implementation issues are particularly apparent for one context versus another?

• Efforts are needed to resolve the tremendous service financing hurdles that many SEd programs in the field face.

3.6.5. Additional Data Necessary to Consider Supported Education an "Evidence-Based Practice"

SAMHSA houses a National Registry of Evidence-based Programs and Practices (NREPP, http://www.nrepp.samhsa.gov/01_landing.aspx). It is one of the leading sources of information on evidence-based practices in substance abuse and mental health. NREPP is a searchable online registry and includes more than 350 interventions to date. NREPP was developed to help the public learn more about evidence-based interventions that are available for implementation. NREPP is a voluntary, self-nominating system in which intervention developers elect to participate. After a nomination is submitted, an independent committee reviews intervention evidence to decide whether it meets certain criteria and rates the methodology of the intervention's supporting evidence. NREPP publishes a report called an intervention summary on its web site for every intervention it reviews.

In considering what might be necessary for SEd programs to be considered "evidence-based practices," it is helpful to consider the NREPP program requirements. To apply to receive an NREPP review, an intervention must meet the following minimum requirements:

1. The intervention has produced one or more positive behavioral outcomes (p<0.05) in mental health or substance abuse among individuals, communities, or populations. Evidence of the positive behavioral outcome(s) has been demonstrated in at least one study using an experimental or quasi-experimental design. Experimental designs include random assignment of participants, a control or comparison group in addition to the intervention group, and pre/post-test assessments. Quasi-experimental designs include a control or comparison group and pre/post-test assessments but do not use random assignment. Studies with single group, pre/post-test designs do not meet this requirement. Significant differences among groups over time must be demonstrated for each outcome.

2. Implementation materials, training and support resources, and quality assurance procedures have been developed and are ready for public use (SAMHSA, 2011).

3. The results of these studies have been published in a peer-reviewed journal or other professional publication (e.g., a book volume) or documented in a comprehensive evaluation report. Information must be included in publications to enable independent ratings of six research quality indicators. Each indicator is given a rating of 0 (total absence of evidence, not acceptable), 2 (some evidence, moderate acceptability), or 4 (acceptable):

   • Reliability of Measures: Outcome measures should have acceptable reliability to be interpretable. Here, "acceptable" means reliability at a level that is conventionally accepted by experts in the field.
   • Validity of Measures: Outcome measures should have acceptable validity to be interpretable. Here, "acceptable" means validity at a level that is conventionally accepted by experts in the field.
   • Intervention Fidelity: The "experimental" intervention implemented in a study should have fidelity to the intervention proposed by the applicant. Instruments that have tested acceptable psychometric properties (e.g., inter-rater reliability, validity as shown by positive association with outcomes) provide the highest level of evidence.
   • Missing Data and Attrition: Study results can be biased by participant attrition and other forms of missing data. Statistical methods as supported by theory and research can be employed to control for missing data and attrition that would bias results, but studies with no attrition or missing data needing adjustment provide the strongest evidence that results are not biased.
   • Potential Confounding Variables: Often, variables other than the intervention may account for the reported outcomes. The degree to which confounds are accounted for affects the strength of causal inference.
   • Appropriateness of Analysis: Appropriate analysis is necessary to make an inference that an intervention caused reported outcomes.

By these criteria, only a handful of studies would be eligible for an NREPP review nomination. Only six separate interventions were tested using an experimental or quasi-experimental design (including a comparison group): Collins et al. (1998), Hoffman & Mastrianni (1993), Gutman et al. (2009), Kidd, Kaur et al. (2014), Killackey, Jackson, & McGorry (2008), Nuechterlein, Subotnik, Turner et al. (2008a), Nuechterlein, Subotnik, Ventura et al. (2008b), and Smith-Osborne (2012a, 2012b). Because trials are ongoing, three of these interventions do not provide sufficient evidence at this time. The two oldest studies did not find sufficient evidence of a positive behavioral impact and lacked key information on implementation. The most promising candidate intervention is the one tested by Killackey, Jackson, & McGorry (2008) that examined SE with integrated SEd components. The SEd aspects of this intervention approach are not well-described in the two publications available; consequently, it is hard to judge the degree to which this approach moves beyond a traditional SE intervention. By this analysis, we would consider SEd programs as a promising practice. This is consistent with SAMHSA materials developed about SEd program approaches (see https://store.samhsa.gov/shin/content/SMA11-4654CD-ROM/BuildingYourProgram-SEd.pdf). Unfortunately, there is not currently a SEd program tested with sufficient rigor and including sufficient evidence of behavioral change to be nominated for consideration as an evidence-based practice.

Evidence-based practice status for SEd is hampered by study design and lack of positive behavioral outcomes. For the SEd program approach to move from a promising to evidence-based practice, a long-term demonstration project is needed. One particular promising SEd model will need to be tested in a way comparable to the Mowbray trial, but without the methodological flaws and including a longer-term follow-up period. Cook & colleagues' (2005a) multisite RCT of SE should be seen as a model. With the development of the SAMHSA (2011)/University of Kansas SEd fidelity scale, there is the opportunity to quantify the degree to which individual programs are abiding by principles seen as core to SEd approaches. Future trials can now include the fidelity scale as a way of understanding how variation in program fidelity affects client outcomes. This should speed up the process of information that will be necessary for SEd programs to be established as "evidence-based."

4. Environmental Scan

4.1. Introduction

The knowledge base on SEd interventions is just emerging. Although Chapter 3 reviewed a growing body of published literature, other evidence exists from ongoing evaluations and researcher experiences with conducting SEd studies in the field. This report summarizes the results of an environmental scan conducted with a select number of SEd researchers, program managers, and other stakeholders involved in funding or supporting the implementation of SEd programs in communities across the country. The individuals selected for participation in these discussions were chosen to represent various perspectives across the SEd research and practice community. The findings from this chapter are not intended to represent all possible issues and should not be taken as conclusive. Rather, this summary describes a snapshot of issues for consideration in designing, implementing, and evaluating SEd programs for individuals with SMI.

4.2. Methods

Fourteen unstructured discussions were conducted by telephone with researchers, program managers, and key informants identified to participate in the environmental scan. This included four researchers, six SEd program managers, and conversations with four other sets of stakeholders relevant to SEd programs and research. Table 4-1 shows a list of the individuals who participated in these discussions.

Members of the project team made preliminary contact with all identified individuals for unstructured discussions via email. One week later, a second reminder email was sent to all contacts who had not yet replied to the initial request for information or to schedule a call. All telephone calls took place between January and February 2015. Before each call, individuals received a summary of the project's goals and sample questions to be included in the discussion. All calls were recorded with the respondents' permission. Recordings were reviewed to ensure the accuracy of notes entered for each call. To summarize information gathered on these calls, a template was created that delineated relevant aspects of SEd programs (e.g., program goals/objectives, target population [diagnosis, age range, setting], number of individuals served/year, program length, agencies involved in the program). The template was used to compile information on each of the SEd programs included in the environmental scan.

The project team identified researchers who had recently published studies focused on SEd in the literature review. Three researchers with diverse research areas were selected to participate in an unstructured telephone discussion. All individuals listed in Table 4-1 were also contacted via email to inquire about unpublished reports, manuscripts in press, or other ongoing SEd efforts that might be missed via a traditional literature review. Five additional researchers identified in the literature review did not participated in an unstructured telephone discussion, but were emailed to inquire about unpublished or ongoing SEd research. Information was received from two researchers, Drs. Trevor Manthey and Alexa Smith-Osborne. Those researchers who did not participate in the environmental scan telephone discussions are not included in Table 4-1.

TABLE 4-1. Stakeholders Who Participated in Unstructured Discussions
Type of Stakeholder	Individual Name	Title	Affiliation
Researcher	Karen Unger	President	Rehabilitation Through Education
Researcher	Gary Bond	Professor of Psychiatry	Dartmouth Psychiatric Research Center
Researcher	Kim Mueser	Executive Director, Center for Psychiatric Rehabilitation	Boston University
Researcher/ program manager	Michelle Mullen	Assistant Professor, Department of Psychiatric Rehabilitation and Counseling Professions	Rutgers University
Program manager	Lisa Mueller	Psychologist and Medical Director for Compensated Work Therapy	Veterans Integration To Academic Leadership, Edith Nourse Rogers Memorial Veterans Hospital
Program manager	Tamara Sale	Program Development Coordinator	EASA
Program manager	Luana Turner	Psychologist/Therapist	UCLA Aftercare Research Program
Program manager	Gary Scannevin, Paul Margolies, Liza Watkins		OnTrackNY
Program manager	Jo-Anne Sharac	Coordinator of Disability Services	Quinsigamond Community College
Program manager	Cara Sams	Program Director	EASA, Transition-Age Youth Programs, LifeWorks Northwest
State or federal official	Sandra Miller	Transition Coordinator	Delaware Division of Vocational Rehabilitation
State or federal official	Denise Juliano-Bult	Program Chief	Division of Services and Intervention Research, NIMH
State or federal official	Leslie Caplan	Rehabilitation and Program Specialist	NIDRR, U.S. Department of Education
State or federal official	Jean Close, David Shillcut, Kathryn Poisal, Margherita Sciulli		CMS

Telephone discussions were held with six SEd program managers. The program managers who participated in the environmental scan were located in five states. One individual (Michelle Mullen) described herself as both a program manager and researcher. Table 4-2 lists the seven SEd programs and characteristics of each program. Program managers were selected to cover, as broadly as possible, the heterogeneity of SEd programs (campus-based, psychiatric rehabilitation through a mental health center). Program selection was guided by: (1) common SEd program models; (2) inclusion of different types of program models; and (3) geographic variability/programs in at least four states. Conversations with program managers centered on key challenges to operating and financing SEd initiatives. The conversations covered key funding sources for each program, whether the program was being formally evaluated, and the challenges to evaluating these programs.

TABLE 4-2. SEd Programs Included in the Environmental Scan
Program Name	Location	Setting	Program Description
Saint Clare's Behavioral Health Services, Labor Education and Research Now (LEARN)	Denville, NJ	Community mental health center	Provides SEd services to adults with a psychiatric disability who have a desire to pursue higher education.
Veterans Integration To Academic Leadership, Edith Nourse Rogers Memorial Veterans Hospital	Bedford, MA	College campus	Provides VA outreach services on college campuses to improve the mental health of veterans while supporting their successful integration into college.
Early Assessment and Support Alliance (EASA)	Multiple locations, OR	Community mental health center	A network of clinical and community-based services that provide SEd services in conjunction with other resources for individuals with first-episode psychosis.
University of California, Los Angeles Aftercare Research Program	Los Angeles, CA	College campus	Outpatient research clinic for recruitment, interventions, and assessments for first-episode patients who are participating in research projects at the Center.
OnTrackNY	Multiple locations, NY	Community mental health center	Affiliated with the NIMH RAISE, OnTrackNY presents an IPS SE and SEd model for individuals with first-episode psychosis.
Quinsigamond Community College	Worcester, MA	Community college	Provides SEd services to students through the Quinsigamond Community College Disability Services office.
EASA, Transition-Age Youth Programs, LifeWorks Northwest	Northwest Oregon	Community mental health center	Provides SEd services in conjunction with other resources for individuals with first-episode psychosis.

The project conducted unstructured discussions with four other stakeholders from agencies funding research on SEd or from organizations involved in financing or serving individuals with SMI. These other stakeholder informants included a manager within a state VR program, as well as federal program officers from the HHS Centers for Medicare and Medicaid Services (CMS), the HHS National Institute of Mental Health (NIMH), and the U.S. Department of Education National Institute on Disability and Rehabilitation Research (NIDRR). These individuals provided valuable information related to potential collaboration in managing SEd programs, funding for SEd research, policies relevant to SEd programs, and SEd program financing.

4.3. Program Characteristics

Participants in the environmental scan discussions represented seven different SEd programs across the country. These programs were providing SEd services to both high school and college-level students. Table 4-2 provides a description of these programs.

4.3.1. Program Characteristics

This section summarizes information respondents provided as they described their familiarity with a particular SEd program's composition. Respondents described the array of services included in each particular SEd program, methods used to recruit and engage program participants, SEd program participant composition, staffing and management for SEd programs, and challenges experienced by SEd program participants in attaining targeted educational goals. Discussions across each of these areas are described as follows.

Participants in Supported Education Programs

The number of participants served per year across the SEd programs ranged from 20 to 900. Programs based in community mental health agencies were described as having smaller teams of staff who served a relatively small number of participants. One program served 50 participants per year who were enrolled in a 2-year program. The VA, campus-based programs, and large community mental health agency-based respondents reported serving a range of 300-900 participants per year. Joint SE/SEd programs indicated that roughly one-third to one-half of their students were pursuing educational goals or receiving some type of on-campus services. One community college campus-based SEd program reported serving 800-900 students per year with SEd specialist caseloads of 150-200 students.

The participant composition described by program managers appeared to vary by the service setting. For example, programs based out of the VA served veterans with a range of mental health diagnoses. Campus-based programs were also described as serving students with a range of psychiatric conditions. One campus-based program manager mentioned that her program was seeing an increase of students on campus with Asperger syndrome and veterans with PTSD and depression, in addition to students with depression, schizoaffective disorders, and anxiety disorders. Community mental health agencies providing SEd services were described as most often serving participants with schizophrenia and schizoaffective disorders in the early stages of diagnosis (1-2 years within the onset of symptoms). The age range of participants served by SEd programs were adolescents starting at 15 years old to adults in their 30s; there were exceptions when participants were younger or older than this range.

Participant Recruitment and Engagement

Respondents described outreach efforts to encourage referrals and service access. Strategies included efforts to reduce stigma, myths, and misconceptions that might prevent participants from seeking services. These perceptions included believing that the specific service setting might not be for them (e.g., disability office, VA, community mental health center) or having a sense that someone who has a mental illness might not be eligible for supportive services. Program managers felt that direct outreach efforts were necessary for all types of program settings providing SEd services, including campus disability offices, VA centers, and community mental health centers. Program manager respondents readily reported specific stories of participants' reluctance to receive services, stressing the importance of outreach activities to engage individuals in SEd programs. According to some program managers, persistent follow-up outreach efforts were sometimes necessary for months or even years until certain target individuals were ready to participate.

Another component of participant recruitment and outreach efforts described by program managers included outreach to community organizations such as schools (high school and college), the campus disability office, medical offices, primary care physicians, hospitals, emergency rooms, urgent care centers, and court programs. Court linkage was less commonly described across program managers but was an interesting approach. The University of California, Los Angeles (UCLA), Aftercare Research Program has begun to work with the Mental Health Court Linkage Program to assist young adults with SMI who are involved in the criminal justice system. In general, these outreach efforts seemed to allow for information sharing through interviews, such as on the radio, or brochures that could be dropped off in medical offices, and for relationship building with service providers of the desired program target population.

SEd publications describe challenges with program retention, engagement and attrition. Environmental scan respondents indicated that SEd service use and program participation varied across time. In general, respondents indicated that program participation is highest immediately after program enrollment and gradually declines over time. New program participants are often meeting with multiple team members and accessing an array of services as often as multiple days per week. One program manager specifically commented that she has observed that participant service use begins to decline around 6 months into the SEd program, as the participant is encouraged to be more independent in identifying and using needed services. Another program manager described it as critical that a SEd program encourage participants to learn how to build their network outside of the program. The last 6 months of a SEd program often include the sharpest decline in the use of program services as participants are often shifted into more supportive relationships with staff or peers to help solidify the progress they have made. At this stage, participants may be accessing services once a week, biweekly, or even less often. So, some decline in program participation may be a natural progression of SEd programs and even a desired outcome.

Most programs described by respondents appeared to be designed to provide services for approximately 2 years. Respondents emphasized how hard it is to describe uniform SEd program progression because very often, supports and services are so tailored to the individualized needs of the participant. Some SEd participants need more time in intense services/supports whereas others can transition more quickly to natural supports and become less reliant on the formal program. A number of program managers noted that it is important to allow for participants who are at different stages of readiness to drop off from services and come back as they are ready and when necessary.

Because of concerns about program attrition, it is important to understand program efforts around participant recruitment, retention, and engagement. Many respondents provided detailed information about their program's recruitment and engagement methods. Referrals to SEd programs most often came from hospitals and mental health agencies but sometimes also from schools and families, and as walk-in participants. A limited number of program managers specifically described efforts to attempt to see participants within 24 hours of an initial contact or referral.

Respondents believed that participant engagement hinged on having participant-driven care and a dedicated SEd program staff member. All program managers indicated that successful programs should have the participant define educational and other goals, with the service team coming together to help the participant succeed. The participant should drive desired services based on his or her educational goal. Respondents believed that the SEd specialist was critical to the engagement process. One program described its SEd specialist as its "secret weapon" in ensuring that participants stay engaged in the program. Participant engagement was often attributed to mobile support and outreach efforts that were provided in the community. Many programs were described as having a minimum percentage of time (e.g., the OnTrackNY program requires its specialist to be in the community 65% of the time) that the SEd specialist must spend in the field doing outreach activities and meeting directly with program participants.

Additional services that were described across program managers for promoting participant engagement were connecting participants with other partners (e.g., campus mental health center); involving family members to increase their knowledge of how to support SEd participants and encourage accountability in service use; and finally, having peer support staff available to provide the participant with peer-to-peer feedback. One program manager noted that this type of peer relationship may be especially useful for military veterans who are now acclimating to the requirements of a college/academic setting.

Challenges for Supported Education Program Participants

SEd program participants face many challenges while trying to reach their educational goals. Some challenges for SEd program participants include:

• Accessing and then being fully engaged in a SEd program;
• Educational barriers;
• Mental illness barriers that affect academic success; and
• Personal and employment barriers.

Program-level challenges for SEd participants included myths, stigmas, and misconceptions about SEd programs that were perceived as barriers and that affected participants' entrance to a program and engagement with other program participants. Program managers described program dropout as a common problem. Participants may disengage from SEd as their mental health symptoms improve, but they need to re-engage when symptoms worsen. Some program managers described participants as transient and indicated that frequent moves often prevented program accessibility. One program manager described efforts to navigate this problem by partnering with sister programs across the state to identify participants who might have moved to resume services at a new program location closer to where they now live.

Program managers reported that participants experienced several challenges in reaching their educational goals related to academic readiness, enrollment needs, supports, completing classes and subsequently minimizing financial aid problems and costs, and mental health needs. Program managers indicated that participants often needed to take pre-college classes on academic skills (e.g., basic math and English) and how to juggle academic demands (e.g., time management, study skills, using technology, coping skills) before they were ready for college-level classes. Then, once participants were ready, program managers described participants needing help to get back into school (or stay in school). SEd programs were described as attempting to "meet the participant where they are," but some respondents working specifically with high school students noted that it was easier to keep a student in school and focused on completing school on time (rather than having to stop and complete school via a GED path that often offered fewer supports). Individualized education plans (IEP) or Section 504 plans were also described as tools to help build supports around successful high school completion. An IEP describes the tailored education objectives and needs of a student who has qualified for special education services. It outlines specific supports and services that will help a student achieve his or her educational goals. If a student is 16 years or older, the IEP must include a description of transition services to support a student moving from secondary school to post-secondary school activities. An IEP can be in place until an individual's 21st birthday. A 504 plan details the modifications and accommodations that might be necessary for a student with a disability to perform at the same level as their peers. A 504 plan does not require that a student meet eligibility for special education services.

Program managers described a sense that participants often come to SEd programs with beliefs that they should have been able to handle their educational goals on their own without help. Consequently, there was a sense that participants have often waited too long for help. Waiting too long for help was perceived to lead to participants requesting help withdrawing from classes instead requesting help with enrolling or completing coursework. Program managers acknowledged that poor GPA, course incompletes, failed classes, and class withdrawal can prevent future college access and sometimes lead to financial aid problems. One program manager offered insight that course incompletes and failed classes can prevent students from obtaining further financial aid, while keeping them in debt for the cost of these dropped or failed classes. Additionally, colleges sometimes have policies in which even small unpaid debts from a previous semester (e.g., library fines, tuition) can prevent a student from being able to enroll in more classes.

For students with SMI, program managers indicated that these academic challenges can be compounded. For these students, respondents indicated that first semester anxiety levels are particularly magnified. Assistance was perceived to be needed for these students as soon as possible to minimize future crisis interventions. Furthermore, periods of poor mental health and hospitalizations can often lead to missed classes and risks of failing or having to withdraw from classes, putting future financial aid in jeopardy. Along with early intervention, program managers reported that well-developed relationships with the professors and the mental health providers are needed to allow for the participant to remain as a student, as well as finding the space and time for students to continue their studies while receiving temporary inpatient mental health care.

Program participants are sometimes also challenged by substance abuse issues. One program manager noted that participants sometimes need counseling about the potential impact of their substance use not only on their academic trajectory but also for future employment (e.g., the need to pass a drug screening). Another program manager noted that participants have limited work experience and often needed help writing resumes.

Service Array

The SEd program service array can encompass a broad set of services designed to support participants in reaching their educational goals. According to program manager discussions, the framework for these services often starts with a participant-focused model that helps the participants define their educational and/or employment goals. Supports are then built around the participants to provide the services that they will need to accomplish their goals. Several respondents reported that an essential component of the participants' success in the program was remaining participant centered, and designing the services the participants need around their individual educational and employment goals.

As noted in the literature review, there are several potential components to a SEd program service array. Respondents to the environmental scan spent the most time describing three aspects of their programs:

• Academic support;
• Outreach to other service providers and potential program participants; and
• Peer support.

Academic support services described during the environmental scan discussions included assistance getting into school, working with teachers/professors about individualized accommodation needs, tutoring, using assisted technology to support disability needs, providing knowledge and instruction about skills needed to succeed in college (e.g., study skills, note-taking, time management), assistance withdrawing from classes, and assistance obtaining and maintaining financial aid. Program managers also described more generalized services that extended beyond academic skills to provide support for individual barriers that might affect the participant's ability to reach his or her educational goals (e.g., medication management, housing, transportation).

The outreach services described by program managers included connecting with organizations in the community about the services provided by their SEd programs, as well as outreach to students who could be potential program participants.

Program managers often described peer support as an important component of SEd services: someone who has "been there" provides participants with peer-to-peer feedback about their progress. For example, in the Quinsigamond Community College program, peer mentors were students who had previously received SEd services, were in recovery, and were succeeding in school. These students had mentoring relationships with two to three students and worked 8-10 hours per week. Despite these reports, peer mentors were also described by respondents as a frequently cut service component because of funding concerns. Less common services described by respondents included cognitive training, aerobic exercise, and working with family members.

Staffing

Programs providing SEd services were often described as including a team of staff members who provide program management, wellness support, and case management. Support staff team members named by respondents included program directors, peer mentors, nurse practitioners, social workers, psychologists, psychiatrists, counselors, occupational therapists, case managers, and SEd and/or SE specialists. Most teams were described as including 3-5 of these staff members who worked part-time or full-time within the individual program. Depending on the program setting, most programs had multiple case managers who served a range of participants in the program, with anywhere from ten to 25 participants per case manager depending on the program. The range of participants served appeared to be determined by program requirements to maintain staffing ratios. But some program managers noted having to reduce the number of case managers (and consequently increase caseloads) because of funding shortfalls. Program managers described the educational level of staff members on the service team as typically BA or MA level or with equivalent work experience, depending on the specific position. For some programs, regardless of educational background, team members received additional training in the IPS model, resiliency training, or positive psychology.

The dedicated staff member providing SEd services was called either a SEd specialist, SE specialist, SEd and SE specialist, or an IPS specialist. When a program had only an SE specialist, the specialist also provided SEd services (again noting that participants often have educational goals along with their employment goals). The majority of programs had a full-time staff member in this position. For some programs, this full-time status was considered an essential service component and was required as a part of their service delivery model. Meanwhile, although respondents described it as ideal to have a full-time dedicated SEd specialist, some admitted that it cannot always be a reality. One program described training a whole service team in the IPS model because it could no longer support a dedicated program staff member. This program had one lead team member who was an expert in SEd and championed this approach throughout the team. She provided ongoing training to all staff and gave all program psychiatrists a book on the IPS model. Even with this approach, the program manager noted that the model suffered without a dedicated SEd staff member on the team.

4.3.2. Service Setting

Program Differences by Setting

Environmental scan respondents indicated that when SEd services were provided on a college campus, they tended to be a SEd-only focused program. For example, Delaware's Division of Vocational Rehabilitation provides SEd services in partnership with its community college system to all students with disabilities. Program staff are housed directly within the college systems and do not provide employment supports. Other respondents from state VR departments and the VA system described the provision of both SEd and SE services. Respondents who represented programs embedded within mental health agencies described the most variation in their SEd program service array. Some provided SEd services within their SE program, others provided SEd and SE services separately but with equal importance, and others provided SEd services alone.

One researcher mentioned that hospitals are another promising setting that have historically provided SEd services. She indicated that this setting does have limitations in terms of the students being able to leave the hospital campus; however, she thinks this setting holds promise for a focus on pre-college academic skills and skills to manage the demands that might be encountered during the transition to a campus setting.

More information on respondents' thoughts about service setting implications for SEd program implementation are discussed further in Section 4.5.3, Program Implementation across Service Settings.

Integration of Supported Education and Supported Employment

Most program managers reported that their programs used an IPS model. IPS is an evidence-based SE service model that aims to help individuals with mental illness gain and maintain employment. It is an approach to VR designed specifically for individuals with mental illness grounded in the philosophy that all individuals with mental illness are capable of working in the community. Many environmental scan participants described the IPS model as an example of how SEd services should be provided; that is, by integrating SEd services with employment services. Respondents described the SEd service array as fitting well within the IPS model. Respondents noted again how commonly participants move between primary education and employment goals and their sense that it is better to keep individuals with the same program and provider(s) throughout these shifting goals. One researcher believed that SE and SEd services could be combined, but it should be done as part of a team approach. She thought that the SEd service component should be provided by one dedicated staff member explicitly focused on education supports within this larger team.

Some respondents did raise concerns about the integrated SEd/IPS service model. These respondents noted some philosophical differences between the two models. For example, one respondent noted that the IPS model may push people into rapid employment. In a more traditional SEd model, participants are encouraged to have some work experience before finishing school, but they are also supported to leave employment for further continued education. The respondent commented that participant employment goals can vary--they can be simply to get work experience or be more targeted toward longer-term employment in a specific career field. These different employment goals likely have different paths with varying educational needs. Also, in a similar example, another respondent noted that it is against the IPS model to build employment skills through volunteer work, yet this respondent believed it was important to encourage these types of volunteer opportunities as steps toward self-confidence and pre-employment skill building. These volunteer experiences were seen as valuable to building a work portfolio but contrary to SE aims of rapid employment.

4.3.3. Primary Partners for Program Implementation

SEd program implementation includes partners with other organizations that enhance, support, and expand on the educational support services provided. Those partnerships include ones that benefit the participants directly, as well as partnerships that ease and facilitate program implementation. Common partnerships mentioned by environmental scan respondents included mental health agencies, veterans services, group homes, clubhouses, VR, hospitals, and schools. The UCLA Aftercare Research Program noted that its partnership with their department of VR was very beneficial in that it provided assistance with the cost of education for participants in trade schools and for specific job skills. Another key partnership respondents mentioned was with college campuses that offered tutoring programs, mental health services, an office for disability services, career services, and campus faculty.

A few program managers mentioned partnerships that do not directly support participant needs but instead ease and enhance SEd program implementation. One program described its participation in the Early Assessment and Support Alliance (EASA) on the West Coast. EASA makes up a state network of programs providing SE and SEd services. EASA provides individual programs with resources on implementation and quality assurance guidelines. Another respondent mentioned the OnTrackNY program on the East Coast that brought in consumer expertise to train its providers to be more participant centered, be more comfortable with rehabilitation language, and have a greater focus on helping participants access disability benefits. Respondents valued staff training and access to program implementation resources in the midst of program management.

Several program managers noted missing partners as well as challenges with building needed partnerships. Although most programs partnered with general or psychiatric hospitals, these partnerships tended to focus on participant referrals. Some respondents lamented that a higher level of partnership with psychiatric hospitals and specialty mental health treatment providers would be helpful in supporting participants while in the SEd program. One respondent commented that the participant's status as a student should be more frequently considered by mental health providers as they make treatment decisions. For example, she suggested that providers might consider postponing midsemester medication management changes unless they were absolutely necessary. Or, hospitals could make it easier for a student to remain a student even during an inpatient hospitalization stay during the semester. Respondents believed that not accounting for a participant's student status sometimes led to the derailment of educational progress. Better partnerships with hospitals and doctors, including primary care physicians, could increase positive outcomes for participants. Respondents noted, however, that these partnerships take time to develop. The biggest challenge identified by several respondents was the time necessary to establish real partnerships with the diverse array of providers and service programs necessary to coordinate and meet participants' needs. They mentioned that funding barriers often do not allow for sufficient time to be devoted to building and sustaining these partnerships.

4.3.4. Financing

Complications and difficulties associated with SEd program financing were common themes across all of the environmental scan discussions. Environmental scan participants described multiple funding streams used to support SEd service components, but there did not appear to be one, single strategy to fund SEd services. In the absence of a core funding strategy, programs draw from many different funding vehicles that vary in terms of their stability and sustainability. To highlight the variability in financing strategies across programs, some programs' specific service funding approaches and some funding challenges experienced are described as follows.

Research Funding

One program respondent indicated that their program was almost exclusively supported by research grant dollars. Research grant funding enabled this program to provide its services free of charge to participants. This program had a long history of strong university partnership and relatively consistent research grant funding. However, the program manager described difficulties associated with this research funding reliance. She noted that the funding agency priorities changed over time; successful grant applications, consequently, had to change foci to meet the funding agency's priorities. Successful grant applications test something new or adapted; once a particular approach has demonstrated positive outcomes, the researcher has to move onto another viable funding idea. This program manager felt that relying on research grant dollars forced her program's service model to shift slightly over time. She believed that the economic and funding issues were dictating the level of care. Also, to compensate for the sporadic grant funding, this program was actively seeking funding from donors and endowments.

Block Grant Funding

Another program largely received its funding from its state's SAMHSA Community Mental Health Services Block Grant. Individual sites then secured various types of grant funding to support the portions of their programs not reimbursed through the block grant funds. The respondent described that one SEd program site within his state had a SAMHSA Healthy Transitions grant; Enhance OnTrack provided funding for two other sites through its block grant. A fourth site received a smaller amount of state reinvestment grant dollars to pay for SEd staff and training. The individual program site models were shaped by the various discretionary grant funding priorities.

Vocational Rehabilitation

Strategies to promote positive educational and employment outcomes for individuals with SMI can be supported through state VR agencies. State VR agencies are designed to promote the employment of people with disabilities. When creating an Individualized Plan for Employment, VR will consider providing any service needed to achieve an agreed-upon vocational goal. This service can include payment for education or training, including college tuition and related supplies. Consequently, VR funding can be one way to support SEd services. For example, Delaware's Department of Vocational Rehabilitation funds SEd-type services for all students with disabilities served by their technical college system. As a stakeholder from Delaware described, the students served by the Delaware program do not typically have a primary mental health diagnosis. However, she mentioned that several students have secondary mental health problems, but their SEd service model is not intrinsically designed for students with psychiatric disabilities. Funding for these services come exclusively from VR dollars. These dollars support both VR transition counselors (housed at the vocational-technical college) and college counselors.

One respondent indicated that VR dollars are very attractive to her program because they have a very high federal match rate, which is a "huge incentive to find ways to capitalize on these dollars." However, as another respondent described, VR funding is not intended to provide the longer-term educational services and supports often needed by individuals with psychiatric disabilities to truly attain an educational goal. He commented that VR-funded services have to be more geared toward education needs that are very explicitly directed at facilitating employment. A few respondents who discussed VR funding for SEd services appeared to recognize that these dollars would have to be supplemented by other funding sources, especially for those needing longer-term support, such as individuals with SMI.

College or University Support

SEd programs often function in partnership with technical colleges or other university systems. Many respondents noted that their college partners valued their services. They sometimes described colleges providing office space, for instance, for education specialists. One respondent did describe a shrinking budget climate for the higher education institutions in their state, making it a difficult fiscal climate for SEd program support. On the other hand, one program was directly funded by its college partner. This program had experienced more than 15 years of funding through community mental health; however, the funding ended. At that point, the program found a new home by integrating the SEd program into the college-based disability services program. Now the program manager's position was funded through the college, and the college paid for tutoring services. This program manager indicated that the college tried to reimburse itself for these services through discretionary grant funding.

Veterans Health Administration

Many veterans also have behavioral health conditions. Consequently, many GI Bill enrollees suffer from war-related traumas and other behavioral health problems that can create significant challenges in the pursuit of their education. To stay on track for achieving their educational goals, these veterans likely need appropriate and accessible supports such as those offered by SEd. Two different respondents noted a sense that the VHA is aware of this problem and is a promising funder for SEd services. One respondent noted that SE is nationally implemented in the VHA. Also, because the VHA has a strong history of incorporating SE into its health care services, it could be a platform for more widespread SEd implementation.

Medicaid

Many environmental scan respondents noted that their programs bill Medicaid to support relevant services wherever possible. However, as one respondent noted, billing Medicaid requires the successful defense of services as a "medical necessity." This criteria is not always a good fit with many SEd services. Respondents noted that their programs were most frequently able to bill Medicaid for the case management function involved within their SEd programs.

Many respondents noted the difference between SE Medicaid billing and that for SEd services. They were aware that Medicaid did have a specific SE billing code and often described SE funding as "well established." In fact, programs with a joint SE/SEd program described billing SE/SEd specialists under the Medicaid SE billing code. Meanwhile, several respondents stated that funding for SEd was unclear, largely because SEd had no Medicaid billing code. One respondent believed that it might be hard to get approval for a SEd Medicaid code. She suggested that most funders want to see evidence of direct program impact; however, demonstrating the most critical outcome for SEd programs (degree attainment) often takes multiple years. This respondent believed that it is hard to solicit funding for SEd when there is such a lengthy time lag between the preliminary SEd program intervention enrollment and its ultimate primary outcome of interest.

Federal participants in the environmental scan described the availability of 1915(c) waivers to support "employment and employment related services." According to the September 16, 2011, Center for Medicaid and CHIP Services (CMCS) Informational Bulletin, SE and "prevocational services may be furnished...under the provisions of §1915(c)(5)(C). They may be offered to any target group for whom the provision of these services would be beneficial in helping them to realize their goals of obtaining and maintaining community employment" (CMCS Informational Bulletin, September 16, 2011). Within this Medicaid provision, there is an SE-Individual Employment Support core service definition. SE-IES services are defined as "the ongoing supports to participants who, because of their disabilities, need intensive ongoing support to obtain and maintain an individual job in a competitive or customized employment, or self-employment, in an integrated work setting in the general workforce for which an individual is compensated at or above the minimum wage, but not less than the customary wage and level of benefits paid by the employer for the same or similar work performed by individuals without disabilities. The outcome of this service is sustained paid employment at or above the minimum wage in an integrated setting in the general workforce, in a job that meets personal and career goals." This is the SE Medicaid billing opportunity referred to by several SEd program managers who participated in the environmental scan. No comparable waiver or core service definition exists explicitly for SEd services.

Federal officials who participated in the environmental scan also described the possibility of using Medicaid reimbursement to cover the costs of educational services through the 1915(c) Home and Community-Based Services waiver option. However, CMS stakeholders also emphasized that Medicaid is intended to be the "payer of last resort." Medicaid dollars are intended for services that cannot be supported by other sources. So, consistent with that, the core definition of the Home and Community-Based Services waiver option states a requirement that "educational services consist of special education and related services [as defined within] the Individuals with Disabilities Education Improvement Act (IDEA), to the extent to which they are not available under a program funded by IDEA." For example, if transportation between a participant's home and an education services site is provided as a component of a 1915(c) request, and a state proposes the cost of this transportation to be included in the Medicaid rate paid to the providers of education services, the state has to include a statement indicating that these transportation costs are not already covered by IDEA.

Stakeholders from the U.S. Department of Education contacted through the environmental scan confirmed that IDEA supports education services for individuals up to 21 years of age. However, the SEd program managers did not describe use of Medicaid waivers to reimburse for educational services. It appears that SEd program stakeholders find funding for education services for individuals older than age 21 (and consequently outside the context of IDEA) more challenging.

4.3.5. Federal and State Policy Context

Several environmental scan respondents referenced the Americans with Disabilities Act (ADA) as the guiding and framing policy for SEd work. The ADA philosophy emphasizes respect for the privileges of all students and enables the provision of necessary education and workplace accommodations. Consequently, respondents felt that it is the ADA law that promotes a philosophy of inclusion for individuals with psychiatric disabilities. However, a few respondents also emphasized that ADA accommodations, particularly on college campuses, are still traditionally focused on physical disabilities (or maybe learning disabilities), with much less emphasis on psychiatric disabilities. These respondents noted that dealing with students with SMI is hard for the staff of disability service offices on college campuses. There was a sense among respondents that staff in these offices often lack a solid understanding of how to accommodate psychiatric disabilities. One respondent mentioned that the episodic nature of psychiatric disabilities is more difficult to accommodate than typical physical disabilities. Psychiatric disabilities often require inter-personal supports at various levels of intensity over time, rather than explicit and often more stable accommodations (e.g., ramps, allowance for animal support partners in buildings, computer support tools for physical impairments). So, although ADA may be the guiding policy for SEd work, some respondents sensed that its philosophy was not being fully embodied for all students with disabilities across typical college campuses.

A policy related to the ADA is IDEA, which is a law that ensures services for children, adolescents, and young adults with disabilities across the United States. IDEA governs how states provide special education and associated services to students aged 21 years or younger. IDEA Part B supports special education services for children and youth aged 3-21 years. Individuals who qualify for IDEA services receive an IEP that describes the types of public special education services that those individuals are eligible to receive. Progress toward meeting educational goals is assessed and measured routinely within the context of the IEP.

Another relevant policy associated with SEd program implementation and mentioned by respondents was SSI. Program manager responses differed in terms of how SSI influenced their SEd program implementation. For instance, one manager indicated that her program tries to steer participants away from getting SSI in order to keep them focused on employment or educational goals. Meanwhile, another program manager noted that one difficulty of program implementation was tracking participant hours worked to ensure that students did not work so much that they risked losing their SSI or Social Security Disability Insurance (SSDI) benefits. There are likely real tensions between the perceived risks of facilitating participants' long-term dependence on SSI for income versus the immediate financial needs that many SEd program participants face that might be eased by SSI/SSDI support.

Several respondents mentioned expanding federal and state interest in transition-age youth and early intervention services for individuals with SMI. For instance, a program manager from Oregon specifically mentioned this state's interest in transition-age youth and early intervention services for first-episode mental illness. Her program was trying to work with the state to consider various requirements for insurers to provide early intervention services along with potential regulations and service provision recommendations. This increasing emphasis on the transition-age population and early intervention can also be seen in recent changes to some federal funding priorities, including changes noted in the Workforce Innovation and Opportunities Act (WIOA) and SAMHSA's Community Mental Health Services Block Grant program.

Workforce Innovation and Opportunities Act

State departments of VR are funded by federal dollars that require a state match. These VR dollars can support some education-related costs for individuals with disabilities (e.g., tuition, books), provided that the education is necessary to achieve a longer-term vocational goal. The traditional VR service population has been mature adults (typically with schooling completed). However, the recent reauthorization of the WIOA changes how states are to spend VR dollars. The revised WIOA indicates that a portion of state VR dollars should be allocated for pre-employment services for transition-age individuals (specific ages are defined by states but are typically 15-21 years). Although this law does not reference SEd specifically and is not designed explicitly for SEd service support, one other stakeholder indicated that changes in the WIOA reauthorization may represent an opportunity for SEd service funding support by states in the future. This stakeholder indicated that her state is now thinking about how to serve students with disabilities earlier, before high school completion, with a renewed focus on career exploration, internships, self-determination counseling, and college preparation supports.

Serious Mental Illness Early Intervention Set-Aside in SAMHSA Community Mental Health Services Block Grants

A respondent noted one potential opportunity to expand SEd services: the new 2014 SAMHSA Community Mental Health Services Block Grant priority focused on early intervention for individuals with SMI. In 2014, Congress directed SAMHSA to require that states set-aside 5% of their Community Mental Health Services Block Grants to address the early intervention needs of individuals with SMI. A priority described by SAMHSA is for early intervention strategies to reduce the likelihood of long-term disability that people with SMI often experience. The block grant dollars are intended to help states supplement Medicaid, Medicare, and private insurance funding to provide prevention, treatment, and recovery support programs. States are encouraged to consider evidence-based practices such as Coordinated Specialty Care (a model supported by the NIMH-funded Recovery After an Initial Schizophrenia Episode [RAISE] research initiative) and OnTrackNY (one of the programs included in the environmental scan discussions). This block grant opportunity could support early intervention services including SEd or SE service components.

4.4. Supported Education Research and Evaluation

To supplement information gathered from the literature review (see Chapter 3), environmental scan respondents were asked to describe ongoing research and evaluation projects--the scope of these projects, early findings (when available), the types of data collected, challenges and solutions to data collection problems, and funding for SEd research and evaluation. This section describes some of these ongoing research projects and manuscripts noted as in press by authors. The list is not exhaustive; it represents only those studies explicitly mentioned by environmental scan respondents.

4.4.1. Ongoing Research Projects and Manuscripts in Press

SEd researchers were queried by email about ongoing research projects and manuscripts in press with a focus on SEd. In response to this email request, Drs. Smith-Osborne, Mueser, and Manthey sent information about work in progress, as well as papers in press or under review.

The Student Veteran Program involves ongoing research led by Dr. Alexa Smith-Osborne. One current project is a RCT of undergraduate student veterans. The Student Veteran Program is open to any veteran and offers free, specialized admissions and counseling services. Preliminary program data indicate that 50% of the sample has a diagnosis of PTSD. The primary goal of the program is dropout prevention. Support services last for two semesters and involve both face-to-face and distance support to veteran students. One unique component of the program is the use of teleherence as part of the case management model. Teleherence provides automated scheduled calls to program participants for appointment reminders, to broker external services, and to provide booster or motivational messages to support goals and encourage actions toward participant change. New veterans are being enrolled in the program through 2015. Outcome data have not yet been analyzed to determine whether the SEd intervention can be effectively adapted for veterans with mental health issues.

NIMH's RAISE initiative focuses on the development and evaluation of first-episode treatment programs designed for the United States health care system. The premise of the NIMH RAISE Early Treatment Program (ETP) was to combine state-of-the-art pharmacologic and psychosocial treatments delivered by a well-trained, multidisciplinary team to significantly improve the functional outcome and quality of life for first-episode psychosis patients. An article currently in press in the Journal of Clinical Psychiatry (lead author Dr. John Kane) presents information on the overall development of the core RAISE intervention and the design of the clinical trial to evaluate its effectiveness (Kane et al., 2015). The RAISE study enrolled patients 15-40 years old with a first episode of schizophrenia, schizoaffective disorder, schizophreniform disorder, psychotic disorder not otherwise specified, or brief psychotic disorder and a history of no more than 6 months of antipsychotic medication treatment. Patients were followed for a minimum of 2 years, with major assessments conducted by blinded, centralized raters using live, two-way video. Thirty-four clinical sites in 21 states were selected for participation; 17 were assigned to the experimental treatment and 17 to usual care. Enrollment began in July 2009 and ended in July 2011 with 404 total subjects enrolled. Results of the trial will be published separately at a later date.

Another paper in press in the journal Psychiatric Services (lead author Dr. Kim Mueser) describes the background, rationale, and nature of one intervention developed by the NIMH RAISE ETP project, the NAVIGATE program. This article has a particular focus on the psychosocial components of the NAVIGATE program. NAVIGATE is described as a team-based, multicomponent treatment program designed to be implemented in routine mental health treatment settings and aimed at guiding people with a first episode of psychosis (and their families) toward psychological and functional health. One component included in the approach is SEd. NAVIGATE is currently being compared in a cluster RCT with usual community care as part of the NIMH-funded RAISE research project.

Dr. Trevor Manthey and his colleagues have a paper under review that examines the characteristics of more than 1,500 clients with psychiatric disabilities receiving community mental health services. Logistic regression analyses were used to measure the impact of various sample demographic characteristics on higher education outcomes. Significant differences were found for gender, age, race/ethnicity, diagnosis, work history, and substance use. Clients with bipolar disorder or major depression had greater odds of having a higher education than those diagnosed with schizophrenia. Clients with a recent work history were five times more likely to have higher education. Individuals who do not use illegal substances were more likely to have higher education.

Dr. Manthey and his colleagues have a second paper under review that explores the educational goals of a small sample of individuals with psychiatric disabilities who did and did not want to return to school. Concerns about returning to school noted by both groups of students were teachers' lack of understanding of mental illness, lack of professional support, experiences with stigma, and financial burden. Individuals interested in returning to school were more likely to have a drive for education and love of learning, greater familial support, and greater perceived support from case managers than those without an interest in returning to school.

Other ongoing research projects were briefly described by the researchers who participated in the environmental scan telephone discussions or by the NIDRR project officer. These projects include the following:

• A recently completed RCT of SEd led by Dr. Mark Salzer at Temple University, with Michelle Mullen at Rutgers University as a collaborator. This project was supported by NIDRR. Outcomes from the study have not yet been published.

• A NIDRR/SAMHSA-funded research and training center (Transitions RTC) directed by Dr. Maryann Davis at the University of Massachusetts Medical School. The Transitions RTC activities are focused on developing knowledge for and about developmentally appropriate services that help transition-age youth and young adults with serious mental health conditions successfully complete their schooling and training and launch their adult working careers.

• Two ongoing projects led by Michelle Mullen at Rutgers University: (1) a project to develop a cognitive remediation training manual to improve executive functioning in the context of an IPS approach; and (2) a NIDRR-funded project being conducted in collaboration with Dr. Marsha Ellison at the University of Massachusetts Medical School. This grant is examining a career development approach for transition-age young adults. The grant will involve a literature review, qualitative interviews, manual development, and program testing.

• An ongoing NIDRR grant to Dr. E. Sally Rogers at Boston University examining participant employment and education outcomes resulting from a SEd/SE service model. One component of this project will be conducting qualitative interviews with providers. This grant is co-funded with SAMHSA.

4.4.2. Funders for Supported Education Research and Evaluation

Three funders for SEd research and evaluation were described by environmental scan respondents: NIDRR, SAMHSA, and NIMH. NIDRR was currently funding the most SEd research, sometimes with co-funding from SAMHSA. The NIDRR project officer indicated that her organization had supported grants focused on SEd since 1995. These grants are largely funded via the NIDRR "field-initiated research project" mechanism. This mechanism supports 3-year projects that are most typically investigator initiated but can sometimes be guided or directed by NIDDR.

Program managers who participated in the environmental scan also noted grant support from SAMHSA for SEd program evaluation through Now Is The Time Healthy Transitions grants. This grant program is designed to create access to treatment and support services for youth and young adults aged 16-25 who either have, or are at risk of developing, a serious mental health condition. Grantees are asked to increase service awareness, screening and detection, outreach and engagement, referrals to treatment, coordination of care, and evidence-informed treatment for this age group. All grantees are required to have a local evaluation.

The NIMH project officer who participated in the environmental scan indicated that NIMH has funded SEd research in the past but has no grants currently focused exclusively on SEd. NIMH is funding the RAISE initiative, which examines the impact of an early intervention approach that includes elements of SEd. The NIMH project officer noted her agency's specific interest in treatments to remediate symptoms associated with early psychosis.

4.4.3. Primary and Administrative Data Used in Research and Evaluation

According to researcher respondents in the environmental scan, data capturing the outcomes of SEd programs largely come from specific program evaluation measures and scales designed to capture project-specific outcomes. These measures generally capture educational outcomes, employment outcomes, and, depending on the program, mental illness symptomology and measures of wellness or life satisfaction and quality. Researchers noted indicators of educational and employment outcomes, along with standardized scales, such as the Quality of Life Scale, Recovery Achievement Scale, and the Brief Symptom Inventory.

College transcripts were mentioned as an administrative data source. They could be used to measure a number of academic milestones, including the number of classes in which a student is enrolled, GPA, course completion, and progress toward degree attainment.

Data Source Challenges and Solutions

There was not consensus about which sources of data are best suited to represent SEd program outcomes. Some respondents believed that this was due to the breadth of SEd program goals. One researcher commented that it is unclear whether the primary goals for SEd programs are educational attainment, employment, reduced psychological symptoms, increased life skills, or life enrichment. This presents a challenge to the use of central, standardized measurement protocols that might be used to assess SEd program impact across studies.

Respondents expressed a desire to collect objective, standard indicators of educational attainment, particularly information contained in student records and transcripts. Unfortunately, a few researchers described difficulties in collecting college transcripts. First, permission needs to be obtained directly from the student to collect this information (a third party cannot request it directly). Then, colleges sometimes hold transcripts until all of a student's outstanding fines are fully paid (i.e., parking tickets, library fines, tuition). One project has students complete transcript request forms at the point of program entry, and the project then directly submits these transcript requests. Finally, some college systems use clearing houses to process transcript requests. Respondents indicated that this third-party relationship makes access to student records and transcripts even more difficult.

Collecting data directly from the program participants can also be challenging, particularly when attempting to follow respondents several years after program involvement. To get around the barriers of collecting long-term follow-up data, one researcher noted using 5 years of retrospective data to capture information on educational, employment, and psychological history. This information was used to demonstrate how program participant outcomes had improved. Another approach described to increase participant response rates was to offer multiple modes of data collection administered outside of the SEd program. Some respondents noted that they collected data over the phone, using web-based instruments, and, in keeping with their community-based model, during meetings with participants in the field to complete outcome measures.

4.5. Perceived Gaps in the Supported Education Knowledge Base

Researchers, program managers, and other stakeholders all described perceived gaps in the SEd knowledge base. These gaps fell into three areas: gaps in agreed-upon definitions of SEd programs and goals, gaps in knowledge of how to implement and fund SEd programs, and unanswered research questions related to program outcomes. Comments from respondents to the environmental scan are summarized across these areas as follows. These comments set the stage for issues to be considered as future SEd program development, research, and evaluation work moves forward.

4.5.1. Supported Education Program Model Definition

Despite having common knowledge of SEd service components, almost every environmental scan respondent mentioned the need for increased SEd program definition. For example, when asked about unanswered research questions, one respondent replied, "Supported education--what exactly is it?" Respondents felt that there is no one commonly agreed-upon and well-validated SEd model. The development of such a model may have been hindered by the now limited relevance of early research on some types of SEd programs. Much early research on SEd focused on standalone classroom-based models; however, several respondents noted that this classroom-based model is now considered antiquated and is no longer consistent with current values around inclusion. Consequently, outcomes from this early line of research are now of only limited value.

Many respondents noted an increase in the use of integrated SE/SEd approaches in the field; however, a similar program definition limitation was noted here. Several respondents described the need to develop a truly integrated SE/SEd program model. Although many programs are attempting to integrate SEd into SE or IPS models, there are no guidelines for this practice that explain how this process should be managed most effectively. For example, one respondent commented that researchers and program managers increasingly have the sense that it is helpful to have separate staff in combined SEd/SE programs focused on education versus employment supports. Some lessons learned have indicated that shared responsibilities may lessen program effectiveness. However, this is largely based on anecdotal evidence in the absence of empirically derived program model guidelines.

Along with a lack of specificity in program definition, some respondents also perceived needs to make the goals of SEd more clear as well. As one researcher respondent commented, what is the ultimate goal of SEd programs--is it employment? Reducing psychological symptoms? Increasing life skills? This researcher pointed out that some in the field would argue that the goal of SEd is also to enrich participants' lives, not just to facilitate ultimate employment. Also, he mentioned that many participants comment about the fulfillment and personal growth associated with completing educational goals. But how do researchers measure and quantify this type of outcome, then integrate those findings into program definitions?

A final issue emerged during the environmental scan that also relates to program definition; this issue has to do with some perceived artificial separations that have developed over time between SEd and other specialty mental health treatments, as well as SEd and SE. Starting with SE, one program manager and researcher described a sentiment expressed by other respondents. She noted that the "artificial" separation between SE and SEd was an "artifact of our history" and a mistake. She went on to describe what is also summarized in the SEd literature; many participants have both education and employment goals and often move back and forth between these goals over time. This researcher/program manager indicated that a preferred model would be focused on career development in which educational and employment goals are tailored to a participant's age and developmental stage. However, definitions for this type of model would be even further from development than those for either SEd or SE separately.

Another "artificial" separation some respondents described was the consideration of SEd services as separate from other specialty mental health treatment. A few program managers expressed frustration that SEd programs and services were not routinely defined as core components of integrated specialty mental health treatment, particularly for individuals with SMI. This issue most often surfaced during discussions of SEd funding. One program manager explicitly stated that SEd services should not be considered (and funded) separate from other standard mental health services, but she noted that they are treated very separately in terms of billing. This same program manager commented that there is also a discrepancy in terms of funding longevity. SEd programs typically cover and support services for the participant's program participation for approximately 2 years, but many participants really need services for much longer. In that instance, programs face a situation in which SEd funds end, but traditional specialty mental health services continue to be reimbursed. There was a general sense among these respondents that considering SEd programs to operate outside of specialty mental health treatment might create inequities in terms of service availability and funding.

4.5.2. Program Model Fidelity

Several respondents in the environmental scan mentioned gaps in the field's ability to track SEd program fidelity and a more general need for user-friendly fidelity measures on SEd. A few respondents mentioned the University of Kansas Supported Education Toolkit 3.0 (Manthey et al., 2012a) as a tool for measuring the fidelity of SEd. However, some respondents either did not know of this toolkit or thought that it did not capture the information necessary for measuring the fidelity of SEd program implementation, particularly with regard to program quality. Another researcher noted that the University of Kansas fidelity tool had not yet been widely tested across programs and that testing was needed. Such testing would allow the tool to be validated and ideally shortened to include essential predictive items.

Another respondent mentioned how helpful it would be to have something more like an "implementation" assessment, rather than a fidelity tool. In this respondent's mind, an implementation assessment would examine the degree to which various SEd program components have been implemented across target agencies and with what type of quality. This respondent thought that guiding program implementation recommendations were as critical to define and measure as core program characteristics.

A secondary issue related to SEd program fidelity that surfaced during the environmental scan was related to the integration of different models and how this might alter program composition and implementation. Different program models for SEd may all have their own core components; one respondent commented that mixing and integrating program models may lead to increased problems in measuring fidelity.

4.5.3. Program Implementation across Service Settings

Several environmental scan respondents noted some gaps in knowledge about how best to implement SEd programs across a variety of settings. One researcher, in particular, provided a concise summary of some key implementation challenges experienced in SEd program implementation:

• Lack of clarity about the program model;
• Lack of service funding;
• Poor integration with other services received by the participant;
• Lack of effective mechanisms to provide consistent, ongoing support for participants across settings;
• Difficulties securing buy-in from college campuses; and
• Length of time necessary to demonstrate the achievement of educational outcomes.

Respondents discussed the advantages and disadvantages of various settings within which to embed SEd approaches, along with the types of knowledge needed to inform which settings might be best suited for which types of participants. Respondents had several thoughts about what might work best across settings, as well as unanswered questions that would provide helpful information to guide program implementation.

• Vocational Rehabilitation: Many respondents described working with local VR departments and services. Some respondents commented that VR departments and programs are likely seeing more individuals diagnosed with mental health conditions. VR generally appeared to be perceived as a natural and important partner for implementing SEd services. However, some respondents commented that VR services may not be well positioned to provide longer-term support services; longer supports may be necessary for serving individuals with more SMI.

• Campus-Based Disability Services: Respondents also described the importance of partnering with college campus-based disability offices. This service setting was perceived as a good nonstigmatizing way to engage students. Respondents described the advantages of combining SEd services under the umbrella of college disability services because students often have more than one disability. The team-oriented approach used within college disability service offices might enable better communication about the different services that each student may need. One program manager of a campus-based SEd program mentioned this location as a huge benefit. She believed that it reduced the redundancy of work conducted by campus personnel and increased the teamwork between on-campus programs and staff as they worked to provide comprehensive services for the student. One researcher mentioned that college campuses would be the ideal location for "pure" SEd-only programs (as opposed to those combined with SE). Meanwhile, that same respondent also indicated that a campus-based setting is likely not best suited to serve students with SMI.

• Clubhouses: The clubhouse service setting is set up to focus on individuals with psychiatric disabilities, particularly those with SMI. This could be attractive in offering peer support to pursue educational goals. However, a few respondents indicated that some students might see clubhouse-based SEd services as stigmatizing because participants often want to identify themselves as "students" and not "patients" or persons with a psychiatric disability.

• Community Mental Health: Community mental health treatment systems offer potential for a central, integrated model for providing comprehensive mental health treatment. One researcher believed that the community mental health setting is likely the best setting within which to implement a SEd/SE model. This location, in his opinion, is the best setting for managing and coordinating the diverse needs of individuals with SMI (such as supporting individuals with first-episode psychosis).

Some unanswered questions related to SEd program implementation that respondents described included issues related to SEd program participants, staffing, services, and collaboration:

• Participants:

  • How long should participants be enrolled in SEd programs?
  • What are the most appropriate populations to be targeted for receiving SEd services (e.g., specific diagnoses, functional severity, developmental life stage)? Should programs accept all interested persons or have explicit eligibility criteria?

• Staffing:

  • What types of staff are best suited to provide SEd services? What types of skills should be required?
  • Can SEd services be provided by any staff member on a service team, or should they be provided by a staff member dedicated for this purpose? Can that role be combined with the staff member who is also providing SE services?
  • How many hours should an educational specialist work? Should they be full or part time? How many hours should they be in the office versus out in the community?
  • What is the role of peer support in SEd programs? Can peers serve in an educational specialist role? For what roles are peer support paraprofessionals best suited?

• Services:

  • How should SEd services be integrated into other services that a participant may be receiving? Is there a particular order or sequence of treatment components in an integrated model that is most effective?
  • What types of SEd programs are best suited for implementation in which types of service settings?

• Community partnerships and collaboration:

  • What types of partnerships are needed to successfully support participants in reaching their educational goals?
  • How can SEd programs work within communities to create more opportunities for skilled part-time employment? One respondent mentioned that many of her program's graduates would be intellectually capable of holding positions in a career field like biotechnology, but that maintaining full-time (as opposed to part-time) employment would be very difficult.

4.5.4. Research and Evaluation Challenges and Needs

In the midst of the unstructured discussions, environmental scan respondents described several challenges to research and evaluation on SEd programs. The largest challenges to research and evaluation related to problems with the SEd program definition and the more general need for researchers interested in this area of study:

• Problems defining and measuring core outcomes of interest:

  • Research respondents noted that many studies do not go beyond examining the number of participants involved to really understand whether programs affect any "real educational endeavor."
  • There is no clear consensus on how to calibrate achieving an educational goal. Acquiring a GED? 2-year degree? 4-year degree?
  • Educational goals and their purpose vary by individual. How do you distinguish and evaluate educational courses taken for the purpose of self-improvement versus those required to define a particular career path?
  • Getting good data on the attainment of educational milestones is difficult; academic records are often hard to acquire.

• Need for field leadership and champions to push the line of SEd research and evaluation forward.

• Need for new, emerging investigators focused on designing rigorous experiments on the impact of various SEd program models.

Some individuals specifically mentioned what they perceived to be missing in the SEd research and evaluation field as well as what specific types of studies they think are needed in the future. The research studies and topics described by environmental scan participants were very diverse and not comprehensive. The recommendations below represent points explicitly mentioned by individuals during the environmental scan unstructured discussions. Individuals noted the need for:

• Longitudinal studies:

  • Studies that follow individuals with mental health problems for several years as they enter school and the workforce. What factors are associated with staying in school through degree attainment and staying in the workforce?
  • SEd intervention outcome trials that follow program participants for well over 5 years. One researcher mentioned the potential need to follow SEd program participants potentially for 15 years to truly see employment and mental health service impact. She pointed out that it takes a long time even for successful SEd program participants to finish. Many SEd participants come in and out of services over the course of 5-7 years as they complete their educational goals. In her opinion, it should not be seen as failure that these students take longer than their peers to complete educational goals, but research designs have to accommodate this reality before concluding that SEd programs are ineffective.
  • A longitudinal multisite study of SEd programs' impact.

• Research on specific SEd models:

  • A progressive series of studies to refine one particular promising SEd model. This series would start with a study of smaller scope to test the outcomes of a well-defined SEd model with an emphasis on also developing a strong fidelity scale. Based on the results of this study, the SEd model would be tweaked and refined. Then, the improved model would need to be tested in a larger RCT.
  • Research that capitalizes on the progress that has been made in understanding how to implement integrated early intervention programs (that include components of both SEd and SE) for first-episode psychosis. One researcher wondered if there was a way to capitalize on the new funding priority to serve individuals with SMI in SAMHSA Community Mental Health Services Block Grants. Will there be the opportunity to track the type of services supported by this funding (e.g., SE, SE/SEd) and observe outcomes for individuals served by these dollars consistently across states?
  • Research on integrating SEd programs into psychiatric hospitals, with a particular focus on pre-college skill building.

• Research on program participants:

  • Studies that identify what kinds of problems colleges have with keeping these students with mental illness engaged and succeeding in their programs.
  • Studies that help illustrate which types of participants can benefit from what type of program emphasis. Do younger participants in their teens and early 20s benefit more from a stronger educational focus, whereas older participants benefit more from a stronger employment focus?

5. Site Visits

5.1. Introduction

In this chapter, we describe how individuals living with mental health challenges are supported as they pursue educational goals. We sought to understand how these supports are operationalized through the eyes and experiences of those who deliver these services. Individual case studies were conducted in settings in Oregon, New Jersey, and Minnesota, where the educational goals of people with mental health concerns are supported. We begin the chapter with a description of the methods used to choose the settings for study and the procedures for the site visits. A summary detailing the service structure, recruitment and engagement strategies, and successes and challenges, among other topics, is included for each setting. Following the three case studies is a synthesis of the important similarities and differences between the sites. This chapter concludes with a list of key findings that the case studies offer for the SEd field.

5.2. Methods

5.2.1. Identification of Case Study Sites

Our selection of sites was informed in multiple ways. We first searched for sites across the United States that help individuals with mental health concerns to pursue their educational goals. An initial list of possible sites was compiled from the literature review (n=10), which was supplemented with sites identified by key stakeholders during the environmental scan (n=13). These sites were reviewed by additional SEd content experts, who added to the list (n=2), resulting in a total of 25 unique initiatives.

The goal was to identify three sites for visitation. Criteria for site stratification were identified to maximize variation in the depth and breadth of the data collected across the three sites, and to highlight important constructs identified in the field of SEd as identified through the literature review and environmental scan. Primary selection criteria included: (1) having one site that targeted individuals experiencing a first episode of psychosis, a schizophrenia-related condition; (2) having one site based in a community mental health setting; and (3) having one site based in a post-secondary education setting. Secondary selection criteria included having geographic diversity among the three sites. Sites that served only a specific target population (e.g., veterans) were also excluded. In addition, environmental scan stakeholders and content experts were asked to nominate SEd programs or initiatives that were, in their opinion, exemplary, innovative, and worthy of site visitation; this resulted in the identification of 15 sites (a subset of the original 25). Sites selected for visits all received at least one nomination from a stakeholder or content expert. Investigators reviewed the remaining 15 sites and identified three that best met selection criteria.

5.2.2. Selected Sites

As a result of our stratified purposeful sampling strategy, we chose three sites for visitation:

• Early Assessment and Support Alliance: The EASA program is a statewide effort in Oregon to address the needs of young adults, which includes educational needs. EASA focuses on individuals experiencing a first episode of schizophrenia-related conditions.

• Learning Enhancement and Resource Network (LEARN): The LEARN program is a standalone SEd program based in a New Jersey community-based mental health center. LEARN supports individuals of any age with mental health concerns in achieving their educational goals.

• University of Minnesota: The University of Minnesota has a campus-wide initiative to support the mental health needs of all students. Their Provost Committee on Student Mental Health has prioritized mental health and wellness campus-wide, and has created a culture of attention and resources to support student mental health.

5.2.3. Site Visit Methodology

A leadership contact person was identified at each site. Investigators emailed site leaders to describe the study and the site selection process, and to ascertain interest in hosting a site visit. Leaders from all three selected sites agreed to participate. Investigators worked with site leaders over a period of 2 months to identify visit dates, discuss key stakeholders to meet with, and work on overall visit logistics. Site leaders were sent a list of domains and questions of interest (see Appendix A) to investigators and asked to identify which stakeholders were most able to address the proposed domains/questions. Investigators had at least one telephone call with each site to discuss draft itineraries and answer questions about the research.

Site visits were conducted in April and May 2015. Two investigators visited each site, and each site visit lasted 2 days. All visits began with a discussion with the identified site leader. Investigators met with some stakeholders one-on-one while others participated in group discussions. Two sites had seven discussions each, most of which were with groups of stakeholders. One site had 11 discussions, most of which were with individual stakeholders. Stakeholders ranged from program, agency or department leaders, to front-line providers, community partners, and individuals with mental health concerns who had participated in SEd initiatives. Each site visit included one or two group discussion with individuals with mental health concerns receiving support with their educational goals; participants for these discussions were recruited by the site leader. One site included participants who were all high school and/or college aged, a second site included college aged and graduate students, and a third site included college aged and mature adult students. Discussions were audiotaped during two site visits, while one site declined because it did not have the appropriate approval. None of the discussions with individuals with mental health concerns were recorded. All sites received a stipend for their participation. Individuals with mental health concerns who participated each received an Amazon gift card. This study received an internal review board exemption.

5.2.4. Data Collection

Domains of focus for the interview protocols were derived from findings identified in the literature review and the environmental scan, and were informed by investigators' previous site visit methodologies to describe innovative programs and make policy recommendations. The interview protocol addressed domains that include: overview of the program/initiative overview; history; services offered; participation engagement; staffing; financing; evaluation efforts; service context; and successes and challenges. A separate interview protocol was developed for individuals with mental health concerns with domains that include: how they were referred; what services and supports were offered; and satisfaction with services and supports. At each site visit, one investigator led the interview while the other took detailed notes on a laptop. Investigators traded interviewing and note-taking roles throughout each site visit.

5.2.5. Analysis

Immediately after each site visit, data were reviewed and cleaned by investigators who had participated in the visit. Data from each investigator were merged into one document and coded for concepts and themes based on the site visit discussion prompts, for example, services offered, participation engagement, and staffing. Increasingly specific and narrow categories of concepts and themes were defined within this framework to condense extensive raw data and to identify common themes. From these themes, a narrative case study was written for each of the three sites. Given the diversity among the sites, some themes spanned all three sites (e.g., funding), while other themes were more specific to individual cases (e.g., relationship between SEd and SE). Each individual case study was reviewed by the identified site leadership contact at least two times. All individual site visit case studies were approved by site leadership. Investigators reviewed the individual case studies for the cross-site analysis, and developed themes regarding sites' similarities, differences, and key findings. The reliability of findings from the individual case studies is enhanced by the coding of data by multiple investigators, the comparisons of these data with findings of previous research on initiatives to support the educational goals of individuals living with mental health concerns (McIntyre, 2008; Patton, 2015), and feedback from site leaders regarding the accuracy and integrity of the individual site visit reports.

5.3. Case Studies

5.3.1. Overview of Supported Education Dimensions across Sites

Table 5-1 provides a brief overview of basic SEd dimensions across each of the three sites.

TABLE 5-1. Summary of SEd Dimensions across Sites
	EASA	LEARN	University of Minnesota
Setting	Community mental health settings	Community mental health settings	4-year university
Service approach	Integrated with other young adult services	Standalone service	Integrated with other university services
Scope	Statewide	Multicounty	Campus-wide
Target population	First-episode schizophrenia-related conditions	Individuals of any age receiving community mental health services	University students
Primary staffing	Occupational therapists	Education coaches	Varies by academic organization
Primary referral sources	Hospital and outpatient mental health settings	Community mental health programs and campus counseling departments	Offices of disability, mental health services, and counseling
Financing	State mental health block grant and state general funds; some department of VR and Medicaid funding	State contract for SEd services	Varies by academic organization; very limited targeted funds

5.3.2. Early Assessment and Support Alliance

Overview

Oregon has a complex set of programs and initiatives, some statewide and some standalone, which exist to support individuals with mental health conditions in meeting their educational goals. The primary initiative in Oregon that helps individuals with psychiatric disabilities to achieve their educational goals is the EASA program. EASA began as a targeted effort to prevent early trauma and disability caused by schizophrenia-related conditions. This initiative was in direct response to the Oregon Health Authority's prioritization of the implementation of evidence-based practices, and had the expressed goal of minimizing disabilities associated with schizophrenia-related conditions. EASA began in 2001 in five counties across Oregon. In 2007, a mandate from the state legislature was introduced to begin disseminating EASA services statewide. To date, EASA has 24 teams in 36 counties in Oregon, and serves the majority of the state. EASA teams are operated by community mental health centers, and some EASA teams serve multiple counties. SEd has been a part of the EASA mandate since its inception.

In July 2013, the EASA Center for Excellence was established at Portland State University's Regional Research Institute. The EASA Center for Excellence provides training, consultation, and implementation support for the EASA programs in Oregon, and for other agencies or organizations interested in using elements of the EASA program model. The Center for Excellence works with EASA programs and other partners to carry out research and build new knowledge about how best to promote positive outcomes for young people experiencing psychosis.

Oregon also has other programs and initiatives focused on SEd. In 2007, Oregon developed pilot programs funded through a state block grant in three community mental health agencies exclusively devoted to SEd. In addition, Oregon is home to a Supported Employment Center for Excellence that includes a focus on SEd.

Early Assessment and Support Alliance Approach

EASA is part of a broader state-level movement to address the needs of young adults and to invest in their specialized needs, including educational needs, at this critical developmental juncture in life. EASA is a transitional program designed to provide services and supports for 2 years. Fidelity to evidence-informed interventions is a cornerstone of EASA since its inception. EASA has several evidence-based practices in its service array including person-centered planning, cognitive behavioral therapy, and IPS. EASA's array of services are based on evidence but also driven by the unique needs of each individual. EASA's practice guidelines encourage SEd services to be provided following the same principles of care as IPS, but there is no one endorsed model of staffing or activities for SEd that is promulgated across EASA sites. Each site decides this separately, dependent on the specifics of the site. As noted by one stakeholder, "We don't have a commitment to a specific model of SEd as much as a commitment to the educational needs as identified by young adults."

All EASA supports are driven by the basic question, "What are the goals of the young adult?" It is this shared approach and philosophy that creates coherence across EASA sites and services. EASA is committed to getting young adults the help they need, as identified by them, in a time sensitive manner--there are no waiting lists for services. In addition, EASA is committed to a participatory approach with young adults, and engages them in all aspects of the work. This includes active involvement in the program from participants receiving services, participation in a leadership group for EASA program graduates, and employing staff within the EASA program who have lived with the experience of mental illness.

Services and Supports

EASA services are based on practice guidelines that build on the work of the Australian Early Psychosis Prevention and Intervention Center as well as the SAMHSA evidence-based toolkits, including multifamily groups, illness management and recovery, dual diagnosis treatment (chemical dependency and psychosis), and SE. EASA services include: outreach and engagement; assessment, diagnosis, and treatment planning by mental health professionals specifically trained in early psychosis work; education and support for individuals and families/primary support systems; crisis and relapse planning; assistance with knowing rights and available benefits; goal-setting and planning; mentoring and opportunities to meet others; independent living skill development; occupational therapy; resource brokering and advocacy; support for vocational and educational settings; group and individual counseling; and medication support.

Specific to educational goals, EASA uses a "whatever it takes" approach to providing services and supports to young adults. As such, there is no manualized set of services; supports often include, but are not limited to, help with setting educational goals, helping develop organizational skills, learning about campus accommodations and policies specific to psychiatric disabilities, working with financial aid, and registering for classes. Supports specific to organization, time management, and self-care were mentioned frequently by both EASA staff and participants. All supports are tailored to the individual needs of the individual participants, and may change over time as the needs of the participants change. EASA staff will often initially assist participants with an educational activity (e.g., registering for classes), then work with participants to help them complete the task independently.

Stakeholders report that school officials are excited about EASA, as it provides a set of services and supports that are not usually offered in post-secondary education settings. EASA team members interact frequently with the campus Offices of Disability Services and Offices of Counseling. EASA teams learn all the processes of how to access campus-based services and accommodations; and although being involved in EASA does not allow for participants to be fast-tracked for accommodations, EASA staff familiarity with school procedures and rules allows services to be streamlined for EASA participants. In some cases, colleges have granted administrative exceptions for allowing participants to return to campus contingent on their being engaged with EASA.

Participant Identification/Engagement

From July 2013 through June 2014, 433 individuals participated in EASA, with 38% under 18 years of age. Forty-one percent of EASA participants were in school at some point during their engagement with EASA. For participants 18 years of age or older, 57% had 12 years of education, while 29% had less than 12 years and only 14% had more than 12 years. Data from 2008 to 2014 suggest that the majority of EASA participants are White (66%) and male (73%). Thirteen percent identify as Hispanic, and 8% as Black/African American. Approximately 60% of EASA participants are on Medicaid. The majority of EASA participants (over 90%) have strong family support, and many EASA participants are still living at home. EASA participants come from all income levels.

EASA referrals come from a variety of sources, with most originating from psychiatric hospitals (28%), outpatient mental health providers (23%), emergency departments or crisis centers (13%), or family (6%). Approximately 42% of all EASA-referred individuals have been hospitalized within the previous 3 months. Occasionally there are self-referrals or referrals from other students, but individuals experiencing psychosis are less likely to self-refer.

Once a referral is made, an EASA clinical intake screener will collect information on why the referral is being made, and assess if the individual meets EASA criteria. EASA criteria include being between the ages of 15 and 25 (some programs may accept individuals as young as 12), and having or being at risk of a first episode schizophreniform or bipolar spectrum psychosis. If eligible, the intake screener will reach out to the young adult and schedule an in-person meeting. After initial contact with EASA, the first task is to conduct a needs assessment to identify goals. Younger EASA participants are usually in school, while older participants are often interested in returning to school.

EASA staff members do not assume that individuals will be ready to begin active engagement with EASA subsequent to the initial meeting. The intake screener will try to learn a bit about the young adult before the initial meeting, to make the first conversation informed by issues that are of specific interest to that young adult. The screener will also try to arrange to be introduced to the young adult by a person who is trusted by that young adult. Part of the initial assessment will include a safety assessment, a strengths assessment, and an overview of family supports and resources. Much of the early work between EASA and a young adult focuses on facilitating family support and engagement.

EASA uses a proactive engagement strategy, and EASA team members spend a substantial time in the community educating people about early signs and symptoms of psychosis, as well as identifying the risk factors for a first episode. In recent years, EASA has shifted its emphasis from being a "first episode" program to also being an "at risk of first episode" program. Outreach efforts focus on hospitals, community mental health centers, and faith communities. EASA also targets 4-year and 2-year colleges, community colleges, high schools, and the occasional middle school. EASA is engaged with approximately 300 schools across Oregon. In recent years, outreach efforts have extended to include property management companies who are often housing young adult college students), high schools, and, in some targeted communities, middle schools. Students may transfer between EASA sites as they move between communities for school or other reasons.

Staffing

EASA team membership varies across sites. At a minimum, all teams include a lead clinical case manager (a MA level therapist), a psychiatrist or psychiatric nurse practitioner, and a SE specialist. When EASA was first launched, the SE specialist would focus on both supporting employment and educational goals. In an effort to follow evidence-based practice guidelines and IPS fidelity standards (that are tied to funding), SE and SEd tasks were separated.

The majority of teams also have an occupational therapist. For most teams, it is the occupational therapist who leads the education support efforts with participants. The skill set of the occupational therapist is particularly valuable; this professional specializes in assessing barriers to the ability to learn, examining how cognitive information is processed, conducting environmental assessments, and identifying sensory needs. In Oregon, occupational therapists are recognized as qualified mental health professionals and are able to bill third-party payers; this is not the same in other states.

There is currently only one EASA site with a dedicated SEd specialist. Current hiring guidelines suggest that a SEd specialist have a BA degree, but not necessarily a clinical background. SEd specialists may have experience in special education or rehabilitation. The most important characteristics of the SEd specialist are the ability to understand the learning experience of young adults with educational goals and the ability to work as part of a team.

The use of peer support staff is not uniform across EASA sites, or in the delivery of SEd supports. There are some peers engaged in SE services, as there are three community mental health agencies across the state that have some state funding to hire peer support specialists. There is, however, an interest in thinking more about the role of peers in EASA teams, and a desire to operationalize their essential tasks.

EASA strive for a 1:10 staff team/young adult ratio. Team membership in rural counties is often hampered by a limited workforce and shortage of specialized practitioners, particularly occupational therapists.

Financing

When EASA first began in 2001, it was financed through a one-time appropriation of locally managed Oregon Health Plan (Medicaid) dollars set-aside for prevention activities and reinvestment. These funds were awarded by Mid Valley Behavioral Health, an Oregon mental health managed care entity, to fund EASA in five community mental health agencies. From 2002 through 2010, EASA relied in part on federal block grant and private foundation funds to support the clinical services in the original five counties. In 2007, the Oregon legislature appropriated ongoing state General Fund dollars directly from the state legislature to support statewide dissemination of EASA. In 2015, there are approximately $6 million devoted to staffing and delivering EASA, which includes state general funds, Medicaid reimbursement dollars, some private insurance payments, and a small amount of VR funds. For the most part, block grant and general funds are used to fund the array of SEd supports. Medicaid can be used to cover services such as case management and skills training. Private insurance, which is the least used funding source, is used to pay for psychiatry and some individual therapy. EASA is committed to providing an equal level of service regardless of the insurance status of its participants.

Some counties have been able to use state VR dollars to fund SEd services. In Marion, Yamhill, Polk, and Linn counties, EASA teams each have a small caseload of young adults who are receiving educational supports that are funded by the Office of Vocational Rehabilitation. The hope is that this can lead to a statewide dissemination of career-related and educational supports through a matching agreement for funding with VR.

With Medicaid, there is a state statute that mandates a SE billing code. This billing code is specifically tied to use of and fidelity to IPS. Oregon is in the process of creating a modifier for the SE Medicaid billing code, which will allow for SEd activities that are part of SE and IPS to be billed accordingly. Stakeholders were clear to state that while this will not increase the dollar amount available via Medicaid for SEd (as these are allocated at the local level), it will "legitimize" the delivery of SEd services, and "give permission" to team members to do SEd work and implement SEd best practices not articulated in the IPS model. It is also possible that the relative allocation of Medicaid funds for SEd may change (i.e., increase) moving forward. Medicaid in Oregon is distributed through local Coordinated Care Organizations, which use varying payment methodologies that are locally determined.

Block grant and state general funds were generally felt to be reliable funding streams since EASA began in 2001. Connecting SEd to IPS, which can use Medicaid, state general funds, and VR dollars, was seen as a potential avenue to increase access to funding for SEd. This, however, is fraught with challenges as fidelity to IPS is tied to funding, and integrating SEd into IPS creates challenges for meeting IPS fidelity standards.

Early Assessment and Support Alliance Sites

While all EASA sites share a set of core principals and philosophy, sites vary in staffing levels, organizational composition, funding streams, and strategies for supporting educational goals. Below are brief descriptions of various EASA sites that explain these variations.

• Lifeworks NW: Lifeworks NW is a community mental health agency located in Washington County, Oregon. Lifeworks is unique in that it is the only EASA site with a dedicated SEd Specialist. The SEd Specialist works half time with the EASA program and half time with the agency's transition-age youth program. This team has identified SEd as a priority. At Lifeworks NW EASA, there are seven FTE positions shared among 12 staff members, serving 52 clients. EASA funding is valuable not only for its monetary contribution but for the flexibility to pay for services and supplies that cannot be funded in other ways.

• Marion County Children's Behavioral Health: In Marion County, EASA is located within the County Department of Children's Behavioral Health, and is colocated with the agency's transition-age youth program. This is somewhat unique, as the majority of EASA sites are located in adult mental health settings. Marion County EASA has 63 young adults, of which 5 are in high school. Approximately half of the 63 receive some sort of educational support.

• Yamhill County Adult Mental Health: In Yamhill County, EASA is located within the County Department of Adult Mental Health. The Yamhill EASA team uses an occupational therapist to take the lead on educational pursuits. In addition, a peer support specialist is part of the EASA team, and is particularly important for helping young adults meet their educational goals. Neither the occupational therapist nor the peer support specialist are full time with EASA. Yamhill County is a smaller county and its EASA serves approximately 8-9 young adults.

  Yamhill County has a long history of being proactive in developing vocational supports for first-episode young adults, and as such has one of the strongest relationships across EASA sites with the Department of Vocational Rehabilitation. A VR counselor is colocated with Yamhill County EASA to streamline referrals and expedite eligibility into VR supports. Working with EASA has required VR to engage and deliver services more quickly, and to be more flexible with their protocols, including promoting educational pursuits as part of a larger career pathway. VR stakeholders emphasize the importance of colocation with EASA--having a counselor on-site with an EASA caseload--but also acknowledged that it was a culture change, and were not sure if all VR stakeholders had "buy in" with this way of providing supports.

Early Assessment and Support Alliance Innovations

EASA continues to grow and evolve to address issues of changing young adult needs. EASA is involved with two efforts that expand and/or modify supports for EASA-involved young adults that are specifically related to supporting educational goals. These efforts are described below.

• Project Access: Project Access is a collaborative pilot in four counties (Marion, Yamhill, Polk, and Linn) with the state Department of Vocational Rehabilitation that was developed in 2010 to expand career-oriented services beyond EASA's 2-year cap on services. Project Access provides longer-term support of EASA participants 15 years of age and older, and extends eligibility to include EASA participants up to age 30. Project Access was initially funded through stimulus dollars but is now funded by VR. Project Access is designed to provide individualized supports based on an individual's developmental stage. Services can include career exploration, school search, school retention, career-related activities, job search, and job retention. Using a case management approach, individuals work with Project Access staff on general career exploration, identify the types of jobs that are appealing to the participant, and determine what kind of education is needed to help achieve the articulated goals. Staff members support students through school and job searches and school and job retention, in addition to helping to secure resources such as financial aid and transportation. The pilot sites also use peer-based care.

• Youth Hubs: EASA Youth Hubs is a pilot project affiliated with four EASA sites (Lane County, Jackson/Josephine Counties, Multnomah/Washington/Clackamas Counties, and Deschutes/Jefferson/Crook Counties) that expands eligibility beyond first-episode to include a range of significant mental health conditions. Youth Hubs are loosely based on an Australia model called "Head Space," an integrated transition-age youth model that provides preventive and early intervention services for a variety of mental health diagnoses, as well as other age-specific supports. Youth Hubs serve young adults aged 15-24 who would normally be screened out of EASA. This program began in 2014 and is funded by the Oregon state legislature through the state general fund. In addition, some Medicaid dollars are used to cover services such as case management, psychiatry, and counseling. Youth Hubs provide individualized services and supports, including supports specific to educational goals; however, no singular model of SEd is articulated.

Performance Measurement and Outcomes

All EASA sites collect data quarterly on referrals, intakes, and outcome review forms. Sites have recently begun to submit data through state-level Measurement and Outcome Tracking Systems. With Project Access, quarterly data are reported to the state Department of Vocational Rehabilitation, including information on school programs, start-date, full-time or part-time status, end date, and reason for completion with the program.

Within the standalone SEd programs, most continue to collect the data on the outcomes tracked during the original grant period, even though there is no requirement or funding tied to these data. These outcomes include number enrolled in school, number of credited registered for and completed, number of individuals who had contact with a SEd specialist, number of students who graduated, and gender, age, and drug and alcohol use status.

Additional Supported Education Efforts

• SEd Pilot Programs: In 2007, SAMHSA Mental Health Block Grant funding was awarded to three community mental health agencies to start three pilot programs exclusively devoted to SEd. These programs were housed at Cascadia Behavioral Care in Multnomah County, LifeWorks NW in Washington County, and Options of Southern Oregon in Josephine County. The pilot programs ran for 3 years, and were open to any publically funded individual in the mental health system of any age. Block grant dollars were supplemented by a small amount of county funds and by Medicaid, which was used to bill for case management and skills training. Since the block grant ended 3 years ago, Medicaid and county general fund dollars have been used to sustain funding. These programs often run at a deficit.

• Jackson County: A fourth standalone SEd program was introduced in Jackson County in 2014. It employs one SEd specialist, is located in three schools, and currently serves 22 students. This program is funded through SAMHSA federal block grant dollars. The sustainability of these block grant dollars is uncertain. This program uses an adapted version of the University of Kansas SEd fidelity tool (Manthey et al., 2012a) for SEd. The program has received positive feedback from students. It is unclear how it will integrate with EASA, which recently began in Jackson County.

• Oregon Supported Employment Center for Excellence: Created in 2008, the center is part of the larger focus in Oregon on Supported Employment, and provides technical assistance to SE providers, conducts fidelity reviews, collects outcome data, and educates and advises policy makers. The center does address issues of SEd, identifying it as a promising practice, and builds on the SEd principles developed by Karen Unger for SAMHSA.

Integration of Supported Education and Supported Employment

SE is part of the array of services provided through EASA, and in many ways has been the gateway for SEd throughout the state. EASA uses the IPS employment model and its eight principles of IPS as a frame for SEd. IPS is available to individuals of all ages in most counties throughout Oregon. IPS services are funded through the state general fund (where EASA is being delivered), Medicaid, and a small amount of VR dollars. Use of the SE billing code for Medicaid requires meeting IPS fidelity requirements.

There is no single identified strategy for integrating SEd into IPS. When EASA began, SEd supports were delivered by a combined SE and SEd Specialist. The emphasis on IPS fidelity has required these positions to become separate, to allow a targeted focus on SE. The State of Oregon requires participation of EASA sites in fidelity reviews by the Oregon Supported Employment Center for Excellence using the Dartmouth IPS fidelity tool. In order to bill the SE code, sites must pass fidelity. The fidelity tool and process strongly emphasizes job search over education-related activities; if EASA teams spend very much time on education their IPS fidelity scores will generally be lower. This can result in a disincentive to support educational goals. However, most young adults using EASA services have educational goals, and see school as a path toward securing employment and establishing a career. EASA staff members are constantly struggling to balance these competing demands.

Successes and Challenges

Stakeholders identified both challenges and successes in the efforts to address the educational needs of the young adults at risk for or experiencing first-episode schizophrenia-related conditions.

Successes:

• Having state-level champions endorsing the importance of addressing educational goals in this population.

• Creating the expectation within EASA that school and work are immediately supported, with no waiting for services to be delivered.

• Using a transdisciplinary model of service provision, under which multiple EASA team members are addressing multiple domains of young adult functioning and goals, including supporting educational goals.

• Creating consistent and reliable relationships between young adult participants and EASA team members, with a focus on educational goals. EASA team members serve as mentors and coaches, and can normalize the experiences of young adults.

• Educating young adults and family members on what is most helpful to young adults in achieving their educational goals, and providing supports.

Challenges:

• The continual push and pull between SE and SEd: The emphasis on IPS fidelity from the state, which is directly tied to funding, does not always encourage supporting educational goals.

• The focus in Oregon on implementing evidence-based approaches while also needing to meet the needs of young adults, in that what young adults often need may not fit into an evidence-model of care.

• Limited resources: Most EASA teams do not have a dedicated SEd Specialist, and there are challenges in identifying and retaining a specialized workforce, especially in some of the more rural parts of the state.

• Metrics collected by Oregon's Community Care Organizations do not include any focus on education and/or school.

Participants' Stories

Seven young adult EASA users participated in group interview settings--four in one interview and three in another. One had graduated from EASA services (and was currently a part of the EASA Young Adult Advisory Committee), and six had been with EASA anywhere from 6 to 20 months. Participants ranged from 17 to 24 years of age; half were men and half were women. Most EASA participants came to the program through inpatient hospital settings or family referrals. Most were in college when their first episode occurred, resulting in withdrawing from school. All participants had educational goals that were addressed and met through working with EASA.

Almost all participants entered EASA with a significant educational goal. For most, it was the desire to reenroll in college and pursue a degree. Participants described a range of supports provided by EASA that included, but were not limited to, help registering for classes, working with college offices of disability services, connecting to community mental health providers, finding summer employment, completing financial aid forms, researching scholarships, and providing reminders about appointments and schedules. Participants were especially grateful for the engagement with family members, both for providing education about mental illness and first-episode events, and for problem-solving with family members when challenging situations arose, for example, working with financial aid forms. Participants were particularly grateful for the flexibility of EASA staff and their willingness to meet them at times and locations convenient to their school and work schedules.

Participants stressed the importance of their relationships with EASA team members. One participant noted, "The SEd Specialist is like a buddy. He treated me with a lot of caring and kindness." Another noted the value of connecting with EASA team members: "You can really trust them and talk to them not just about educational or employment goals, but also about life and how things are going." Participants highlighted and appreciated the holistic focus of EASA, compared with a more medical model approach in a hospital setting, and the emphasis on communication and creating relationships with EASA team members. EASA participants were concerned, however, about the time limitation of 2 years for using EASA services. Overall, participants were very satisfied with their experiences with EASA. As one participant noted, programs like EASA "make it possible to progress out of psychosis and be independent again."

5.3.3. Learning Enhancement and Resource Network

Overview

LEARN of northern New Jersey provides services for adults with a psychiatric disability residing in four counties. LEARN is situated in a community-based mental health center. LEARN provides services to students across ten community and 4-year colleges and technical schools in the LEARN catchment area. LEARN coaches are trained to develop relationships with higher education staff with whom they interact. Services are provided to adults who wish to pursue higher education. LEARN provides information, resources, and support to help program participants gain access to post-secondary, vocational, and certificate programs. LEARN of northern New Jersey is administered by the Saint Clare's Health System, Behavioral Health Services in Denville, New Jersey.

The goal of LEARN is to create a climate of encouragement and success while assisting students in completing their course of study. LEARN helps with the educational enrollment process, connection to educational resources, and assistance in finding financial aid, grant, and scholarship opportunities. Educational coaches assist with the development of learning skills and provide ongoing assistance and support throughout the educational experience.

LEARN offers SEd as a standalone service that is administered through a community mental health agency. Its model is derived directly from principles of psychiatric rehabilitation, where skills are taught and supports are provided so that individuals obtain valued social roles by meeting their chosen goals in their chosen environments.

Services

LEARN provides a highly detailed and systematic set of services. All students are assessed for their academic readiness, following the trans-theoretical model of behavioral stages of change. LEARN uses a template to categorize whether the student is at low, medium, or high levels of change, and will tailor the services accordingly. For example, coaching for students at low levels of academic readiness will involve providing hope and instilling confidence, while clarifying the requirements of being a student. Moderate-level students will explore student loan forgiveness (if needed) and using a pay-off matrix to clarify goals. Actions for students at a high level of academic readiness may involve linking to on-campus supports and exploring intersession employment opportunities.

Saint Clare's also uses a "Comprehensive Plan of Care" form to clearly state a student-identified problem, related student goals and objectives, the LEARN intervention that should be applied, and target and achieved dates of goal completion. For example, a student may profess educational stress with difficulty meeting deadlines. One goal may be to make big assignments manageable by breaking long-term assignments into shorter steps. LEARN staff may also work with students to strategize about how to minimize distractions.

LEARN uses a variety of developed and tested tools. These include "Wellness in Eight Dimensions" by Peggy Swarbrick; a variety of smartphone applications such as "PTSD Coach" and "Exam Support"; a problem checklist for students that covers issues in 13 dimensions (e.g., self-care, communication); and an "Academic Wellness Plan and Crises Plan" based on Copeland's Wellness Recovery Action Plan.

LEARN's model of service delivery uses the Boston University framework of "Choose, Get, Keep" (Danley & Anthony, 1987). This means services are provided throughout the course of initial career planning, through educational application and enrollment to matriculation, until educational goal completion. Services are designed to assist with all these phases and so may, for example, provide connecting to resources that can assist with defaulted student debt. LEARN also may assist with developing a plan for dealing with prior failing grades, acquiring medical leaves of absence, and ongoing time management and study skills. LEARN stresses concrete skill development to address problems. For example, difficulties with time management are handled by developing a "time budget" with clear demarcation for periods of study, sleep, socializing, and library time. Memory and organizational difficulties are handled with concrete organizational tools, such as the use of planners, calendaring, and task prioritization.

LEARN emphasizes the rehabilitation aspect of SEd. This means that rather than just doing something for the student, (e.g., talking to a professor on the student's behalf), LEARN emphasizes teaching the inherent skills. Students interviewed seconded the assertions that LEARN coaches teach skills so that students can apply learned skills to new settings such as employment.

Importantly, unlike SE, SEd at LEARN is time-unlimited. Services will continue throughout an educational career, even through graduate work. Services continue despite interruptions in college careers. Students can leave and return to LEARN services over the years. Similarly, there is varying level of intensity according to student need. Staff members note that some students need a small amount of guidance or information, while others need ongoing and regular involvement. Services can wax or wane according to student need and preferences. LEARN is also community-based; Coaches travel to meet students in the community and on campuses. Most coach time is spent in the community.

• Coordination of SEd with Related Services: The LEARN team is a standalone support service. Referrals will be made to other services as needed (mental health, substance abuse, etc.), but achieving education outcomes is the sole focus of this service. LEARN coaches will refer and interact closely with other service providers. For example, they work closely with college mental health counselors and will do joint case reviews. However, these individuals are not specifically a part of an interdisciplinary team. Saint Clare's has participated in the National Institute of Mental Illness multisite study using interdisciplinary teams for first-episode psychosis and noted that this kind of closely knit team was very beneficial. However, a concern was noted about integrating SEd with SE. The Team Leader noted that due to SE's longstanding history in the state, when these two services are integrated, SE will always take priority over SEd. She noted the importance of having dedicated time for SEd, because without it, SEd "will take a back seat."

  LEARN staff are encouraged to form collaborative, mutually supporting relationships with college personnel and service directors. LEARN staff noted that targeting counseling and disability services staff for an initial contact to explain services is a successful strategy to achieving buy-in with colleges. Issues to be worked out with schools include access to school computers or log-in, office space (especially for meetings with students), permission to park, and credentials for security checks such as staff IDs. Case reviews with mental health counselors has been one successful strategy to building collaborative relationships around the program in for individual students.

Recruitment/Engagement

LEARN currently serves nearly 80 students. Students are referred from a variety of sources but especially mental health counseling departments of colleges and the mental health centers of the four counties served. LEARN also markets its program at college fairs. LEARN does not report particular difficulties with engagement; staff note that the clear focus of the service on students' identified academic challenges is motivating. Also, some note that the youthfulness of the education coaches, and the ability to relate personally to having academic goals, aid student engagement.

Staffing

The LEARN team is composed of education coaches (BA level); educational specialists/clinician (MA level); and a team leader (MA level with clinical supervisor's license). Due to a large geographical catchment area (four counties), some coaches are assigned specific areas or are assigned to a specific college. Coaches travel and conduct community-based visits with students and school personnel. There is a 25-student caseload size, and visit frequency is determined by need and preference. Some students on the caseload will need infrequent contacts, such as at exam time only. LEARN managers stress staff team building, because this work can easily lead to burnout. The team meets weekly and receives individual clinical supervision from the Team Leader.

Presently new LEARN staff receive 4 days of training from a state contracted trainer from Rutgers University, Integrated Employment Institute, Department of Psychiatric Rehabilitation and Counseling Professions. New staff will be shadowed by experienced staff at Saint Clare's. Staff participate in quarterly "roundtables" group training sessions and ongoing technical assistance. LEARN staff stresses that coaches need detailed knowledge that is specific to the many school settings they encounter, for example, when is the drop/add period over.

As Saint Clare's has had an existing SE program, SEd was easily added to the service array. There are important similarities in the two services, and coaches from one may help the other during busy periods. Hence, staffs and coaches are cross-trained in SE and SEd. The two services together are called career services.

Financing

LEARN of northern New Jersey is a contracted provider of SEd services funded by the state of New Jersey Division of Mental Health and Addiction Services (DMHAS). SEd is funded by the state as a standalone service. Community agencies bid on contracts with the state to deliver SEd. Through four contracted providers, SEd services are available in nearly all counties of New Jersey. The program for this site visit is housed in a hospital-based health care system (Saint Clare's) that delivers an array of outpatient behavioral health services including Assertive Community Treatment Program teams, SE, and partial hospitalization.

Before the LEARN initiative, New Jersey had a state-sponsored SE program. Members of the New Jersey DMHAS, some of whom were alumnae of the Boston University Center for Psychiatric Rehabilitation, designed a SEd service based on the "Choose, Get, Keep" model. An Request for Proposal (RFP) was released by the DMHAS in 2007. Saint Clare's bid and was selected, along with three other agencies. The initial RFP intended that SEd programs would be awarded to and housed within existing New Jersey SE programs. Saint Clare's had an existing SE program that was expanded to include SEd services. A contract was released in 2008 to SE programs originally at $137,000 (each) a year. The contract for SEd services has never yet been recompeted. Each year, Saint Clare's resets its contract for the numbers of students they will serve. Services are billed to the state for every 15 minutes of staff time (rates do not vary by whether the coaches or the team lead provides the service).

Presently, Saint Clare's does not bill private insurance for SEd services as these services are not covered. When the program was initiated the costs were covered 100% by DMHAS. However, state funding has not changed with increased cost of living expenses of providing services. The hospital provides additional funding to offset general and administrative costs. Individuals enrolled in LEARN may be eligible for additional funding through the Division of Vocational Rehabilitation. This funding can be used towards student tuition.

Performance Measurement and Outcomes

LEARN at Saint Clare's has a highly specified quality control and tracking effort for SEd services that is reported quarterly. Among the measures are: numbers of individuals (i.e., served, received, and completed educational readiness services, enrolled in schools, graduated, and linked to employment); numbers of educational outcomes (courses enrolled in, courses completed, diplomas or certificates awarded); service utilization (hours of educational readiness activities, hours of educational coaching, hours of consultation to schools); and client satisfaction. Findings show very high ratings of satisfaction and 200-300 courses satisfactorily completed per year across all participants. Since July 1, 2011, LEARN of northern New Jersey has served 306 clients who have passed 1,218 courses and earned 51 degrees and certificates including AA degrees, BA degrees, and MA degrees.

Successes and Challenges

Stakeholders identified both challenges and successes at the LEARN program in helping participants achieve educational goals across the age spectrum and at different points of college careers.

Successes:

• Developed a comprehensive and highly specified approach to helping participants and students.

• Uses various career tools to help participants discover their strengths and interests and to determine what educational and career paths best suit them.

• Providing time-unlimited services that support students throughout their education; participants received the education they needed to move into the primary labor market, from certificate programs to college degrees, or just a few classes to brush up on necessary skills.

• Forged a close working relationships with several nearby colleges.

• Outcomes demonstrate a successful program using normalized and demanding standards for academic achievement (certificates, credits, and degrees).

Challenges:

• Difficulty in efficiently staffing a community-based service involving multiple college campuses.

• Providing rapid supports to students before challenges turn into crises.

• Finding sources of funding for students to go to school.

• Helping students integrate socially on campus.

Participant Experience

Two groups of LEARN participants were interviewed. The first received behavioral health services from Saint Clare's and included people of both traditional and nontraditional student ages. The second was a group of young people who were students enrolled at Ramapo College. In the interviews, the services received by participants were in accord with how those services were described by LEARN staff. Students noted receiving help with time management, organizational skills, coping skills (e.g., using mindfulness exercises to cope with anxiety); help with acquiring accommodations, dealing with prior educational problems such as defaulted loans, reenrolling after failures, or applying for financial aid. Students reported that LEARN coaches will check in on how students are doing and offer concrete help with understanding assignments or reviewing papers. The students seconded what was reported by staff, that LEARN does not "do it" for the students, but rather that they help the students with issues so that they learn how to handle problems on their own.

Participants were nearly unanimous in their praise for LEARN coaches and for their experiences with LEARN. Students noted that coaches were very patient, kind, and responsive. They described having open and honest relationships with coaches, that coaches were "there for them." They appreciated that coaches would come to campus to meet and pick them up from their homes for an appointment if needed. Students noted differences between what they got from college counseling and what they got from LEARN. As one said, "I felt LEARN really was addressing more of what I needed help with at the moment, and this is different from what I got from the counseling center. It doesn't take the place of counseling."

5.3.4. The University of Minnesota

Overview

The University of Minnesota is the largest post-secondary education system in Minnesota, with over 62,000 students across five campuses and 48,000 on the flagship campuses in the Twin Cities. Over the last 15 years, the University of Minnesota and its leadership have experienced a paradigm shift in thinking about mental health, and have put mental health and wellness at the forefront of the conversation about how to support students in their education as well as their life goals. Although the university does not have an identified SEd program per se, it has instead created a culture of understanding and support around mental illness and mental health that pervades all levels of university organization. As such, the university has many different initiatives that work collaboratively to address the mental health needs of the students, faculty, and staff on campus. These efforts have evolved over many years, and have involved multiple players from across the university. The result is a campus where student mental health and well-being is very much part of the day-to-day conversation about supporting individuals in their academic pursuits, and where the experience of mental health challenges is normalized for students, faculty, and staff alike.

Creating the Blueprint for Addressing the Mental Health Needs of Students

While many campus organizations recognized the challenges that mental health concerns presented to the student body, it was the University Disability Resource Center (DRC) that originally proposed to examine barriers for college students with mental health disabilities. This focus came about because the single largest group served by the DRC was that of students with psychiatric disabilities. In 2001, the DRC leadership applied for and were awarded a Department of Education Fund for the Improvement of Postsecondary Education (FIPSE) grant. The Needs Assessment Project: Exploring Barriers and Opportunities for College Students with Psychiatric Disabilities grant allowed DRC investigators to visit 13 college campuses across the county and conduct focus groups with students, faculty, and campus and community mental health providers to understand the gaps in existing mental health supports and services in campus settings. Additionally, investigators used focus groups to explore and identify potential mental health strategies that could reduce or remove the gaps and barriers identified.

The final FIPSE report included an executive summary (available at https://diversity.umn.edu/disability/educationandtraining) with detailed recommendations to remove the barriers associated with student mental health issues. Key strategies included increasing awareness on campus, decreasing stigma, fostering effective referrals and, most important, clarification, coordination, and communication among key university stakeholders. After the grant ended, a core set of university staff remained committed to supporting student mental health across the campus. This group continued to meet informally, to strategize about how to actualize the FIPSE recommendations. Over time, this group came to the attention of the Office of Student Affairs, which in turn brought the group to the attention of the Provost. A meeting with the Provost was held, where the FIPSE recommendations were reviewed. Stakeholders involved in the meeting with the Provost reflected that the keys to their success in securing the Provost's support were: (1) having data that quantitatively demonstrated the challenges and gaps; (2) providing a set of recommendations for action; and (3) gathering a group of partners interested in collaborating on promoting student mental health. The meeting resulted in the formation of the Provost's Committee on Student Mental Health, as well as some small seed money ($10,000) to support the Committee's initiatives and infrastructure. This seed money comes from a University contract with Coca Cola to sell only Coke products on campus: part of this contract provides the Office of Student Affairs with funds to distribute to student related activities.

Provost's Committee on Student Mental Health

The Provost's Committee on Student Mental Health was established in 2005 with the goal of changing the overall outlook on mental health at the University of Minnesota. Whereas mental health had historically been viewed as a private issue where students were solely responsible for finding help for themselves, the Provost's Committee pushed to address mental health as a campus-wide, public health issue, with the entire community working together to provide support. The four main goals for the Provost's Committee are to: (1) raise awareness about issues related to student mental health; (2) effect policy change; (3) improve conditions on campus for students with mental health conditions; and (4) serve as a model of collaboration for the campus and other universities.

Currently, the Provost's Committee contains 22 members, each of whom represents a distinct part of the University of Minnesota and Twin Cities community. Membership includes individuals from the Athletics Department, the Boynton Mental Health Clinic, the Office of Student Affairs and DRC, the Academy of Distinguished Teachers, the Chief of Police and Public Safety, the Center for Teaching and Learning, the Graduate and Professional Student Assembly, the Office of eLearning, the Parent Program, the Department of Psychiatry, Housing and Residential Life, the Office of Equity and Diversity, the Women's Center, the Student Counseling Center, and student members of Active Minds. Current cochairs of the Provost's Committee represent the DRC and the Boynton Mental Health Clinic.

One of the first and largest projects of the Provost's Committee was to develop a web site dedicated to student mental health. Launched in 2006, the web site provides mental health information and resources related to the University of Minnesota-Twin Cities campus, for students, their parents, faculty, and staff. The site (http://www.mentalhealth.umn.edu) contains information for crisis services, essential numbers to call for information about mental health, events on campus that raise awareness, and details about available mental health and stress management resources.

The Behavioral Consultation Team (BCT) is another initiative from the Provost's Committee that was created in response to the Virginia Institute of Technology shootings. The BCT provides coordinated advice and response to students at risk of harming themselves or others. The BCT is available to students, staff, and faculty for confidential consultation between 8:00 a.m. and 4:30 p.m., Monday through Friday. Once contacted, the BCT will use a team approach to determine the best way to respond to the situation. Minimally, the BCT will keep track of contacts to identify areas or people of concern and to ensure process and professional protocols are used.

Another project of the Provost's Committee is a suicide prevention initiative. A student Provost's Committee member introduced a concern about students and faculty attempting suicide by leaping from University of Minnesota bridges. The Provost's Committee proposed to create and install a series of signs on the bridges that read, "There is Hope" with a 24/7 number to call to speak with a crisis counselor. Part of the work of the committee was to ensure that whenever calls came in from the campus, they would be answered by university-trained crisis counselors, because it will help individuals in crisis to talk with someone who understands the specifics of campus life at the university and the college experience.

Campus-Wide Mental Health Supports

The University of Minnesota has a wide range of departments and organizations dedicated to enhancing and promoting student mental health on campus. The university has three main entities that actively support and provide services for students with psychiatric disabilities: the DRC, the Boynton Mental Health Clinic, and Student Counseling Services (SCS). These three entities work closely together, and triage students among themselves depending on the presenting concerns and needs. Although these are three distinct programs each with a clearly defined mandate, there is a culture of shared responsibility among these programs to: (1) address the individual mental health needs of students: (2) to educate and promote a campus-wide culture of understanding about mental illness and mental health: and (3) and to reduce barriers for students with mental health disabilities.

• Disability Resource Center: The DRC is housed within the Office of Equity and Diversity, and provides accommodations to students with various documented disabilities as mandated by the Rehabilitation Act of 1973 and the American with Disabilities Act (ADA) of 1990 and its subsequent revisions. In fiscal year (FY) 2014, the DRC served 2,125 students and 1,886 faculty, for a total of 4,011 individuals with disabilities and medical conditions. Mental health conditions are consistently the most prevalent of all disabilities seen in the DRC. In FY 2014, students registered with the DRC identified the following primary disabilities: mental health conditions (45%), attention deficit hyperactivity disorder (20%), medical and chronic health conditions (14%), learning disabilities (7%), brain/head injuries (3%), mobility/physical disabilities (3%), autism spectrum disorder (2%), blind/low vision (2%), deaf and hard of hearing (2%), and >1% unknown or with speech disabilities. For students with psychiatric disabilities, anxiety and depression are most prevalent.

  Students are most often referred to the DRC through faculty or advisors. Initially, a student meets with an access consultant to discuss his or her particular concern, any previous experience with receiving accommodations, and to review medical documentation and the student's course load. For students with mental health concerns, some of the most common accommodations include extra testing time, modified attendance requirements, and modified assignment dates. Imperative in the DRC mandate is that accommodations do not compromise the essential elements of the course. The ADA states that students seeking accommodations for classes must be otherwise qualified to take the class.

  A large part of the work of the DRC entails educating faculty and staff about what mental illness may look like, how it can present itself, and what resources and supports exist on campus. DRC staff provides in-person trainings to various departments and schools, and is currently creating an online training module that will be rolled out in the coming year. These DRC trainings are voluntary for University of Minnesota faculty and staff.

• Boynton Mental Health Clinic: The Boynton Mental Health Clinic is housed within the larger Boynton Health Services, which is the primary health care provider on the University of Minnesota campus. The majority of students accessing the clinic self-refer, or are referred through the DRC or the International Student and Scholar Service (ISSS). Although all students using the clinic are assigned to individual therapists, group therapy and medication consultation are also available. Students have a limit of 11 individual sessions a year. While this is sufficient for most, staff will facilitate community referrals as needed if continuing care is indicated.

  Boynton Health Services conducts a bi-annual College Student Health Survey to provide a comprehensive look at the overall health of university students. Data from 2013 suggest that depression (19.3%) and anxiety (18.2%) were the two most frequently reported diagnoses. In addition, a total of 43.3% of students reported having 1-2 stressors within the past 12 months. The Boynton Mental Health Clinic uses these data to provide the most appropriate care to students and to determine how to focus services and supports.

• Student Counseling Services: SCS offers counseling, academic support, trainings, and workshops to help students succeed academically. SCS staff work with students on a wide range of issues including mental health concerns, academic challenges, career uncertainties, and stress management. Referral and communication between the SCS and the Boynton Mental Health Clinic are fairly frequent. In addition, one SCS case manager is colocated at the DRC 1 day per week.

  In 2014, there were approximately 1,600 students that received individual counseling through the SCS. Nearly 80% of students receiving SCS services are classified as having some sort of mental health concern. The most commonly reported issues include depression, anxiety, and stress related to academia and personal life circumstances. There is a 15-session limit for all SCS services; when the session limit is reached, counselors will review the individual student's needs to determine if he or she qualifies for additional supports.

  SCS is launching a new pilot project with Boynton and ISSS called Feel Better Fast, which will offer a semester-long set of online mental health treatment modules addressing depression, stress, and anxiety. Students will have reading assignments and homework and will interact with a counselor providing feedback electronically. This project will be offered to all interested students and will have an integrated a research component to assess usability, satisfaction, and individual mental health outcomes.

Other University Partners in Promoting Student Mental Health

Part of what is unique about the University of Minnesota approach to promoting student mental health is the broad scope of departments and entities across campus invested in this initiative. The DRC, the Boynton Mental Health Clinic, and the SCS are obvious campus partners in addressing student mental health, but there are many other ancillary partners that actively participate in creating a culture to enhance and support student mental health. A few examples are detailed below.

• International Student and Scholar Services: At the University of Minnesota, all international students (approximately 6,200 representing 142 countries) are required to be actively engaged with the ISSS. All international students must meet with ISSS staff regarding various issues specific to study in the United States (e.g., immigration forms, health care). ISSS stakeholders noted that many international students experience stress, depression, and anxiety. In response to the high prevalence of mental health concerns, ISSS leadership intentionally hired staff advisors with counseling backgrounds, a practice that is unique among offices of international affairs on other campuses. ISSS leadership considers mental health an important part of overall student health and well-being. Although advisors do not ask explicitly about mental health, they do look for signs of mental health need. ISSS staff uses a case management/social work approach, and focuses on whole student wellness.

  Learning Abroad Center (LAC): The LAC provides advising and support services to the approximately 3,300 students who study abroad each year. Although the LAC does not track how many of its students have mental health concerns, psychiatric disabilities and needs for accommodation make up the largest portion of its referrals from the DRC, at above 50%. LAC staff brings mental health into the conversation with students before they travel abroad. Among learning abroad programs on other campuses, the LAC is seen as a leader in addressing issues of student mental health, and LAC administrators are often asked to speak at conferences and in other college settings about their experiences. This attention to and awareness of mental health throughout the travel abroad process (both before and during oversees study) demonstrates the university's commitment to integrating mental health and wellness into all aspects of student life.

  Office of Student Affairs: The Office of Student Affairs has been key in setting the tone for embracing and promoting student mental health across the university. Student Affairs was involved with the Provost's Committee from its inception, and provides intermittent small grant funding to fund ongoing activities. The Office of Student Affairs reported that in interactions with other campus offices of student affairs, the most relevant issues to campus life were found to be mental health and sexual assault. As such, the leadership tends to shy away from funding cuts in these areas. As one stakeholder noted, "We want people to be successful and we recognize and address the whole person. We talk about mental health regularly, and help educate all our campus Deans."

Mental Health Promotion Groups and Events

There are countless groups and events throughout the year that promote and educate on student mental health. These include but are not limited to the following.

• Cirque De-Stress: Cirque De-Stress is an annual event on campus where a Boynton Mental Health Clinic psychiatrist is the ringmaster and leads participants through a variety of stress reduction activities, including balancing a peacock feather on your hand or riding a unicycle. This event is very popular and provides a setting where students can enjoy a circus performance, actively participate in relieving their stress, and learn more about mental health and mental health resources on campus.

  Pet Away Worry and Stress (PAWS) Program: The Boynton Health Center sponsors the PAWS program. PAWS is a weekly event that features therapy dogs and rabbits, as well as a therapy chicken and a therapy miniature pony. Students can spend up to 2 hours with the animals as a way to relieve stress and anxiety. This program is very popular, and there are ongoing discussions to expand its presence on campus.

  Active Minds: The university has a robust chapter of Active Minds, the student-run national organization that focuses on raising awareness, promoting support, and eliminating stigma around mental health issues on campus. They host events on campus to educate the community on mental illness and mental health, participate in community service, and collaborate with other on-campus groups.

  Stressing Academic Success: Stressing Academic Success is a forum hosted by the Provost's Committee on Student Mental Health. The forum debuted in 2014 and brought over 200 faculty, staff, student, and university leaders together to have a conversation about stress on campus, the challenges faced by students, and strategies to address these concerns. Three active task forces emerged from the forum to move the work forward. Another forum will be held in October 2015.

Financing

Funding to support mental health accommodations and promotion activities varies across departments. What all departments stressed is that even with the vast prevalence of students with mental health concerns on campus and the recognized importance of providing mental health supports and resources, there have in essence been no new dollars (except the $10,000 in seed money) to support these activities. Departments have had to make choices about how to organize their staff and where to focus their resources within the financial packages they receive. Some noted that the high prevalence of students with mental health concerns may have prevented some departments from receiving funding cuts, but in general there were no new dollars to address mental health on campus.

Specifically, the DRC is centrally funded from the university. The Boynton Mental Health Clinic is funded through student service fees and third-party insurance payments. Funding for the SCS come through student tuition, as well as some funding from the state legislature dedicated to the university; the Office of Student Affairs determines how the legislature dollars are allocated. While public funding for universities in general has decreased over time, the SCS supports are considered critical services and have not received any funding cuts to date.

Successes and Challenges

There were many challenges and successes identified by program directors and stakeholders at the University of Minnesota.

Successes:

• Campus culture regarding mental health is generally positive; there is little stigma associated with mental health issues.

• The Provost's Committee on Student Mental Health has been able to engage the University Provost in decision-making and outreach efforts.

• The large body of resources available on campus for both students and faculty--events, student counseling, mental health services, accommodations in classroom settings, trainings--makes it very easy for anyone to obtain mental health information or support services.

• Trainings have been implemented to help staff and faculty better understand student mental health and mental illness, their role in responding to students in distress, disability accommodations, and resources available on campus.

Challenges:

• Although there are a growing number of students who are in need of mental health services, the university has not been receiving additional funding.

• The large student population of 48,000 on the Twin Cities campuses sometimes makes it difficult to outreach to all students, especially those who are at the graduate or professional school levels.

• The campus has no policy that mandates trainings regarding mental health disabilities; there is variability in interest and follow through among staff and faculty to learn about all available mental health resources and accommodations.

• Boynton Mental Health Center and the SCS have issues with effectively sharing files and documentation on medical records and student information.

Participant Stories

Two University of Minnesota students experiencing depression and anxiety participated in a single group interview. Participants described actively using the three main campus entities that support students with psychiatric disabilities: the DRC, the Boynton Mental Health Clinic, and the SCS. Specific accommodations included securing flexible classroom attendance, extensions on some assignments, a semiprivate classroom for testing, and facilitating extra time for exams. The SCS provided academic and study skills support, while the Boynton Mental Health Center provided therapy. Both students noted that their first step in finding mental health supports on campus was to check the university's student mental health web site.

Students felt that the university's resources met their health care and support needs. Students noted that their professors were very understanding about mental health accommodations. They did reflect, however, that some faculty could be skeptical about the need for accommodation, perhaps because psychiatric disabilities were less visible than other disabilities. Students noted that while there is perhaps less stigma on the University of Minnesota campus regarding mental health than other campuses, there are still many people who do not view mental illness as a disability. The students felt that there is still work to be done to raise awareness and destigmatize the topic.

5.3.5. Successes and Challenges across Sites

Each of the three sites were able to highlight specific successes and challenges in bringing SEd efforts to scale. These are summarized in Table 5-2.

TABLE 5-2. Summary of SEd Successes and Challenges across Sites
	EASA	LEARN	University of Minnesota
Successes	State-level prioritization of EASA efforts	Comprehensive and specified approach to SEd	Creation of Provost's Committee on Student Mental Health
	No waitlist for services	Services are time-unlimited	Large body of mental health resources across campus
	Strong relationships between EASA participants and staff	Strong working relationships between LEARN and area colleges	Campus commitment to supporting student mental health
	Educating participants and family members on supports to achieve educational goals	Standardized data collection on education indicators that suggest positive outcomes	Training for staff and students to understand mental health, disability accommodations, and available resources
Challenges	Balancing "fit" of evidence-based approaches to immediate needs of EASA participants	Providing rapid supports before challenges turn into crises	Increase in demand for mental health services with no additional funding
	Integration of SEd into IPS, resulting in concerns about IPS fidelity and decreased attention to educational goals	Finding sources of funding for students to go to school	Comprehensive outreach to a large student population
	Lack of SEd specialists on most EASA teams	Efficiently staffing multiple campuses with limited SEd personnel resources	No mandates for faculty or staff to receive disability accommodation training
	Limited standardized data collection on education outcomes	Supporting students to integrate socially on campus	Effectively sharing confidential student mental health information across campus departments

5.4. Case Study Cross-Site Integration

5.4.1. Similarities and Differences among the Three Settings

The three settings of the case studies are distinctly different. Nonetheless, we find notable similarities. Elucidation of these similarities provides guidance on how SEd may be construed and operationalized in the future.

Similarities across the Case Studies

• The Importance of Academic Success: All sites shared a commitment to the educational success of students. All three sites indicated the criticality of academic achievement to the development of human and social capital. In terms of human capital, education was viewed as a lynchpin to later vocational success, and employment as the link to higher income that can reduce dependence on disability benefits. In terms of social capital, all three sites espoused that educational attainment is a critical step in human development. When this step is truncated by mental illness, there is a natural desire for completion, thus, for many, higher education was a central goal that should be honored. EASA considers education to be one part of a holistic approach to mental health recovery after illness. At the University of Minnesota, there is campus-wide recognition that poor student mental health will lead to poor outcomes as a student and later in life. As such, University of Minnesota leaders noted that mental health initiatives are "the last place we'll make cuts."

• Initiation and Support "from the top down": At the start, all three sites essentially responded to a call from leadership to provide education supports. For LEARN, it was the state Department of Mental Health that initiated a contract to deliver these services. The EASA program responded similarly to a statewide initiative, and at the University of Minnesota it is the Provost's Committee on Student Mental Health that was instrumental in assuring a campus-wide approach to mental wellness. Although clearly there was interest and involvement from front-line and administrative staff in supporting educational goals, the impetus and ongoing backing of services and practices that meet these goals was defined by leadership. This suggests that future implementation of SEd would need similar upper-level backing. However, it is also notable that a "top-down" approach does not mean a regulatory approach, nor a federal initiative. In some sense, the SEd efforts were "home grown" on a local or state level.

• A Functional Approach to Academic Success: Despite widely different settings, the focus of activities at all three sites was on how to help SEd participants or students successfully function in an academic environment. This focus is resonant with both occupational therapy and psychiatric rehabilitation approaches (i.e., teaching skills needed to meet the demands of an environment) and an intention to teach the skills so that individuals can later use these skills on their own. For students not yet enrolled in school there are important SEd activities pertaining to developing a clear educational goal, and choosing an academic or training program in keeping with that goal. Subsequent skills and activities pertain to obtaining educational enrollment and were highly important, especially for those SEd participants who were returning to college after prior failed attempts. These skills included: applying for a school or training program, completing FAFSA and other financial aid applications, clearing records of past student loan defaults or of college dropouts or failing grades, selecting a course load that is manageable, and registering for classes. Other skills concern strategies that promote student retention. These include organizational skills, time management and calendaring, study skills, note-taking, and use of campus resources. Importantly, all three sites focused on obtaining and using academic accommodations. Accommodations could include extended time for assignments, use of assistive technology in the classroom, adjustments to class attendance policies, preferred seating, isolated areas for test taking, and others. Sites would provide assistance and support in working with the student disability services office to develop an "accommodation letter," and to work with professors so that accommodations are applied. Providers and stakeholders in all three sites would help SEd participants or students manage requests for medical leaves and for returning to school after leaves.

• Ameliorating Mental Health Distress that Impinges on Academic Functioning: In all three settings, academic skill development was complemented by attention to managing a mental health condition while pursing educational goals. Such efforts would be offering strategies for coping with or reducing stress or anxiety, assisting with depression or mental health crises, providing medication management as appropriate, accessing mental health counseling or treatment, teaching strategies for wellness, and assisting with socialization.

• A Normalized, Flexible, and Individualized Approach: Services across the settings were uniformly community-based and integrated, making the three sites consistent with state-of-the-art approaches to mental health service delivery. That is, participants were enrolled in normalized community settings, real colleges and training programs open to anyone. Services were largely delivered in the community. LEARN coaches would meet SEd participants or students at their campuses. EASA staff would accompany SEd participants or students to community settings when needed, and University of Minnesota supports were fully integrated into natural campus settings. Another key feature of the strategies used is that they are tailored to the unique needs of the individual. Skills are taught and strategies are used that are in keeping with individual student needs and according to their choice and preference. This individualization lends itself to the flexible model of service delivery, "doing whatever it takes," espoused by EASA. Even the University of Minnesota, which has the most structured and defined set of strategies, will work with individual students to meet their unique needs as much as possible. Thus, much like SE, there is no one approach, or "one size fits all" model to deliver educational supports; creativity and flexibility of the provider are required.

• Stable Funding and Longevity: Although the funding sources for educational supports differed, we noted that all three sites appeared confident that they would be able to continue to provide service and supports. All three sites had relatively long periods of sustained funding, suggesting that SEd has "staying power." This may be unique to these programs, and a function of and dependent on, the leadership support that first established these programs.

• Participant Satisfaction: Participants at all three sites noted high degrees of satisfaction with the supports received. Participants consistently noted that providers were caring, patient, and knowledgeable. Students were able to identify specific skills they learned and critical activities of the SEd provider that promoted success. In addition, students valued the lack of stigma around mental health, and appreciated a focus on education and not simply one's mental health challenges. Some noted that their academic success was largely due to the help they received. We must acknowledge, however, that the participants in the site visits were selected by the sites themselves.

Differences between the Sites

• Service Structure: We note that the structure of educational supports across the three sites were widely different: One serves a distinct population (first-episode psychosis), where education is one service embedded within an integrated system of care (EASA); one provides a public health approach including mental wellness in a large campus setting (University of Minnesota); and another is a standalone service delivered under the auspices of a community mental health agency (LEARN). Despite differences, we note that all three approaches were successful, suggesting that SEd models can differ and still be very strong.

• The Culture of SEd Services: Each site embodies a unique service culture in which educational supports were delivered. While all three sites emphasized skill building, there were differences in how relationships with the provider were seen. EASA emphasized the role of the relationship between participants and EASA staff as critical to working with participants on skill development and to delivering education-focused services. At LEARN, the relationship was equally valued and important but seemed to develop as a result of skill building activities. There was less importance attached to singular relationships at the University of Minnesota site, where supports and skills were spread over three campus centers and embodied within the broader culture. Similarly, across the sites we observed a continuum of the degree to which the sites adhered to the "medical model." EASA can be considered as operating at one end of the medical model continuum, with an emphasis on recovery from psychiatric illness and recent hospitalizations. LEARN's approach is midway on the continuum--that is, operating out of a community mental health center but with a strong emphasis on community bridging. The University of Minnesota is on the opposite end of the continuum functioning entirely independently of the mental health system, and providing bridges back to the system as needed.

• Models Used: We note that there was no one model for SEd available or used. Hence, each site developed approaches and services that best met the needs of their participants, resulting in differences across sites. Correspondingly, each site offered different opportunities. For example, by going through the mental health "door" (as in EASA), it may be easier to implement SEd especially when there is a SE component. On the other hand, the environmental approach used by the University of Minnesota may identify students who are in need of, but who have not yet accessed, mental health services. It is possible that this public health approach can work to prevent both student mental health crises and student academic failure.

• Performance Measurement and Outcomes: The performance measures and evaluation strategies in the three sites differed. LEARN has the most rigorous outcomes data collection that focused on academic achievement, enrollments, credits earned, and degree completion. At EASA, academic outcomes are not systematically collected for all sites, but some of the pilot programs--specifically those working with VR--collected data on school starts and stops and enrollment status. At the University of Minnesota, there is no program per se to evaluate, and services are spread across various campus organizations, each with their own benchmarks and outcomes.

• Relationship of SEd to SE: The degree to which education supports were integrated with SE varied by site. At EASA, there is very close integration of SEd to SE. Given the nature of the interdisciplinary team used in Oregon, this is not surprising. EASA commented that SEd often took a back seat to SE, particularly as fidelity to the IPS model of SE was tied to financial reimbursement for services. At LEARN however, SEd is a standalone service from SE with separate teams, trainings, and evaluation. Our visit at the University of Minnesota dealt primarily with their mental health campus culture and did not address how the University may be preparing students for employment.

Case Study Key Findings

We summarize the following key findings, taken together:

• Educational goals of individuals with mental health conditions were supported across the lifespan.

• Sites promoted educational success to improve employment prospects, personal development, mental health recovery, and acquiring social and human capital.

• The practice of SEd of individuals with mental health conditions has common core elements pertaining to strategies for choosing, getting, and keeping an educational goal.

• Widely different settings can successfully practice the core elements of providing educational supports.

• The context in which SEd services are deployed will influence who is served and how they are served. Different settings will offer different opportunities and continued experimentation with how to deliver educational supports.

• There was no consistency in evaluation attempts or methods across the sites.

• Leadership endorsement and buy-in is needed to develop, implement, and sustain supports for educational attainment.

• With leadership support, financing SEd programs appears more feasible.

6. Synthesis: Supported Education Needs and Opportunities

6.1. Current Supported Education Program Context

A confluence of contemporary policy and practice make this investigation of the feasibility of a SEd demonstration fortuitous. Both the recent WIOA and the early intervention for SMI set-aside in the SAMHSA Block Grants described earlier provide new opportunities for funding for SEd to key populations. This is in addition to existing funding streams and related policies that can be accessed for this service. These include, for example, special education, VR, Medicaid waivers, and SE. Opportunities for SEd in terms of funding and policy are complemented by the increased need for, and experimentation with, SEd practice. The urgency of the need for SEd programs is also seen among institutions of higher education. These institutions have a burgeoning student population with mental health conditions, and college counseling centers are swaying under the weight. Retention and graduation rates for these students are particularly poor. In terms of practice and model development, SEd program development and evaluation have recently received increased attention, especially for individuals with first-episode psychosis, as can be seen in the ongoing NIMH-funded RAISE study. This confluence sets the stage for a feasible SEd demonstration project and indicates that SEd is on the cusp of widespread and sustained implementation, given the opportunity to be tested as an evidence-based practice.

The previous chapters provide guidance on how to capitalize on this fortuitous occasion for SEd. The following synthesis presents project findings on model development, model development needs, funding, funding needs, evaluation and research, evaluation and research needs, and the feasibility of a future SEd demonstration project.

6.2. Model Development

The literature review, environmental scan, and site visits shed light on principal issues concerning development of a model of service to support the educational goals of individuals living with mental health conditions. This section presents findings shared across these activities. Findings include recognizing that the variability among SEd program models is largely due to differences in service context. Despite differences, a shared set of core components is present across SEd efforts. Findings show that SEd is often integrated and delivered in tandem with SE services, but this integration can be beneficial and disadvantageous. Finally, post-secondary campus settings can offer unique opportunities, distinct from traditional SEd services, to support students with mental health conditions in a college environment.

6.2.1. Program Variability

Great variability exists across programs and services that provide education supports to individuals with mental health conditions. Much of the variability stems from the service setting, which can range from specialty mental health settings (e.g., hospitals, clubhouses, community mental health centers) to primary and post-secondary education settings and to state VR agencies. A specific target population (e.g., veterans, first-episode psychosis, transition-age youth) can also dictate how a program is structured and delivered. Variability can also be attributed to the shifting of available financial and staffing resources and to SEd efforts being modified as needed to address real-time needs of individuals working toward educational goals.

6.2.2. Core Components/Goal Consensus

Even with considerable variability across specific SEd programs and efforts, there appears to be consensus on the critical components of the service. As highlighted in the literature review (Chapter 3), (Waghorn, Still, Chant, & Whiteford, 2004) identified ten core features of SEd programs:

1. Service coordination with professionals outside of the SEd program.

2. Specialized career counseling, including vocational planning and exploration.

3. Specialized, program-trained staff with time allocated explicitly to SEd programs.

4. Financial assistance.

5. Skill building to facilitate integration into the academic environment, including stress and time management and academic or study skills training.

6. On-campus information about student rights and resources.

7. On-campus or off-campus mentoring and support, individual or group support, or peer support.

8. Coordination with post-secondary education institutions to facilitate course access or within-course assistance.

9. Access to tutoring, library assistance, and other forms of supplemental educational support.

10. General support (off-campus support preferred) for the multiple individual barriers and life stressors that can lead to educational attrition.

These same core features, or slight variations thereof, were also noted as key SEd components in the environmental scan and the site visits. Although the specifics may vary depending on program setting (e.g., mental health vs. campus), common components included specialized staff with a dedicated effort to SEd, counseling for careers and educational goals, facilitating financial aid, skill building for educational success, facilitating educational enrollment and retention including acquiring educational accommodations, information about rights and resources, mental health support, coordination with post-secondary education institutions, accessing supplemental educational supports, and providing general supports regarding other noneducation-specific barriers and life stressors. All SEd programs and efforts provided some combination of the aforementioned components.

Some features stood out and were consistently noted and valued, specifically, the presence of dedicated staff, who had supporting educational goals as part of their work and who were committed to helping individuals with mental health conditions meet these goals. This commitment to the work was identified as equal to, if not more important than, a staff person's professional discipline or level of education. Also shared is the understanding that these components need to exist within an environment--be it campus-based or a mental health care setting--that supports mental health and recovery and is dedicated to being free from stigma.

Although no singular standard exists for measuring SEd participant outcomes and tracking success, SEd efforts consistently reported similar goals for participants across program settings. These goals included individuals having an identified educational goal (preferably student led) and individuals enrolling in relevant classes, accruing course credits, and attaining certificates or degrees. Most programs identified an ultimate goal as better employment opportunities, higher income, and lessened dependence on disability benefits, although these distal outcomes could not be measured. These shared SEd components, combined with these shared education outcomes, suggest a common conceptual framework that unifies programs and initiatives that support the educational goals of individuals with serious mental health conditions.

6.2.3. Increasing Supported Education/Individual Placement and Support Integration with Supported Employment

The integration of SEd and SE models--specifically IPS--was frequently seen in SEd programs targeting young adult populations with psychotic or related disorders. This integration is considered particularly appropriate for individuals experiencing first-episode psychosis, because educational experiences are often interrupted by illness onset. Moreover, many individuals with psychiatric conditions need to work and go to school simultaneously, suggesting the need for integrated SE and SEd services. However, as with SEd alone, no singular model integrates SEd and SE. Furthermore, some environmental scan respondents expressed concern that SEd receives fewer resources and attention when combined with IPS, especially when IPS fidelity, rather than education outcomes, is tied to service reimbursement.

6.2.4. Supported Education/Post-Secondary Education Integration Success

Partnerships between SEd programs and post-secondary education settings were key to supporting individuals in their educational goals. SEd programs emphasized the importance of outreach and involvement on college campuses, not only to educate faculty, staff, and students about available SEd resources, but also for SEd staff to fully understand campus services and processes and to create relationships with key campus-based stakeholders (e.g., offices of disability services, counseling, and health). The University of Minnesota provided an exciting example of how to create a campus-wide culture supportive of mental health and wellness, while simultaneously tackling stigma about mental illness from top administrative levels (e.g., Provost's Committee on Student Mental Health) and on-the-ground advocacy efforts (e.g., Active Minds chapters). Campus initiatives are challenged, however, by limited resources dedicated to supporting student mental health.

6.2.5. Supported Education Model Development Needs and Opportunities

This synthesis suggests the following needs for the development of a fully specified, replicable, and testable model for SEd.

• Specifying SEd Core Components: Specification around the core components of SEd should be increased. This could include matching specific components to SEd activities and to measureable outcomes. A first step could include examining existing program-specific SEd manuals and various SEd efforts being implemented across the country to further operationalize components and activities.

• Identifying SEd Staffing Requirements: Staffing requirements should be further elucidated to reflect the range of education, disciplines, and training that contribute to skilled SEd staff. This should include an emphasis on specified skill sets and the capacity to support individuals with educational goals. It should also include developing and routinizing training supports coupled with ongoing coaching and mentoring.

• Defining SEd Specialist Tasks: The role of an education specialist needs to be clearly defined, not only specific to a standalone SEd program, but also when integrated with SE. This includes defining discrete tasks and activities associated with supporting educational goals, while also emphasizing inter-personal and relational skills that facilitate the strong relationships that are the foundation of the work between a SEd participant and a SEd specialist.

• Operationalizing SED/IPS SE Integration: Strategies on how to integrate SEd and IPS SE need to be further defined and operationalized. This should include defining measurable goals and outcomes specific to educational goals and milestones, as well as strategies for staff on how to balance and integrate education and employment goals.

• Operationalizing Campus Best Practices Supporting Student Mental Health: A set of best practice guidelines should be developed to highlight successful strategies for improving campus-based supports for students with mental health conditions. Specifics should include how to secure administrative and leadership buy-in and how to partner with key campus departments (e.g., disability services), as well as more ancillary departments (e.g., travel abroad) to address student mental health. Additionally, strategies to normalize mental illness and decrease stigma on campus should be considered.

6.3. Funding

Funding challenges to support SEd program services was a common theme across the literature review, environmental scan, and site visits. Environmental scan participants, in particular, described multiple funding streams used to support SEd service components without one clear, central funding strategy. In the absence of a core funding strategy, SEd programs relied on different funding vehicles that varied in terms of their stability and ultimate sustainability. Some specific funding issues are described in more detail as follows.

6.3.1. Braided Funding

Ultimately, the most feasible funding model for SEd programs will likely be braiding funding from a variety of sources (municipal, federal, state, collegiate, and/or private corporations). Environmental scan respondents hypothesized about this possibility. For example, public special education services can fund education supports for individuals with psychiatric and other disabilities up to 21 years of age but can abruptly end thereafter. Some environmental scan respondents noted the availability of VR dollars for tuition and books. However, VR is not designed to provide the ongoing and sometimes intensive support needs of people with serious mental health conditions. Complementary funding strategies are needed to fill in such funding gaps. Funding from campus disability services offices (for those enrolled in post-secondary education) or Medicaid may be better suited to complement the limitations of VR or special education services.

6.3.2. Medicaid

The research literature supports that education is intrinsically a part of rehabilitation. Educational attainment is necessary to achieve maximum positive occupational outcomes. Medicaid supports rehabilitation services; consequently, many environmental scan and site visit respondents noted that their programs bill Medicaid for SEd program services whenever possible. Many respondents noted the difference between the availability of an SE Medicaid billing code and the lack of such a code for SEd services. Also, programs with joint SE/SEd services described billing specialists' time under the SE billing code. It is interesting that none of the environmental scan or site visit participants noted the opportunity for states to use the 1915(i) Home and Community-Based Services plan option to fund SEd services. If a state amendment is approved and if individuals meet state-defined need criteria, this plan option could offer an opportunity to fund long-term services.

During the site visit, leadership within the EASA program in Oregon specifically described their efforts to expand the use of Medicaid funding for SEd services in the context of IPS. An Oregon state statute mandates an SE billing code. This billing code is specifically tied to use of and fidelity to IPS. Program leaders indicated that Oregon is in the process of creating a modifier for the SE Medicaid billing code. This modifier would allow for SEd activities that are part of SE and IPS to be billed accordingly. This effort represents one example of how states might consider Medicaid support opportunities for SEd activities.

6.3.3. Vocational Rehabilitation Support

VR state agencies were seen as important partners to SEd efforts; this is partly because of their high federal match rate, their ability to fund tuition and books, and new WIOA legislation. However, VR funding is not intended to provide the longer-term educational services and supports often needed by individuals with psychiatric disabilities to truly succeed in attaining an educational goal. Further, VR can fund education activities only when these activities are explicitly directed at facilitating employment. VR funding for SEd services would have to be supplemented by other funding sources that would fulfill the ongoing needs for skill training and support.

6.3.4. Supported Education Funding Needs and Opportunities

This section describes strategies for identifying opportunities that can sustain funding for SEd programming.

• Braided Funding Case Studies: Those working in the SEd field need to better understand how various programs across the country have and are currently braiding funding to support their SEd program activities. Published case studies that demonstrate successful braided funding strategies in support of SEd services could be widely used to help program administrators circumvent the funding challenges noted in stakeholder discussions across this project.

• Medicaid Billing Code: The availability of an SE Medicaid billing code has helped to disseminate and sustain SE approaches for individuals with psychiatric disabilities across the United States. A similar Medicaid billing option could support and extend the availability of SEd services to complement employment supports. The availability of this type of billing option would directly benefit young adults with mental illness who are highly likely to have both education and employment goals.

• Guidelines for SE/SEd Medicaid Billing: Programs described using the SE Medicaid billing code to support the activities of SE/SEd specialists' time; however, procedures for billing joint SE/SEd program activities vary. SE/SEd program administrators could benefit from guidelines that describe how to bill SEd activities that occur as part of IPS or other SE services.

• Increased Clarity around Medicaid Waiver Option Processes: Program administrators implementing SEd programs could benefit from enhanced clarity around the availability of Medicaid funding to support education services through the 1915(c) or 1915(i) Home and Community-Based Services plan options. This guidance could come in the forms of a state Medicaid director letter, program guide, frequently asked questions document, or fact sheet.

• WIOA Expansion: The recent WIOA expansion offers an opportunity for SEd program implementation and support through VR. The expanded emphasis on WIOA to address career needs of 15-21-year-olds will certainly involve supporting their education goals. VR dollars, with their high federal match for state dollars, can incentivize SEd services for this population. There is also an opportunity to braid the dollars associated with WIOA with Medicaid to provide the rehabilitation services that are concomitantly needed.

6.4. Evaluation and Research

Synthesizing across the environmental scan, site visits, and literature review, concordance was found on the readiness of providers to conduct data collection and on the perceived key outcomes of SEd, thus setting the stage for future evaluation efforts. It also became clear that additional data that will be necessary to establish a platform for considering SEd as an evidence-based practice. These issues are described in more detail here.

6.4.1. Data Collection Readiness

The potential feasibility of a SEd program evaluation was evident in the ongoing data collection occurring across many programs included in this project via the environmental scan and site visits. SEd program sites appeared ready and able to support data collection efforts. In fact, many of the SEd programs included in this project were collecting data, sometimes even outside of the requirements of their particular funding source. These current individual SEd program data collection methods and procedures already in practice could be used as a "springboard" for a broader initiative to study SEd program outcomes.

6.4.2. Agreement on Key Outcomes

Across the literature review, environmental scan, and site visits, there was consensus on what outcomes are important to measure in order to assess SEd program impact. Along with capturing data on service utilization and participant characteristics, key agreed-upon outcomes for SEd programs focused on educational attainment as measured by indicators such as course enrollment data, the number of credits completed, and graduation rates.

6.4.3. "Evidence-Based Practice" Status

SAMHSA houses the NREPP (http://www.nrepp.samhsa.gov/01_landing.aspx). It is one of the leading sources of information on evidence-based practices in substance use and mental health treatment. NREPP offers guidelines that are helpful in considering what might be necessary for SEd programs to be considered an "evidence-based practice." These guidelines involve the following basic minimal requirements: (1) demonstration of positive behavioral outcomes in at least one study using an experimental or quasi-experimental design; (2) availability of implementation and quality assurance materials ready for public use; and (3) published results documented in a peer-reviewed or other professional publication. By these criteria, only a handful of studies examining the impact of SEd intervention could even be considered for an NREPP review nomination based on their study design. Also, upon further examination, those experimental or quasi-experimental studies available either have not demonstrated sufficient positive behavioral outcomes or do not have publicly available implementation or quality assurance materials. Currently, no SEd program has been tested with sufficient rigor or includes sufficient evidence of behavioral change to be nominated for consideration as an evidence-based practice.

Progress towards designation as an evidence-based practice status for SEd is hampered because there have been few trials of SEd with comparison groups; trials with comparison groups have not shown sufficiently compelling outcomes of interest. Program enhancements to the SEd approach have been made since the first SEd RCT conducted by Collins et al. (1998); consequently, it may be the case that these new program enhancements will lead to improved education and employment outcomes. There is suggestive evidence that this will be the case. For the SEd program approach to move from a promising to evidence-based practice, a long-term demonstration project is needed. This project would need to use random assignment, measure degree completion, and include other key program impacts such as employment; health, mental health, or recovery; and community participation. The project should be large enough to examine outcomes across various program setting types and special populations of interest.

6.4.4. Evaluation and Research Needs and Opportunities

Following the example of SE, establishing SEd as an evidence-based practice will in turn promote expansion and implementation of the service. Thus, additional evaluation and research are needed for SEd. Foremost is the need for RCTs with sufficient power to enable adequate analysis of SEd outcomes alone and for long-term follow-up data collection efforts to demonstrate ultimate impacts on employment.

Randomized Controlled Trials

Many promising findings highlight the positive impact of SEd programs on youth and young adults with SMI. However, the current state of evidence is not sufficient to support SEd programs as an evidence-based practice. No RCT with sufficient power to identify differences in SEd outcomes for youth and young adults has been conducted and published. This level of rigor is necessary for any future trial of SEd. Moreover, the ideal SEd trial will be powered sufficiently to identify differences in SEd outcomes (education and ultimate employment) for individuals with educational attainment goals. The importance of this was noted within the literature review, in which findings suggested that currently published studies frequently bundle education and employment outcomes together, prohibiting examination of the singular impact of the intervention on education OR employment. Future research and evaluation studies will need to be organized to better understand possible connections between educational/degree attainment, subsequent employment, wage/income, job stability, and ultimately disability status to fully capture the potential impact of SEd programs.

Long-Term Follow-Up

A long-term commitment to tracking key SEd program outcomes is needed. Longer-term follow-ups are absent in the existing literature; many stakeholders participating in the environmental scan mentioned this problem, and sites visited often reported no funding support to continue long-term data collection. Only a handful of research articles included in the literature review included follow-up data collection beyond an immediate post-program assessment. Only seven of the original research study designs collected follow-up data for 8 months to 3 years beyond preliminary participant program enrollment/baseline. Meanwhile, a typical post-secondary degree/certificate program takes 2-4 years to complete. With the exception of one study with a follow-up data collection period of up to 3 years, none of the follow-up data collection periods in existing SEd trials would have been positioned to gather information on post-secondary degree completion or program certification attainment for program participants.

Taken together, the following specific research and evaluation needs were identified:

• Rigorous Evaluation and Research Designs: SEd programs demonstrate a strong ability to support evaluation studies and data collection efforts; however, existing evaluation efforts are not systematic. Rigorous evaluation and research designs are needed that capitalize on the existing SEd program infrastructure and data collection readiness.

• RCTs: Rigorous research designed to understand the impact of SEd on core outcomes of interest is needed. In particular, a well-designed RCT could help establish the evidence base necessary to move SEd from a "promising" to an "evidence-based" practice.

• Follow-Up Data Collection for 3-5 Years (minimum): Any future SEd research or evaluation trial must be designed with follow-up data collection that extends a minimum of 3 years and ideally 5 or more years from baseline to adequately capture longer-term educational degree attainment and ultimately job sustainability outcomes. Most SEd studies are limited by 1-2-year follow-ups (or less), which is an insufficient amount of time for most individuals to complete a full degree requirement.

• Large Sample Size: Larger sample sizes in SEd outcome studies are needed to analyze differences in outcomes by demographic characteristics and mental illness/symptomology. Larger sample sizes are also needed to allow sufficient power to disentangle the additional benefit of SEd to IPS approaches, separate from their impact on employment outcomes. This would not be feasible in a multisite design.

6.5. Need for and Feasibility of a Future Supported Education Demonstration Project

Results from our literature review, environmental scan, and site visits clearly suggest that a demonstration of SEd is needed. One goal of this study was also to determine the feasibility of such a SEd demonstration project. Project results also indicated that the SEd field would be ready to support a demonstration project. The state of the practice indicates that such a demonstration would need to have two sequential and progressive stages: (1) refine existing fidelity measures, interventions, implementation tools or guides, and develop a demonstration project design; and (2) launch a multisite RCT demonstration project.

6.5.1. Stage 1: Refine Existing Fidelity Measures, Interventions, Implementation Tools or Guides, and Develop a Demonstration Project Design

Because of the existing variability in SEd implementation, some procedures and measures would need to be specified and refined before launching an RCT. Several design details would also need to be resolved. Stage 1 would take approximately 6-12 months to complete.

Test Existing Fidelity Measures

At least two fidelity measures exist for SEd: the University of Kansas Supported Education Fidelity Scale (Manthey et al., 2012a) and the Supported Employment/Supported Education Fidelity Scale for Young Adults with Mental Health Challenges (Frounfelker, Bond, Fraser, Fagan, & Clark, 2014). One measure will need to be selected and/or revised based on its specificity and match with core SEd program goals and practices and the intended RCT intervention. For example, the Frounfelker et al. (2014) scale would be well suited for an SE/SEd demonstration project, whereas the Manthey et al. (2012a) scale would work best within a demonstration project focused exclusively on SEd. In this stage, broader testing of the fidelity scale is needed to establish psychometric properties, validity, sensitivity, and ability to assess changes in program variation over time.

Standardized Supported Education Program Intervention

Given the variability observed in SEd programs, a standard set of activities needs to be established. There are many existing SEd manuals, which can be culled for reproducible procedures and used to standardize the core service delivery components across sites.

Develop Implementation Tools or Guides

Implementation tools and guides are needed, particularly for establishing protocols for training staff, recruiting the target population, and establishing connections with campuses. The SAMHSA SEd promising practice toolkit provides an excellent foundation for this step.

Develop a Demonstration Project Design

Several demonstration project design parameters need to be considered before launching an RCT (Stage 2). These include, but are not limited to, making literature-informed decisions about an expected program effect size (affecting sample size and site capacity recommendations), finalizing a study design (type of control group, decisions about site structure and variability), and giving attention to participant recruitment, retention, and expected attrition over time, as well as within the intervention and control groups. These design development decisions could be easily folded into Stage 1 activities.

• Expected SEd Program Effect Size: Decisions about statistical power and sample size parameters along with recommendations about the number of demonstration sites will be informed by estimates of an expected program effect size. Effect size estimates are necessary for power calculations. Underpowered studies will not have a good chance of finding a statistically significant difference between a treatment and comparison group (even if it exists). An anticipated effect size can be informed by the SEd literature. However, if adequate detail in the literature does not exist (as might be the case with SEd-specific interventions), an effect size may be estimated from expert discussions around the smallest effect size deemed meaningful to test the impact of SEd programs. Once an effect size is determined, decisions about sample sizes and the number of demonstration sites needed to achieve this sample can be decided.

• Type of Control/Comparison Group: A comparison group is critical to the next stage of SEd program research and evaluation. Questions around SEd program impact still remain after decades of smaller-scale, nonexperimental trials. These program impact questions can be definitively addressed only within the context of a well-designed RCT. The most probable comparator for a SEd demonstration project is a "treatment-as-usual" condition (rather than a no-treatment control group). An alternative could be a comparison group with comparable attention from a provider that does not deliver SEd services, or perhaps an "active" control that provides minimal SEd services such as informational fact sheets. The choice of a comparison group will affect the degree to which treatment differences are detected between the SEd intervention group and the comparator. Thus, demonstration project design decisions will ultimately affect recommendations around sample size as well as site quantity, so they need careful consideration.

• Site Structure: Study design consideration will need to be given to a recommendation around the desired state agency structure for administering SEd services within the demonstration project. Study design recommendations will also need to address any required cross-agency partnerships considered necessary for demonstration project SEd service administration. For example, the home for SEd has historically been state mental health agencies and community psychiatric rehabilitation providers; this would be a feasible administering structure for SEd. However, as noted in the report, state agencies of VR can provide important funding for tuition and books and a high federal match for services. Also, each state has a network of local VR offices and counselors that can be accessed for statewide implementation of SEd. A demonstration project design team could consider these issues.

• Site Variability: This project found that SEd programs can be administered in a variety of settings. One important design development decision will be to determine the degree of interest in understanding how SEd program effects vary by setting or type of site (e.g., campus-based vs. specialty mental health based). More variability and heterogeneity across sites will lead to the need for a higher number of sites and larger demonstration project sample size.

• Participant Recruitment, Retention, and Attrition: Environmental scan participants and the published literature note challenges related to recruiting and retaining the participation of individuals with SMI in SEd programs. These challenges have direct implications for a SEd demonstration project. For example, attrition should not be anticipated to occur at random. Those program participants who are at highest risk for poor outcomes are especially likely to drop out of an intervention or control group. Attrition may also be of particularly high concern among individuals within the control condition where participants may have less service contact or engagement over time. High, nonrandom attrition affects the demonstration project's power to detect treatment differences and creates biases in project data. Special attention (e.g., participant incentive plans, tracing procedures, engagement with young adult consumers to review proposed study procedures design and instrumentation) should be paid to these issues at the study design phase to maximize the demonstration project's power and minimize the potential for biased data.

6.5.2. Stage 2: Launch a Multisite Randomized Controlled Trial Demonstration Project

Stage 2 provides the basis for establishing SEd as an evidence-based practice through an RCT. This trial would include a multisite design with all sites required to adhere to SEd consensus goals and core components. The program model selected would ideally not include an integrated IPS/SEd program (because a few trials of various integrated approaches are under way). Rather, the greatest field need is to explicitly test the impact of SEd programs rigorously to understand the unique impact of this type of program component (separate from an emphasis on employment supports).

It is important that the demonstration project involve an experimental design, including random assignment with a control group, to best position SEd for consideration as an evidence-based practice. As in the SE trial (Cook et al., 2005a), the control condition need not be a "no-treatment" control; instead, a treatment-as-usual model would be highly encouraged.

The process evaluation would use the tools constructed in Stage 1 to assess program fidelity and implementation activities. The outcome evaluation structure could include short-term, mid-term, and long-term goals and assessments. The demonstration project must evaluate outcomes beyond the 3-year mark--to not do this runs the risk of SEd programs being deemed ineffective because core outcomes of interest have not been allowed sufficient time to develop. Similarly, the trial must include a sufficient sample to be statistically powered to detect program impacts on either employment or educational goal achievement (outcomes), measured independently.

The outcome evaluation would track key service utilization and participant characteristics. Importantly, the outcome evaluation should also include those key outcomes for SEd programs noted across the literature review, environmental scan, and site visits. These include a particular focus on educational attainment as measured by course enrollment data, the number of credits completed, and graduation rates.

Stage 2 would require an additional 3-5 or more years (depending on follow-up length). Many options would support such a process--Stage 2 could proceed conditional upon the completion of Stage 1, or the two stages could be supported simultaneously. It is not uncommon for early multisite trials to involve a design phase in which program fidelity models and training procedures are refined first, then program enrollment begins 12-18 months after this early design phase is completed. A process similar to this, for example, was followed for the recent RAISE trial (Kane et al., 2015).

Designing a multisite SEd study would be comparable with Cook & colleagues' trial of SE (Cook et al., 2005b). In this trial, each site was permitted some variation in implementation, although prior standards were well set. Despite program variability, "fidelity" was conceptualized across diverse programs, and common outcomes were agreed-upon during the trial process. All programs were required to use the same measures, and data were submitted to a central repository. Given the noted variability in SEd programs, this approach would make the most sense. Results of the environmental scan and case studies, in particular, illustrate how helpful (and necessary) natural variation is. Programs will need to be allowed to vary but be held to uniform standards, goals, and components. Standardization and replication of SEd across communities can be achieved by developing tools in Phase 1 that support the implementation of core SEd program goals and components. This, again, would be very comparable with the trial of SE (Cook et al., 2005b).

6.6. Summary

Findings from the current review of research, policy, and practice indicate that SEd is on the cusp of widespread and sustained implementation. A synthesis of the literature review, site visits, and environmental scan suggests that although settings vary widely, there are also common core practices of SEd. Creative braiding of funding will likely be the solution to the absence of a clear funding stream, and guidance on how to accomplish this will aid provider organizations. Taken together, data suggest that a demonstration trial of SEd is both needed and feasible. Existing research and evaluations of SEd programs lack sufficient rigor, adequate sample sizes, and long-term follow-up assessments to produce the platform necessary to demonstrate SEd program impact. Furthermore, provider organizations are well poised to conduct systematic data collection on SEd processes and outcomes. However, to surpass the limitations of the current SEd research described in this report, a two-stage demonstration program is needed: Stage 1 to prepare fidelity and implementation guides and Stage 2 to conduct a multisite RCT with long-term follow-up. Such a program would provide the platform necessary to generate the potential evidence needed to move SEd from a promising practice to an evidence-based practice, thus encouraging future funding and widespread adoption.

References

Armstrong, K.H., Dedrick, R.F., & Greenbaum, P.E. (2003). Factors associated with community adjustment of young adults with serious emotional disturbance: A longitudinal analysis. Journal of Emotional and Behavioral Disorders, 11(2), 66-76. doi:10.1177/106342660301100201.

Baksheev, G.N., Allott, K., Jackson, H.J., McGorry, P.D., & Killackey, E. (2012). Predictors of vocational recovery among young people with first-episode psychosis: Findings from a randomized controlled trial. Psychiatric Rehabilitation Journal, 35(6), 421-427. doi:10.1037/H0094574.

Becker, D., Whitley, R., Bailey, E.L., & Drake, R.E. (2007). Long-term employment trajectories among participants with severe mental illness in supported employment. Psychiatric Services, 58(7), 922-928. doi:10.1176/ps.2007.58.7.922.

Beguet, V., Fortier, S., & Gauvin, L. (2004). Supported education for people with severe mental health problem. [Meeting Abstract]. Schizophrenia Research, 70(1), 148-148.

Bellamy, C.D., & Mowbray, C.T. (1998). Supported education as an empowerment intervention for people with mental illness. Journal of Community Psychology, 26(5), 401-413. doi:10.1002/(sici)1520-6629(199809)26:5<401::aid-jcop1>3.0.co;2-u.

Best, L.J., Still, M., & Cameron, G. (2008). Supported education: Enabling course completion for people experiencing mental illness. Australian Occupational Therapy Journal, 55(1), 65-68. doi:10.1111/j.1440-1630.2007.00690.x.

Bond, G.R. (1998). Principles of the Individual Placement and Support model: Empirical support. Psychiatric Rehabilitation Journal, 22(1), 11-23.

Bond, G.R., Drake, R.E., & Becker, D.R. (2008). An update on randomized controlled trials of evidence-based supported employment. Psychiatric Rehabilitation Journal, 31(4), 280-290. doi:10.2975/31.4.2008.280.290.

Burke-Miller, J.K., Cook, J.A., Grey, D.D., Razzano, L.A., Blyler, C.R., Leff, H.S., Gold, P.B., Goldberg, R.W., Mueser, K.T., Cook, W.L., Hoppe, S.K., Stewart, M., Blankertz, L., Dudek, K., Taylor, A.L., & Carey, M.A. (2006). Demographic characteristics and employment among people with severe mental illness in a multisite study. Community Mental Health Journal, 42(2), 143-159. doi:10.1007/s10597-005-9017-4.

Bybee, D., Bellamy, C., & Mowbray, C.T. (2000). Analysis of participation in an innovative psychiatric rehabilitation intervention: Supported education. Evaluation and Program Planning, 23(1), 41-52. doi:10.1016/s0149-7189(99)00036-1.

Campbell, K., Bond, G.R., & Drake, R.E. (2011). Who benefits from supported employment: A meta-analytic study. Schizophrenia Bulletin, 37(2), 370-380. doi:10.1093/schbul/sbp066.

Chandler, B.E. (2007). Hidden in plain sight: Working with students with emotional disturbance in the schools. OT Practice, CE1-CE8.

Chandler, D. (2008). Supported education for persons with psychiatric disabilities: California Institute for Mental Health.

Collins, M.E., Bybee, D., & Mowbray, C.T. (1998). Effectiveness of supported education for individuals with psychiatric disabilities: Results from an experimental study. Community Mental Health Journal, 34(6), 595-613.

Collins, M.E., & Mowbray, C.T. (2005). Higher education and psychiatric disaabilities: National survey of campus disability services. American Journal of Orthopsychiatry, 75(2), 304-315. doi:10.1037/0002-9432.75.2.304.

Collins, M.E., Mowbray, C.T., & Bybee, D. (1999a). Establishing individualized goals in a supported education intervention: Program influences on goal-setting and attainment. Research on Social Work Practice, 9(4), 483-507.

Collins, M.E., Mowbray, C.T., & Bybee, D. (1999b). Measuring coping strategies in an educational intervention for individuals with psychiatric disabilities. Health and Social Work, 24(4), 279-290.

Collins, M.E., Mowbray, C.T., & Bybee, D. (2000). Characteristics predicting successful outcomes of participants with severe mental illness in supported education. Psychiatric Services, 51(6), 774-780.

Cook, J.A., Leff, H.S., Blyler, C.R., Gold, P.B., Goldberg, R.W., Mueser, K.T., Toprac, M.G., McFarlane, W.R., Shafer, M.S., Blankertz, L.E., Dudek, K., Razzano, L.A., Grey, D.D., & Burke-Miller, J. (2005a). Results of a multisite randomized trial of supported employment interventions for individuals with severe mental illness. Archives of General Psychiatry, 62, 505-512.

Cook, J.A., Leff, H.S., Blyler, C.R., Gold, P.B., Goldberg, R.W., Mueser, K.T., Toprac, M.G., McFarlane, W.R., Shafer, M.S., Blankertz, L.E., Dudek, K., Razzano, L.A., Grey, D.D., & Burke-Miller, J. (2005b). Results of a multisite randomized trial of supported employment interventions for individuals with severe mental illness. Archives of General Psychiatry, 62(5), 505-512. doi:10.1001/archpsyc.62.5.505.

Cook, J.A., & Solomon, M.L. (1993). The Community Scholar Program: An outcome study of supported education for students with severe mental illness. Psychosocial Rehabilitation Journal, 17, 83-83.

Danley, K., & Anthony, W. (1987). The Choose-Get-Keep model serving psychiatrically disabled people. American Rehabilitation, 13(4), 6-9.

Dougherty, S.J., Campana, K.A., Kontos, R.A., Flores, M.K.D., Lockhart, R.S., & Shaw, D.D. (1996). Supported education: A qualitative study of the student experience. Psychiatric Rehabilitation Journal, 19(3), 59-70.

Downing, D.T. (2006). The impact of early psychosis on learning: Supported education for teens and young adults. OT Practice, 7-10.

Drake, R.E. (1998). A brief history of the Individual Placement and Support model. Psychiatric Rehabilitation Journal, 22(1), 3-7.

Ellison, M.L., Klodnick, V.V., Bond, G.R., Krzos, I.M., Kaiser, S.M., Fagan, M.A., & Davis, M. (2014). Supported employment and supported education for emerging adults with SMHCs. Journal of Behavioral Health Services and Research, 1-16.

Ellison, M.L., Rogers, E.S., & Costa, A. (2013). Tools for system transformation for young adults with psychiatric disabilities. In M. Davis (Ed.), Supporting the Education Goals of Young Adults with Psychiatric Disabilities. Worcester, MA: University of Massachusetts Medical School, Department of Psychiatry, Systems and Psychosocial Advances Research Center, Transitions RTC.

Ellison, M.L., Russinova, Z., Lyass, A., & Rogers, E.S. (2008). Professionals and managers with severe mental illnesses: Findings from a national survey. Journal of Nervous and Mental Disease, 196(3), 179-189. doi:10.1097/NMD.0b013e318166303c.

Ennals, P., Fossey, E.M., Harvey, C.A., & Killackey, E. (2014). Postsecondary education: Kindling opportunities for people with mental illness. Asia Pac Psychiatry, 6(2), 115-119. doi:10.1111/appy.12091.

Evans, L.J., & Bond, G.R. (2008). Expert ratings on the critical ingredients of supported employment for people with severe mental illness. Psychiatric Rehabilitation Journal, 31(4), 318-331. doi:10.2975/31.4.2008.318.331.

Frounfelker, R., Bond, G., Fraser, V., Fagan, M., & Clark, H. (2014). Supported Employment/Supported Education Fidelity Scale for young adults with mental health challenges. Shrewsbury, MA: University of Massachusetts Medical School, Systems and Psychosocial Advances Research Center, Transitions RTC.

Gilbert, R., Heximer, S., Jaxon, D., & Bellamy, C.D. (2004). Redirection Through Education: Meeting the challenges. American Journal of Psychiatric Rehabilitation, 7(3), 329-345.

Gutman, S.A. (2008). Supported education for adults with psychiatric disabilities. Psychiatric Services, 59(3), 326-327. doi:10.1176/appi.ps.59.3.326-a.

Gutman, S.A., Kerner, R., Zombek, I., Dulek, J., & Ramsey, C.A. (2009). Supported education for adults with psychiatric disabilities: Effectiveness of an occupational therapy program. American Journal of Occupational Therapy, 63(3), 245-254.

Hain, R., & Gioia, D. (2004). Supported Education Enhancing Rehabilitation (SEER): A community mental health and community college partnership for access and retention. American Journal of Psychiatric Rehabilitation, 7(3), 315-328.

Hartley, M.T. (2010). Increasing resilience: Strategies for reducing dropout rates for college students with psychiatric disabilities. American Journal of Psychiatric Rehabilitation, 13(4), 295-315.

Hoffmann, F.L., & Mastrianni, X. (1993). The role of supported education in the inpatient treatment of young adults: A two-site comparison. Psychosocial Rehabilitation Journal, 17, 109-109.

Holter, M.C., & Paul, T. (2004). Education as a Signature Program at Breakthrough Club of Sedgwick County. American Journal of Psychiatric Rehabilitation, 7(3), 301-313.

Housel, D., & Hickey, J. (1993). Supported education in a community college for students with psychiatric disabilities: The Houston Community College Model. Psychosocial Rehabilitation Journal, 17(1), 42-50.

Hutchinson, D., Anthony, W., Massaro, J., & Rogers, E.S. (2007). Evaluation of a combined supported computer education and employment training program for persons with psychiatric disabilities. Psychiatric Rehabilitation Journal, 30(3), 189-197.

Jacobs, E., & Glater, S. (1993). Students, staff, and community: A collaborative model of college services for students with psychological disabilities. Psychiatric Rehabilitation Journal, 17(1), 201-209.

Kane, J.M., Schooler, N.R., Marcy, P., Correll, C.U., Brunette, M.F., Mueser, K.T., Rosenheck, R.A., Addington, J., Estroff, S.E., Robinson, J., Penn, D.L., & Robinson, D.G. (2015). The RAISE Early Treatment Program for First-Episode Psychosis: Background, rationale, and study design. Journal of Clinical Psychiatry, 76(3), 240-246.

Kidd, S.A., Kaur-Bajwa, J., & Haji-Khamneh, B. (2012a). Cognitive remediation in a supported education setting. Psychiatric Services, 63(5), 508-509. doi:10.1176/appi.ps.20120p508a.

Kidd, S.A., Kaur-Bajwa, J., McKenzie, K.J., Ganguli, R., & Haji Khamneh, B. (2012b). Cognitive remediation for individuals with psychosis in a supported education setting: A pilot study. Rehabilitation Research and Practice, 2012, 715176. doi:10.1155/2012/715176.

Kidd, S.A., Kaur, J., Virdee, G., George, T.P., McKenzie, K., & Herman, Y. (2014). Cognitive remediation for individuals with psychosis in a supported education setting: A randomized controlled trial.Schizophrenia Research, 157(1-3), 90-98. doi:10.1016/j.schres.2014.05.007.

Killackey, E., Jackson, H.J., & McGorry, P.D. (2008). Vocational intervention in first-episode psychosis: A randomized controlled trial of individual placement and support versus treatment as usual.British Journal of Psychiatry, 193, 114-120.

Kirsh, D.J., Pinder-Amaker, S.L., Morse, C., Ellison, M.L., Doerfler, L.A., & Riba, M.B. (2014). Population-based initiatives in college mental health: Students helping students overcome obstacles. Curr Psychiatry Rep, 16, 525. doi:10.1007/s11920-014-0525-1.

Krupa, T., & Chen, S.-P. (2013). Psychiatric/psychosocial rehabilitation (PSR) in relation to vocational and educational environments: Work and learning. Current Psychiatry Reviews, 9(3), 195-206.

Leonard, E.J., & Bruer, R.A. (2007). Supported education strategies for people with severe mental illness: A review of evidence based practice. International Journal of Psychosocial Rehabilitation, 11(1), 97-109.

Lieberman, H.J., Goldberg, F.R., & Jed, J. (1993). Helping seriously mentally ill patients to become students. Psychosocial Rehabilitation Journal, 17(1), 99.

Manthey, T., Coffman, M., Goscha, R., Bond, G., Mabry, A., Carlson, L., Davis, J., & Rapp, C. (2012a). The University of Kansas Supported Education Toolkit 3.0. The Office of Mental Health Research and Training: The University of Kansas School of Social Welfare.

Manthey, T.J., Goscha, R., & Rapp, C. (2014). Barriers to supported education implementation: Implications for administrators and policy makers. Administration and Policy in Mental Health, 2014 Aug 22. [Epub ahead of print]. doi:10.1007/s10488-014-0583-z.

Manthey, T.J., Holter, M.C., Rapp, C.A., Davis, J.K., & Carlson, L. (2012b). The Perceived Importance of Integrated Supported Education and Employment Services. Journal of Rehabilitation, 78(1), 16-24.

Marder, C. (1992). How well are youth with disabilities really doing? A comparison of youth with disabilities and youth in general. A Report from the National Longitudinal Transition Study of Special Education Students.

McFarlane, W.R., Cook, W., Downing, D., Ruff, A., Lynch, S., Adelsheim, S., Calkins, R., Carter, C.S., Cornblatt, B., & Milner, K. (2012). Early detection, intervention, and prevention of psychosis program: Rationale, design, and sample description. Adolescent Psychiatry, 2(2), 112-124.

McFarlane, W.R., Levin, B., Travis, L., Lucas, F.L., Lynch, S., Verdi, M., Williams, D., Adelsheim, S., Calkins, R., Carter, C.S., Cornblatt, B., Taylor, S.F., Auther, A.M., McFarland, B., Melton, R., Migliorati, M., Niendam, T., Ragland, J.D., Sale, T., Salvador, M., & Spring, E. (2014). Clinical and functional outcomes after 2 years in the early detection and intervention for the prevention of psychosis multisite effectiveness trial. Schizophrenia Bulletin, 41(1), 30-43. doi:10.1093/schbul/sbu108.

McIntyre, A. (2008). Participatory action research. Thousand Oaks: CA.: Sage Publications.

Morrison, I., Clift, S.M., & Stosz, L.M. (2010). Supported further education provision for people with long-term mental health needs: Findings from a survey of further education colleges and primary care trusts across the south east of England. Perspect Public Health, 130(2), 78-85.

Mowbray, C. (2000). The Michigan Supported Education Program. Psychiatric Services, 51(11), 1355-1357.

Mowbray, C.T., Bybee, D., & Collins, M.E. (2001). Follow-up client satisfaction in a supported education program. Psychiatric Rehabilitation Journal, 24(3), 237-247.

Mowbray, C.T., Bybee, D., & Shriner, W. (1996). Characteristics of participants in a supported education program for adults with psychiatric disabilities. Psychiatric Services, 47(12), 1371-1377.

Mowbray, C.T., & Collins, M. (2002). The effectiveness of supported education: Current research findings. In C.T. Mowbray, K.S. Brown, K. Furlong-Norman & A.S. Soydan (Eds.), Supported Education and Psychiatric Rehabilitation: Models and Methods. Linthicum, MD: International Association of Psychosocial Rehabilitation Services.

Mowbray, C.T., Collins, M., & Bybee, D. (1999). Supported education for individuals with psychiatric disabilities: Long-term outcomes from an experimental study. Social Work Research, 23(2), 89-100.

Mowbray, C.T., Gutierrez, L.M., Bellamy, C.D., Szilvagyi, S., & Strauss, S. (2003a). Replication of a psychosocial rehabilitation program: A case study analysis of supported education. Journal of Community Psychology, 31(5), 437-457. doi:10.1002/jcop.10060.

Mowbray, C.T., Megivern, D., & Holter, M.C. (2003b). Supported education programming for adults with psychiatric disabilities: Results from a national survey. Psychiatric Rehabilitation Journal, 27(2), 159-167.

Mueser, K.T., Aalto, S., Becker, D.R., Ogden, J.S., Wolfe, R.S., Schiavo, D., Wallace, C.J., & Xie, H. (2005). The effectiveness of skills training for improving outcomes in supported employment. Psychiatric Services, 56(10), 1254-1260. doi:10.1176/appi.ps.56.10.1254.

Mueser, K.T., & Cook, J.A. (2012). Supported employment, supported education, and career development. Psychiatric Rehabilitation Journal, 35(6), 417-420. doi:10.1037/h0094573.

Newman, L., Wagner, M., Knokey, A.M., Marder, C., Nagle, K., Shaver, D., & Wei, X. (2011). The post-high school outcomes of young adults with disabilities up to 8 years after high school: A report from the National Longitudinal Transition Study-2 (NLTS2). Menlo Park, CA: SRI International.

Nuechterlein, K., Subotnik, K., Turner, L.R., Ventura, J., Becker, D.R., & Drake, R.E. (2008a). Individual placement and support for individuals with recent-onset schizophrenia: Integrating supported education and supported employment. Psychiatric Rehabilitation Journal, 31(4), 340-349. doi:10.2975/31.4.2008.340.349.

Nuechterlein, K., Subotnik, K., Ventura, J., Gitlin, M., Gretchen-Doorly, D., Green, M., Becker, D., Drake, R., Mintz, J., Wallace, C., & Liberman, R. (2008b). A randomized controlled trial of supported employment and education and workplace skills training in recent-onsent schizophrenia: Notable improvements in work recovery. Schizphrenia Research, 102(1-3), 279.

Parrish, C. (2009). MH supported education literature review: Department of Behavioral Health and Mental Retardation Services Research and Information Management.

Patton, M. (2015). Qualitative research and evaluation methods. Thousand Oaks, CA: Sage Publications.

Rinaldi, M., Perkins, R., McNeil, K., Hickman, N., & Singh, S.P. (2010). The Individual Placement and Support approach to vocational rehabilitation for young people with first episode psychosis in the UK. Journal of Mental Health, 19(6), 483-491. doi:10.3109/09638230903531100.

Robson, E., Waghorn, G., Sherring, J., & Morris, A. (2010). Preliminary outcomes from an individualised supported education programme delivered by a community mental health service. British Journal of Occupational Therapy, 73(10), 481-486. doi:10.4276/030802210x12865330218384.

Rogers, E.S., Anthong, W.A., Lyass, A., & Penk, W.E. (2006). A randomized clinical trial of vocational rehabilitation for people with psychiatric disabilities. Rehabilitation Counseling Bulletin, 49(3), 143-156. doi:10.1177/00343552060490030201.

Rogers, E.S., Chamberlin, J., Ellison, M.L., & Crean, T. (1997). A consumer-constructed scale to measure empowerment among users of mental health services. Psychiatric Services, 48(8), 1042-1047.

Rogers, E.S., Kash-Macdonald, M.B.D., & Maru, M. (2010). Systematic review of supported education literature, 1989-2009. Boston, MA: Boston University, Sargent College, Center for Psychiatric Rehabilitation.

Salyers, M.P., Becker, D.R., Drake, R.E., Torrey, W.C., & Wyzik, P.F. (2004). A ten-year follow-up of a supported employment program. Psychiatric Services, 55(3), 302-308.

Salzer, M.S., Wick, L.C., & Rogers, J.A. (2008). Familiarity with and use of accommodations and supports among postsecondary students with mental illnesses. Psychiatric Services, 59(4), 370-375. doi:10.1176/appi.ps.59.4.370.

Schindler, V.P., & Sauerwald, C. (2013). Outcomes of a 4-year program with higher education and employment goals for individuals diagnosed with mental illness. Work, 46(3), 325-336. doi:10.3233/wor-121548.

Sharpe, M.N., & Bruininks, B.D. (2003). Services for students with psychiatric disabilities in the big ten schools. University of Minnesota.

Sharpe, M.N., Bruininks, B.D., Blacklock, B.A., Benson, B., & Johnson, D.M. (2004). The emergence of psychiatric disabilities in postsecondary education examining current challenges in secondary education and transition (vol. 3). Minneapolis, MN: National Capacity Building Institute on Issues of Transition and Postsecondary Participation for Individuals with Hidden Disabilities.

Shor, R., & Avihod, G. (2011). The conceptual model and guiding principles of a supported-education program for Orthodox Jewish persons with severe mental illness. Community Mental Health Journal, 47(5), 568-572. doi:10.1007/s10597-011-9387-8.

Smith-Osborne, A. (2012a). Supported education for returning veterans with PTSD and other mental disorders. Journal of Rehabilitation, 78(2), 4-12.

Smith-Osborne, A. (2012b). Supporting resilience in the academic setting for student soliders and veterans as an aspect of community reintegration: The design of the Student Veteran Project Study. Advances in Social Work, 13(1), 34-50.

Souma, A., Rickerson, N., & Burgstahler, S. (2006). Academic accommodations for students with psychiatric disabilities: DO-IT. University of Washington.

Substance Abuse and Mental Health Services Administration. (2011). Supported education: Evaluating your program. Rockville, MD: Center for Mental Health Services, Substance Abuse and Mental Health Services Administration, U.S. Department of Health and Human Services.

Thompson, C.J. (2013). Supported education as a mental health intervention. Journal of Rural Mental Health, 37(1), 25-36.

U.S. Department of Education. (2004). Twenty-sixth annual report to Congress on the implementation of the Individuals with Disabilities Education Act. Washington, DC: U.S. Department of Education.

U.S. Department of Labor. (2010, 2014). Employment projections: Education pays in higher earnings and lower unemployment rates. Retrieved from http://www.bls.gov/emp/ep_chart_001.htm.

Unger, K.V. (1990). Supported postsecondary education for people with mental illness. American Rehabilitation, 16, 10-14.

Unger, K.V. (1993). Creating supported education programs utilizing existing community resources. Psychosocial Rehabilitation Journal, 17(1), 11-23.

Unger, K.V. (1998). Handbook on supported education: Providing services for students with psychiatric disabilities. Baltimore, MD: PH Brookes Publishing Co.

Unger, K.V. (2011). The evidence. Supported education: A promising practice. Evidence-Based Practices KIT (Knowledge Informing Transformation): Substance Abuse and Mental Health Services Administration.

Unger, K.V., Anthony, W.A., Sciarappa, K., & Rogers, E.S. (1991). A supported education program for young adults with long-term mental illness. Psychiatric Services, 42(8), 838-842.

Unger, K.V., & Pardee, R. (2002). Outcome measures across program sites for postsecondary supported education programs. Psychiatric Rehabilitation Journal, 25(3), 299-303.

Unger, K.V., Pardee, R., & Shafer, M.S. (2000). Outcomes of postsecondary supported education programs for people with psychiatric disabilities. Journal of Vocational Rehabilitation, 14(3), 195-199.

Unger, K.V., Pfaltzgraf, B., & Nikkel, R.E. (2010). A supported education program in a state psychiatric hospital. Psychiatric Services, 61(6), 632. doi:10.1176/appi.ps.61.6.632.

Waghorn, G., Still, M., Chant, D., & Whiteford, H. (2004). Specialised supported education for Australians with psychotic disorders. Australian Journal of Social Issues, 39(4), 443-458.

Wagner, M. (2005). Youth with disabilities leaving secondary school. Journal for Vocational Special Needs Education, 27(2), 6.

Wagner, M., & Newman, L. (2012). Longitudinal transition outcomes of youth with emotional disturbances. Psychiatric Rehabilitation Journal, 35(3), 199-208.

Walsh, D., Sharac, J., Danley, K., & Unger, K. (1991). The campus support project: An innovative supported education program model. Innovations & Research, 1(1), 15-21.

Watkins, D.C., Hunt, J.B., & Eisenberg, D. (2012). Increased demand for mental health services on college campuses: Perspectives from administrators. Qualitative Social Work, 11(3), 319-337. doi:10.1177/1473325011401468.

Yahaya, A., Ramli, J., Yahaya, N., & Yen, G.S. (2010). Correlation between self-esteem, coping difficulties, self-efficacy, and illness symptoms towards supported education for students with psychiatric disabilities. In Z.M. Jelas, A. Salleh & N. Azman (Eds.), International Conference on Learner Diversity 2010 (vol. 7, pp. 642-651).

Appendix A. Site Visit Discussion Prompts

Program Administrators and Staff

1. Introduction to Program
2. Overview (population)
3. Context
4. History
5. Implementation
6. Roles

Services Offered

What services/supports are included in your program, and how do they interact?

Probe for:

1. Coordination of SEd with:

   1. mental health services
   2. SE services
   3. student disability offices

2. Specific services provided:

   1. Education program planning (setting goals, choosing program)
   2. Financial aid, benefits
   3. Campus resources and rights
   4. Acquiring accommodations
   5. In-course assistance
   6. Organizational skills
   7. Other

3. Which of these services are most critical?

4. Are there gaps in the services you provide?

Participant Recruitment/Engagement

1. Demographic characteristics of participants (documentation?)
2. Recruitment and engagement strategies

Staffing

How is your program staffed and managed?

Finances

How is your program financed?

Probe for:

1. What is the core funding? (Has this changed over time?)
2. Funding partners and sources including VR, Medicaid and Medicaid waivers, state block grants, state mental health, federal or other grants
3. Use of braided funding
4. Funding sustainability
5. Financing challenges
6. Impact of local and federal financing and policies (e.g., Workforce Investment Innovation and Opportunities Act and VR for transition-age youth; 1915c waiver)

Evaluation

What are current evaluation efforts? (Report availability)

Probe for:

1. Method of evaluation? (Data collected, data sources - primary, administrative?)
2. What outcomes are most interested in?
3. What numbers do you include in proposals and/or wish you had?
4. Frequency, timeframe (long-term follow-up?)
5. How is evaluation used?
6. Do you assess program fidelity? Describe

What do you think is needed in future evaluation efforts to move the field of SEd services forward?

Service Context

How does your service organizational context influence how you deliver services? (e.g., you are in a large MH agency in multiple counties; how does this impact your service of a targeted population)

Probe for:

1. Impact of contextual policies
2. Impact of population served
3. Who are your community partners (are partners missing?)

Challenges and Successes

What do you think is most important in serving this population?

What are the key successes of your SEd program?

What are key challenges?

Probe for:

1. Finances
2. Evaluation
3. Services offered

SEd Participants

Referral

1. How were you referred, or how did you get started here?
2. What made you want to start services here?
3. What were you hoping to get help with when you came here?

Services Offered

1. What services were you provided?
2. How long?
3. Did these services meet your needs?
4. Did you get help with other things?
5. What made you want to stay involved in this program?

Satisfaction

1. What did this program help you with?
2. How did it help you?
3. What were the best parts/worst parts of service?
4. Would you recommend this program to a friend?
5. What do you think the program should do to be better?

Project Information

This report was prepared under contract #HHSP23320095651WC between the U.S. Department of Health and Human Services (HHS), Office of Disability, Aging and Long-Term Care Policy (DALTCP) and the Research Triangle Institute. For additional information about this subject, you can visit the DALTCP home page at http://aspe.hhs.gov/office-disability-aging-and-long-term-care-policy-daltcp or contact the ASPE Project Officer, Emily Jones, at HHS/ASPE/DALTCP, Room 424E, H.H. Humphrey Building, 200 Independence Avenue, S.W., Washington, D.C. 20201.  Her e-mail addresses is: Emily.Jones@hhs.gov.