Improving Employment Outcomes for People with Psychiatric Disorders and Other Disabilities. APPENDIX B: Methodology for the Literature Searches


In this appendix, we detail the methodology we employed to conduct two targeted literature searches: one on employment programs and outcomes for people with psychiatric disorders and one on employment programs and outcomes for people with other disabilities. The literature search and review process for both searches consisted of five stages: (1) systematically retrieving peer-reviewed literature; (2) sorting references; (3) identifying gray literature and other studies; (4) identifying studies for inclusion in the review; and (5) summarizing relevant articles.

A. Systematically Retrieving Peer-Reviewed Literature

We first developed a systematic information-retrieval plan. In consultation with a library information specialist, the chapter authors identified the suite of relevant databases and indexes to search: PubMed, PsychInfo, Cochrane Database of Systematic Reviews, Scopus, and CINAHL. Searching several databases minimizes the risk of selection bias from any one database. We narrowed the field to articles published in English in the last 20 years (1992-2012).

We then selected appropriate key word terminology for each subgroup and SE (Table A.1). We tailored the search terms to each database to allow for differences in terminology, search syntax, spellings, use of truncation, and organization and indexing of content. For the review of evidence on employment for people with psychiatric disorders, the search for each subgroup included general terms for: (1) disability type, including psychotic disorders, mental disorders, severe mental illness, and schizophrenia; and (2) employment terms, such as employment outcomes, employment supports, and work supports. To these key words we added language specific to each subgroup and retrieved the following:

  • Long-term users of mental health services. Additional key terms included specific target populations (veterans and homeless, for example) and disorders, (PTSD, for example). We retrieved 192 unduplicated articles.

  • Individuals at risk of job loss. Additional key words included likely employment-support programs (employer disability insurance, employee assistance programs, VR, and reasonable accommodation, for example). We retrieved more than 600 unduplicated articles.

  • Individuals experiencing an FE of psychosis and TAY. Due to the overlap in terminology, we combined the database searches for these two subgroups. Language to isolate the populations included FE, young adults, onset of psychosis, and youth in transition. We retrieved 64 unduplicated articles.

  • SE. Given the specificity of this topic and the extensive literature base, we restricted the search terms to SE, systematic reviews, and reviews. We retrieved 112 unduplicated articles.

For the review of evidence on employment supports for people with other disabilities, we searched for and summarized existing systematic reviews and meta-analyses concerning services and supports designed to increase employment among people with disabilities. We excluded employment interventions for people with mental disorders, as our previous literature review covered employment-support programs for this population. In addition to the databases mentioned above, we searched Campbell Collaboration, AHRQ, Academic Search Premier, SocINDEX, EconLit, the National Center for the Dissemination of Disability Research Registry of Systematic Reviews, and Business Source Corporate.29 Keywords and phrases developed for the searches were assigned to four groups and truncated if necessary (Table A.1). The searches yielded 221 potentially relevant studies.

B. Sorting Peer-Reviewed Literature

We stored reference information for articles retrieved through this process in RefWorks, an online reference-management application. In RefWorks, we reviewed article titles, authors, and abstracts to sort through the unduplicated studies by subgroup. Studies were excluded that: (1) focused on a disability group other than psychiatric disorder, or focused on the disability population as a whole; (2) did not focus on employment; (3) did not report on study results (such as opinion pieces). Remaining articles were sorted by study design: systematic review, non-systematic review, RCT, quasi-experimental design (QED), pre-post design, implementation study, and "other."

For the review of evidence on employment programs for people with other disabilities, two reviewers independently examined systematic review titles and abstracts related to disability, RTW, DM, and employment outcomes. We excluded summaries of disability laws and policies, such as the ADA and Medicaid Buy-In, which do not include employment or RTW services at the individual or employer levels. We also excluded tax incentives for businesses to hire persons with disabilities. We also obtained relevant reviews and reports through Mathematica's subscription service and assessed them for inclusion. For each review or report included, we extracted pertinent information about the study types, design, key findings, and strength of the evidence reported (Table A.8). Those retained for screening met the following four criteria, as determined through independent examination of their titles and abstracts by two reviewers:

  1. They did not focus on populations consisting solely of individuals with mental disorders.

  2. They could be classified as systematic reviews, reviews, or meta-analyses.

  3. They did not focus on medical/clinical interventions, unless these interventions were combined with employment services or supports.

  4. They were published between January 1992 and October 2012.

C. Identifying Gray Literature and Additional Peer-Reviewed Studies

In addition to conducting an up-front, systematic search for peer-reviewed literature, we searched for gray literature and other suitable studies on a continuous basis by: (1) obtaining articles and reports of demonstration and research projects known to the authors, such as reports based on EIDP and TTW; (2) scanning the references of published articles identified above; and (3) searching websites of relevant government agencies, such as SAMHSA, the National Institute on Disability and Rehabilitation Research, and HHS, and research centers, such as the Rehabilitation Research and Training Centers on youth transition. In some instances, we contacted study authors to obtain unpublished articles or data, or published articles we could not locate.

Our initial scan did not identify published systematic reviews on workplace interventions for veterans or reviews that included national demonstrations for Social Security disability beneficiaries. To address this issue, we also included four reports that describe research on Social Security RTW demonstrations and Veterans Health Administration programs.

D. Identify Studies for Inclusion in the Review

We used a tiered-inclusion process to prioritize the most rigorous research available for each study group. We first selected systematic literature reviews to focus on the most rigorous research and take advantage of work already completed, then included RCTs, QEDs, and pre-post test designs. Our inclusion criteria for each subgroup varied according to the rigor of the literature base. For example, given the robust literature on SE, we primarily included systematic reviews, whereas for the newer area of FE research, we included a mix of RCTs, QEDs, and pre-post test designs. The number of studies included in the review is shown in the right column of Table A.1.

The review on evidence of employment programs for people with other disabilities included only systematic reviews. The review team double-checked each other's decisions about whether each study met the criteria listed above. If both agreed that the criteria were fulfilled, a librarian obtained the study's full text. Full-text articles underwent further assessment for inclusion, based on the germaneness of their subject matter. Reviews were included if they centered primarily on employment supports and services for people with physical, sensory, developmental, or intellectual disabilities, and if the outcomes of interest included employment. We included 15 studies in the review.

E. Summarizing Relevant Articles

After identifying articles for inclusion, we summarized each one in separate tables for each study group in order to consistently track key information. Each table included study identifiers, study description, research methodology, study population, sample size, results, and limitations. Simplified versions of these tables appear in Appendix A.

TABLE B.1. Results of Literature Review
Area of
Search Terms Number of
Number of
SE Supported employment; systematic reviews; reviews 112 22
Long-term users of mental health services Serious mental illness; schizophrenia; bipolar disorder; psychiatric disability; post-traumatic stress disorder; depression; employment outcomes; vocational outcomes; job outcomes; work supports; employment supports; job supports; disability supports; vocational services; employment services; disability services; Social Security Disability Insurance; Supplemental Security Income; veterans; military; Temporary Assistance for Needy Families (TANF); homeless; criminal justice; immigrants; economic recession 192 6,516
Risk of job loss Employer disability insurance; employee assistance programs; mental health parity; reasonable accommodations; job retention; job tenure; job support; depression; workplace; risk of job loss; mental health friendly workplace; return to work; employment outcomes; supported employment; vocational rehabilitation; mental disorders >600 13
FE/TAY First episode; onset of psychosis; onset of schizophrenia; adolescents; young adults; youth in transition; psychotic disorders; mental disorders; serious emotional disturbances; supported employment; job support; employment outcomes; employment supports; work supports; individual placement; vocational rehabilitation; wrap-around services 64 9 in FE section
5 in TAY section
Other employment supports Articles were referred from research results on the above populations 13 13
Employment programs for people with other disabilities Employment services/supports; disability management; vocational rehabilitation; Veterans Administration; impairment; blind; deaf; physical disability; intellectual disability; Supplemental Security Income; Social Security Disability Insurance; Medicaid; private disability insurance; systematic review; meta-analysis; review 221 15
NOTE: Studies on which evidence of the effectiveness of specific interventions is based. Excludes articles used for background material. This number includes articles identified through other sources, such as reference lists of literature reviews or reports.

View full report


"EmpOut.pdf" (pdf, 860.01Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®