Survey Design for TANF Caseload Project: Summary Report and Recommendations. Common Measures Used, Pros and Cons


General Screening Instruments for Global Mental Health

The NSAF administered a brief, general measure of parental mental health intended to cover four broad dimensions. Five questions ask about anxiety, depression, psychological well-being, and loss of behavioral or emotional control. These questions were adapted from the five-item Mental Health Inventory scale (MHI-5), used in the Medical Outcomes Study (MOS). The measure was developed by selecting the five items that best predicted summary scores on the longer 38-item version of the MHI scale. The MOS sample was drawn from patients waiting in doctors offices, rather than from the general population. It is important to note that general measures such as the MHI-5 can only suggest the respondents perception of his or her mental health, rather than determine the presence of any actual mental health conditions. In addition, general measures are not informative with respect to specific disorders, their severity, or whether they interfere with daily functioning and employment. Research that attempts to use the MHI-5 to distinguish between respondents who have clinical conditions and those who do not, does not have a track record as strong as other measures that focus on individual conditions.

The Alameda survey included 53 questions selected from the 90-item Symptom Checklist-90, a measure developed in the early 1970s to gauge symptoms of mental distress: depression, interpersonal sensitivity, obsessive-compulsive behavior, paranoia, hostility, anxiety, phobic anxiety, somatization, and psychotic thoughts. Although this measure covers symptoms and distress in many different areas, it is important to note that it cannot suggest the presence or absence of particular mental health conditions. A recent report on this study (Driscoll, Speiglman, and Norris 2000) indicated that the measure failed to predict employment at the multivariate level, unlike results from other studies that use diagnostic assessments, such as the WES and the CalWORKS Prevalence Project.

A short, dimensional measure of nonspecific psychological distress was recently developed by Ronald Kessler at the Department of Health Care Policy at Harvard Medical School (documentation currently under review for publication). The measure captures the severity of symptoms associated with a broad range of mental disorders. The scale includes questions about cognitive, behavioral, emotional, and psycho-physiological symptoms that have been found to be elevated among people with mental health diagnoses. Six- and 10-item versions were carefully developed from a larger set of items, and extensive tests have demonstrated their reliability and validity. The 6-item measure is currently being used in the NHIS, and the 10-item measure is being used in the annual SAMSHA National Household Survey of Drug Abuse and the World Health Organizations (WHO) World Mental Health 2000 surveys (a set of representative community surveys being administered in 25 countries).

Survey Instruments for Specific Disorders

Depression. Depression, or depressive symptomatology, has been measured in dozens of surveys of welfare recipients. A 20-item measure of depressive symptoms developed by the Center for Epidemiological Studies at the National Institutes of Health (NIH) has repeatedly been shown to be reliable and valid when administered to welfare recipients, including studies in Iowa, Florida, and Minnesota, the New Chance Demonstration, the National Evaluation of Welfare to Work Strategies (NEWWS), and other studies. The CES-D, as it is known, has an established clinical cut-off point, permitting researchers to identify respondents who are at risk for depression versus those who are not (Ensel 1986). Other survey measures of depression include the Beck Depression Inventory (BDI), a list of 21 symptoms and attitudes that can be self-administered or administered by interviewers (Beck, Steer, and Garbin 1988). The major limitation of measures such as the CES-D and BDI is that they provide only screening-type estimates of the risk for clinical depression. Instead of suggesting a probable diagnosis, these measures provide an estimate of the number of depressive symptoms relative to others.

The more recent development of the Composite International Diagnostic Interview (CIDI) permits researchers actually to assign a diagnosis of depression based on the Diagnostic and Statistical Manual of Mental Disorders, version IV (DSM-IV).(3) The CIDI was developed by WHO for use in epidemiological surveys around the world, and was administered in the NCS, the largest psychiatric, epidemiological study in the past decade. Since then, the CIDI has been included in hundreds of studies that have documented and confirmed its reliability and validity. Designed for use with trained interviewers (rather than clinicians), the full CIDI was converted to a short form by Ronald Kessler (1998), the lead investigator of the NCS, making it especially useful in brief surveys. The CIDI-SF consists of several different modules for various diagnoses, including major depression.(4) The depression module of the CIDI-SF was used in the WES, the CalWORKS Prevalence Project, and the Healthcare for Communities Survey, among others. MPR also employed the CIDI-SF depression module in the Nebraska survey.

Generalized Anxiety Disorder (GAD). Compared to major depression, the potential presence of GAD has not been frequently measured in studies of welfare recipients, perhaps due to the unavailability (until recently) of a reliable survey instrument. The GAD module of the CIDI-SF was administered in the CalWORKS Prevalence Project, the WES, and the Healthcare for Communities Survey. The Alameda survey screened for GAD but used an instrument other than the CIDI-SF.

Post-Traumatic Stress Disorder (PTSD). The prevalence of PTSD in the past 12 months was measured in two surveys of welfare recipients by using the PTSD module of the CIDI-SF: the WES and the CalWORKS Prevalence Project. To be diagnosed with PTSD, respondents must first have experienced a traumatic event causing helplessness or horror, and then to persistently re-experience the trauma. Recurring symptoms include avoidance of trauma-related stimuli, outbursts of anger, disturbed sleep, and numbing of general responsiveness. Although there are many possible causes of PTSD  such as war trauma, being in a life-threatening accident, or witnessing someone being killed or badly injured  the CalWORKS Prevalence Project abbreviated the PTSD module by asking the respondent the PTSD questions from the CIDI only if he or she had experienced a physical or sexual assault some time in their past (instead of the 10 different events in the PTSD module).

Phobias. The CIDI-SF provides modules for specific phobias, social phobia, panic disorder, and agoraphobia. The CalWORKS Prevalence Project used the CIDI-SF modules to assess each of these conditions among their current welfare recipients and applicants. The major benefit of including measures of phobias is that they can be included in prevalence estimates that reflect all anxiety disorders. Yet all surveys must consider the trade-off between the constructs that should be measured and the survey length. Wave II of the WES included the CIDI-SF eight-item measure of social phobia in place of the GAD module that appeared in Wave I. The Healthcare for Communities Survey, a nonwelfare sample, screened for panic disorder.

Other Mental Health Conditions. Dysthymia and Lifetime Mania were assessed using the full CIDI in the Healthcare for Communities Survey, but these and other conditions have not typically been measured in studies of welfare recipients.

View full report


"report.pdf" (pdf, 367.08Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®