National Study of Child Protective Services Systems and Reform Efforts. Findings on Local CPS Practices. A.6 Analysis


Planning for the analysis began prior to data collection with the preparation of a detailed outline of the analyses to be performed. For each chapter of the final report, the outline listed the topics and tables that would appear in the chapter. For each table, the outline showed the section of the report, the specific variables needed for analysis, the topic of the table, the unit of analysis for the table, an indication of whether derived or recoded variables were needed to perform the analysis, an indication of whether the table required a subset of the data, and the statistics to be included in the table. The outline also included table shells to show how the table would appear in the final report.

The next step in preparing for analysis was to derive the variables needed for analysis. Using the outlines described above, programmer specifications were written with instructions on how to create all of the needed variables. Once the SAS program had been written and executed, the output was reviewed to check the accuracy of the derivations.

Before beginning the analysis, several sets of variables were merged onto the main database. First, the derived variables and the full sample and replicate weights were merged. Then, a select group of variables from the State policy analysis database were added to the analytic file. This included variables classifying the county as State administered, county administered or State administered with strong county structure, based upon the policy review analyses. Once all of the needed variables were merged onto the database, the analysis file was uploaded into WesVar.

In designing the analyses, the project team considered that specific subgroups of agencies could potentially provide quite different response profiles on the survey. A number of agency characteristics were identified as potentially important subgroup markers in this regard. The first analytic task was to test the utility of these candidate categorization schemes in relation to answers on a select set of survey items. The candidate schemes identified by the project team included number of CPS staff, average referrals per CPS worker, overall level of agency responsibility for functions, number of disposition categories, number of response types, administrative structure, metropolitan status, urbanicity, and median income. The project team conducted analyses on certain key tables to identify which scheme would be most informative. The tables that were examined included the following:

  • Agencies that share lead responsibility with CPS for different functions;
  • Involvement of law enforcement agencies for different CPS functions;
  • Most common referral source;
  • Overall level of responsibility for screening/intake function;
  • Response options for screened-in referrals;
  • Response options for screened-out referrals;
  • Types of responses;
  • Overall level of responsibility for functions;
  • Scope of the investigation response;
  • Scope of the alternative response;
  • Length of time current processes in place; and
  • Number of changes implemented in the last 6 months.

The categorizations schemes were tested by producing the crosstabulations necessary to complete the key tables. The project team reviewed these findings and found that income and administrative structure emerged as good candidates for further exploration. Chi-square tests were performed for most of the key tables to determine whether the distributions were significantly different for the various income categories and for the various administrative structure categories (see Table A-11). County per capita income was used for the income analysis. The agencies were divided into two groups — above the median and below the median. The chi-square analyses compared the two groups for each of the key items with the results in the table. The agency administrative structure classification used information from the State policy reviews to classify each agency into one of three groups — State-administered, county-administered, or State-administered with strong county structure. For each key item, the Chi-square analyses compared these three groups.

Table A-11:
Results of Chi-Square (X2) Analysis for Income and Administrative Structure
Key Table* Income Structure
X2 Value X2 Probability X2 Value X2 Probability
Overall responsibility for Screening/intake 5.893 .253 22.729 .012
New referral alleging maltreatment 20.195 .000 7.633 .470
Referral for child/household with an open investigation 6.196 .123 7.577 .476
Referral for child/household with prior substantiated report 18.142 .001 7.389 .495
Referral for child/household with prior unsubstantiated report 16.041 .003 7.39 .495
Referral for child who is in foster or substitute care 1.834 .676 7.458 .488
Type of response 1.829 .401 4.62 .329
Overall responsibility for investigation response 6.154 .188 32.054 .000
Overall responsibility for alternative response 2.238 .815 11.137 .347
Scope of investigation response 1.962 .521 15.278 .018
Scope of alternative response 1.664 .609 3.631 .534
Length of time current process in place: screening/intake 0.487 .912 3.203 .619
Length of time current process in place: investigation 2.716 .444 24.414 .000
Length of time current process in place: alternative response 2.592 .484 15.772 .009
Number of changes in last 6 months 4.26 .284 5.738 .432
* Includes those key tables with mutually exclusive categories.

The objective of this report is to present estimates of the percentage and number of the Nation's 2,610 CPS agencies . The data in this report are presented in one of several formats. When the rows of a table are mutually exclusive and total to 100 percent, the estimate, confidence interval, and percentage are given for each row. The 95-percent confidence interval provides a lower and upper bound for the estimate. This means that if the current study were exactly replicated 100 times, 95 of the replications would produce an estimate within this range. For other tables, the rows of the table are not independent and thus an agency can appear in any applicable row. For these tables, estimates and/or percentages based on the total number of agencies are given. For all tables, the estimates were rounded to the nearest 10.