Building a More Responsive Federal Workforce: Lessons from the SC2 Pilot. 1.1 Methodology



In this report, we attempt to address some of the gaps in knowledge that have resulted from the split between child protection and child welfare services by comparing child welfare system data from two states (Illinois and Michigan) and sub-populations within those states. We will focus on the indicators which we can make comparable in the two states. Our long term goals include analyzing each state separately in order to illuminate important issues about child and family experience with the child welfare system that are unique to each state.

Comparability means that the data in each state has similar definitions. Making data comparable is a time-consuming task, requiring deep understanding of each of the jurisdictions which are to be compared. There is variation in how and where states keep certain pieces of information in their data systems about the service events the children experience, and it often takes considerable detective work to determine whether data items are the same across states and often within them.


The basic method of this study is the aggregation of the investigative and service events for a particular child into a sequence of events that they experience across the system. We accumulate events so that, for each child, we have the sequence of events that he or she experienced.

We follow the child from first investigation through last event of his or her first episode of service in the system. A complete episode would include the following events:

  • Birth of the child
  • Investigation records to which the type of allegations and the results of investigations are attached. Investigations occur after a report of abuse or neglect to a state central registry (a “hotline”). Not all reports are investigated and those reports that are not investigated are usually not recorded. Often the results of investigations are recorded soon after the report, but they must be concluded within 21 days in Michigan and 180 in Illinois. An investigator can remove the children from the home immediately (protective custody) and place them in an out-of-home placement before the investigation is complete. (Juvenile court must rule on the appropriateness of the removal within 48 hours.) We use the date of the report as the date of the investigation. Since we are interested in whether a child is part of a substantiated report, we assign the outcome of the family investigation to the child.
  • Case openings and closings for the child occur when the child is going to be placed in an out-of-home placement (the vast majority) or when the child is going to be the focus of some intensive in-home service.
  • Out-of-home care placements include foster family care, kinship care, specialized foster care, group homes, shelter care, residential care and other out-of-home placements (e.g. hospitalization) that the child may experience while in the custody of the state.

Of course, all children may not experience each of the events. We follow each child through all of his or her events until she or he has experienced a complete episode.

All of the events are merged into one file and the characteristics of child, caretaker or family are attached to these events. From the series of events that each child has experienced, we create a sequence of events. We then count how many children experience each type of sequence and cross-classify these by independent variables (region, year, type of initial allegation). The children are classified into cohorts by the year of their first contact with the system. In our database for this study, we include children from the 1990-1994 cohorts and follow their experiences through the end of 1994.

When tracking children from 1990 to 1994, unless otherwise noted, it is important to note that for each subsequent cohort, there is one less year of follow-up time that we have in our database. For children who had their first contact in 1990, there is follow-up through 1994. For children who had their first contact in 1994, we only look at that year’s subsequent events for that child. To adjust for this, we have also analyzed our data for all of the events that children experience only one year after their first contact with the system.

Because unsubstantiated records are expunged from the system, in compliance with state law, meaning that all identifying information on the records of the involved individuals is deleted, if a child does not have a substantiated investigation previous to an unsubstantiated one, that child may be double counted over a period of time. If a child experiences two unsubstantiated reports within a short period of time (usually less than a month), the child will likely be double counted. Also, if a perpetrator requests that the record not be expunged, because he or she wants to prove harassment on the part of a reporter, a child’s unsubstantiated reports will also be “trackable” over a longer period of time. The privacy concern is that states do not track perpetrators who have not been involved in substantiated cases.

Child welfare case opening refers to how many children have case openings of their own. This is the least comparable event in our report, since case opening is very much an administrative function and because families can also have case openings under which children can receive services. In Illinois, we are able to track family case openings for each child, while in Michigan, there is no source for such information. Thus, this will be an instance where separate analyses will have to be done for each state.

There are cases which come into the child welfare services domain without first experiencing an abuse or neglect investigation. These are cases in which for some other reason, the child and parents cannot live together. These reasons include the child’s behavior (children too young to be incarcerated), dependency (those cases in which the child may be an orphan or the parent in institutionalized) and, in some cases, status offenders. Since the practice around children entering child welfare services without an abuse or neglect report is highly variable from one county to another, not to mention one state to another, an argument could be made to drop these children from the study altogether. However, since these children are receiving service from the child welfare system and they do represent a sizable portion of the child welfare caseload, we include them.

Given that some children can have upwards of 20 either substantiated or unsubstantiated investigations and 5 case openings and 5 placement spells, the possible combinations that we have put together are in the thousands. We must make certain assumptions about types of cases in order to reduce our analytic units to a reasonable number.

We have taken the string of investigations that children experience and reduced them to three types. The first grouping is only one substantiated investigation with any number of unsubstantiated investigations. The second grouping is 2 or more substantiated investigations and any number of unsubstantiated investigations. The third is any number of unsubstantiated investigations. These investigations all have to occur before a case opening, which must occur before an out-of-home placement. Investigations that happen after a case opening or out-of-home placement constitute another type of sequence or maltreatment while in foster care.

In Michigan, the 1992 and 1993 cohorts’ placement figures will increase significantly since we could not obtain 1994 placement data from the state of Michigan. Children from the 1992 and 1993 cohorts could be placed at any time in the future. In 1992, there was a change in policy in Michigan, where the state had to inform the perpetrators of alleged child abuse or neglect. While a decrease in the number of substantiated reports is seen at that time, it is unclear what proportion is due to the follow-up period being short or due to the change in policy. With additional follow-up, the reason will become clearer.


Structure of the answer to each question

Each of the sections will discuss a comparison of Illinois and Michigan - not only the numbers, but the different ways in which practice and policy affect the experiences of children. The answer to the question will come first. We will study children and families who came into contact with the system for the first time from 1990 to 1994 (trend). We will compare the major urban area (Chicago in Illinois, Wayne County in Michigan) to the balance of the state. We will also analyze the sequences by differences in initial allegation type (sexual abuse, physical/medial neglect, social neglect/abandonment, other abuse). Subsequent to the statistics, there will be a short discussion of policy and practice issues.

What percentage of children in the general population have substantiated abuse or neglect allegations for the first time in a particular year?

We find that 0.91% in Illinois and 0.85% in Michigan of the children 0-19 years of age had substantiated abuse or neglect allegations for the first time in 1990. When one compares the 1 major urban area and the balance of the state, a different picture emerges. In Chicago, 1.52% of the children in 1990 were a newly substantiated case of child abuse or neglect, while only 0.72% of the children in the balance of Illinois fell into this category. Thus, twice as many children are being substantiated in Chicago compared to the balance of Illinois. However, in Michigan the difference is much less, with 0.92% of the children in Wayne County and 0.81% of the children in the balance of Michigan being a newly substantiated case in 1990.


One must look to the child protective side of the child welfare system for possible explanations of the difference between Chicago and other three regions (balance of Illinois, Wayne County and balance of Michigan. Possible explanations include a greater maltreatment in certain sub-populations (Chicago) and an environment of more surveillance and reporting by reporters, (police, medical personnel, teachers and neighbors) with greater pressures to substantiate allegations of abuse and neglect.

Given that the socioeconomic and demographic characteristics of Chicago and Wayne County are so similar, it is unlikely that there is greater maltreatment in Chicago. Thus, one must further investigate the possibility that the environment is the cause of this must be studied further. Such a study is beyond the scope of this report. However, in Illinois, particularly in Chicago, a great deal of media attention is paid to the issues of child abuse and neglect and foster care. (A comparative study of media attention has never been done, thus a statistical analysis of media attention is not available.) It may be that public vigilance is greater in Illinois, particularly in Chicago, resulting in more media coverage. Illinois medical personnel are also required to report all confirmed cases of illegal substances in a child’s urine. All of these substance-exposed infant cases are substantiated cases of neglect.

Finally, Michigan’s AFDC benefits are clearly more generous than that of Illinois’s and may account for a large fraction of the difference since most experts believe that many families come to the attention of the child welfare system because of reasons associated with poverty. While difficult to prove without a more rigorous experimental design, more generous benefits may act on the margin to keep certain families from being reported and substantiated for child maltreatment. It may be that this effect is different in Chicago and the balance of Illinois.


1 We use 1990 because the decennial Census figures are available for that year, making the calculations most accurate.

Of children who have contact with the child welfare system, how many are placed into foster care?

While there is great concern around the high numbers of children being placed, in actuality, a small percentage of the children who come in contact with the child welfare system enter substitute care.A greater percentage of children who have contact with the child welfare system in Illinois are placed into foster care than those in Michigan (See Tables 1 and 2). Over 7% of all contacts with the child welfare system in Illinois resulted in foster care placements during the period from 1990 to 1994. In Michigan between 1990 and 1993, almost 4% of all children with initial contacts resulted in foster care placements. When the two states are compared by year, Illinois’ figures are 35% higher in 1990 and more that 100% higher in 1993.

Since the follow-up periods varied in each of the states, we controlled for this by simply looking at what happened one year after the first contact (See Tables 3 and 4). Between 70 and 80% of all placements we found in the longer follow-up periods occur within one year of the time of the initial contact. When using this period of time, we still find that the placement rate is 30% (in 1990) to 80% (in 1993) higher in Illinois.

Table 1.

Percent of First Contacts Placed: Illinois Full Histories

Percent of First Contacts Placed: Illinois Full Histories

19901991199219931994Grand Total
Balance of Illinois5.7%5.9%5.4%5.6%5.0%5.5%


Table 2.

Percent of First Contacts Placed: Michigan Full Histories

 1990199119921993Grand Total
Balance of Michigan5.9%5.1%3.9%3.0%3.9%

This one-year follow-up period also allows us to study the trends within each state. In Michigan, the percentage of children being placed is decreasing, while in Illinois, the placement rate within one year of initial contact increased through 1994.

Wayne County, which has slightly over a quarter of the initial contacts in the state during this period, has slightly lower one-year placement rates than the balance of the Michigan. Chicago, with about a third of Illinois’ first contacts, has a one-year placement rate 80% greater than that of the balance of Illinois. Chicago’s rate is also growing and driving the increase in the entire state. It grew from 7% in 1990 to 10% in 1994, with the 1994 figure certain to increase because of a censored follow-up period for the 1994 cohort of children with initial contacts.

As one can see from Figures 3 and 4, the majority of the children placed are those with either substantiated investigations or those who enter the system without an investigation.

Table 3.

Percent of First Contacts Placed One Year Histories

 19901991199219931994Grand Total
Balance of Illinois4.5%4.6%4.5%5.2%5.0%4.8%


Figure 4a. MICHIGAN One Year Histories 1990-1993


Figure 3a. ILLINOIS One Year Histories 1990-1994


More about the substantiated rates is discussed in the following section.

From 1990 to 1994, one fifth to one quarter of those children entering placement do so without initially experiencing an abuse or neglect investigation. The numbers in Michigan have been decreasing, while in Illinois there has been a slight increase. These children have entered placement without any previously substantiated case of abuse or neglect. These children, often put in the custody of the child welfare agency by juvenile courts to avoid incarceration or because of the need for children to live in foster care because of a parent’s inability to care for them because of illness, death or incarceration, do account for a high percentage of the foster care entry.


Until both the child protection data and child welfare services data were combined, it was not possible to determine how many children came to the attention of the entire child welfare services domain for the first time in a particular year and who were not previously known to the system and then placed into foster care. This is because one never knew whether or not a child’s entry into one domain in a particular year had been preceded by an entry into the other domain. With a longitudinal design and data, one is able to properly classify children into their actual entry cohorts.

The placement figures are very consistent with the findings of the Multistate Foster Care Data Archive results, where the incidence rates of first entry to foster care for the entire population of children is from 15 to 75% greater in Illinois compared to Michigan from 1990 to 1993.

The issue of children who enter foster care without a child abuse or neglect report may be one that will become more important in the near future. These children are often not considered in discussions of the child welfare population, which has been dominated by the issues of child abuse and child neglect. Because of the increasing numbers of children entering foster care in Illinois, as well as the proportion, the Illinois legislature has passed legislation (implemented in July 1995) to exclude those children who are over 13 and have been adjudicated as being delinquent. While it is still unclear as to whether or not the legislation will have an actual effect on practice, it certainly symbolizes an effort (as may the decrease in Michigan), to remove the children without a substantiated report of abuse or neglect from the foster care population.

Of those children who are investigated, what percentage are substantiated?

In both states, about two-thirds of the children who have first contacts with the state never experience a substantiated allegation of abuse or neglect in their first contact or any subsequent contact during the period of our study . In two-thirds of the cases, investigators do not find evidence of abuse or neglect (See Table 4).

Table 5. First Contact Types by Finding Illinois & Michigan

Illinois Investigation Finding19901991199219931994


Chicago Investigation Finding19901991199219931994


Balance Of Illinois Investigation Finding19901991199219931994


Michigan Investigation Finding19901991199219931994


Wayne County Investigation Finding19901991199219931994


Balance Of Michigan Investigation Finding19901991199219931994

* Michigan CPS data complete only through August of 1994

Over the study period, the percent of initial contacts in a year that are substantiated remains remarkably consistent in Illinois. The differences between Chicago and the balance of Illinois are small. In Michigan, while the 1990 and 1991 substantiation rates were very similar to Illinois’s, 1992 and 1993 each showed a large decrease (more than 15%) from the earlier years, so that now only 25% of investigations are substantiated. We assume this was due to the policy requiring investigators to notify perpetrators and the investigators’ behavior in reaction to this policy. The decrease that we see statewide is more pronounced in Wayne County (more than 25% decrease) than in the balance of Michigan, which suggests that the impact of the policy change mentioned above on investigatory practice is less in the balance of Michigan that in Wayne County.


Since, as will be discussed below, less than one percent of children’s reports that are unsubstantiated receive foster care, this is obviously a key gate-keeping mechanism. In all of these cases, the child welfare agency has sent an investigator to these homes and determines that there is no credible evidence for the substantiation of abuse or neglect.

If we had only studied 1990 and 1991, we would have put forward a hypothesis that a 33% substantiation percentage can be expected in these states and perhaps this might be a standard by which other states could gauge themselves. However, given the significant decrease in Michigan in 1992 and 1993, we now see how quickly this statistic can change due to a policy change. While we again have not done a study that proves that the decrease in substantiation rate is due to the change in notification of the perpetrator, few other types of changes (e.g. socioeconomic ones) could have such a great effect.

Of children who are investigated for abuse or neglect, how many are placed in foster care and how does it vary by whether or not the report is substantiated?

Statewide in Michigan, 8.5% of those children substantiated are placed in foster care, while in Illinois nearly 14% are placed, a nearly two-thirds larger placement rate (See Table 6). Less than half a percent of children with unsubstantiated cases are placed. Within states, Wayne County has a slightly lower placement rate than the balance of Michigan, while Chicago’s placement for substantiated cases is nearly twice that of the balance of Illinois.


Given the somewhat higher substantiation rate in Illinois than in Michigan over this period of time, one might expect a lower percentage of placement for those substantiated in Illinois. This is because one might expect the severity of cases in Michigan to be greater, all else being equal. However, this is not the case. More children come to the attention of the child welfare system in Illinois, more are substantiated out of those that come to the attention and more are placed of those who are substantiated. The distribution of sequences that children experience in Illinois is certainly skewed toward more children being substantiated and placed.

Table 6. Percent of Children Investigated for Abuse/Neglect Placed in Foster Care

 Illinois 1990-1994Michigan 1990-1993

* We do not have Michigan foster care placement data for 1994

How does placement into foster care vary by the type of initial abuse or neglect allegation?

In both states, the children who are substantiated cases of social neglect and physical or medical neglect are most likely to be placed in foster care (18% in IL, 12.5% in MI; see Table 7). When one compares the major urban area with the balance of the state, significant differences are seen. Chicago has a much greater placement rate for socially neglected children (31% in Chicago to 12% in the balance of the state). The balance of Michigan has nearly twice the placement rate for physically neglected children as Wayne County.

Table 7. Placement By Allegation Type For Substantiated Investigations Illinois & Michigan

IllinoisNo AllegationSexual AbusePhysical AbusePhys/Med NeglectSocial Neglect
Not Placed95.8%92.2%83.9%82.1%81.1%


ChicagoNo AllegationSexual AbusePhysical AbusePhys/Med NeglectSocial Neglect
Not Placed93.6%90.0%75.3%83.0%69.0%


Balance Of IllinoisNo AllegationSexual AbusePhysical AbusePhys/Med NeglectSocial Neglect
Not Placed97.4%92.8%87.0%80.9%88.4%


MichiganNo AllegationSexual AbusePhysical AbusePhys/Med NeglectSocial Neglect
Not Placed96.0%94.2%91.8%87.5%87.5%


WayneNo AllegationSexual AbusePhysical AbusePhys/Med NeglectSocial Neglect
Not Placed97.1%95.5%91.8%91.3%86.8


Balance Of MichiganNo AllegationSexual AbusePhysical AbusePhys/Med NeglectSocial Neglect
Not Placed95.6%94.0%91.9%85.0%87.6%

Except for those children who do not have a specific allegation, in both Illinois and Michigan the lowest percentage placed are those children substantiated as being sexually abused. The placement rate for sexually abused children is low because, often, the perpetrator has been removed from the family’s home as the result of an arrest or a court order.

Appendix Tables 1, 2a-2c, and 3a-3c describe the allegation classification, and provide more detail on the relationship between initial allegations and substantiation.


The difference in the placement rate between Chicago and the balance of state in Illinois is likely the population that enters kinship care in Chicago. Significant analysis of this issue has taken place in Illinois recently and it has been found that a great proportion of the children entering care are those who are reported by their grandparent for being abandoned by their parent. Recent legislation aimed at attempting to provide permanency for these children with relatives has been passed in Illinois in an attempt to address what some think is an inappropriate use of the child protective system.


Similarities and Differences

In studies comparing two or more regions or sites, especially when not using an experimental design, one must often speculate as to the reasons for differences between the sites, simply because of not having data on all the possible explanations. In this study, we have attempted to explain differences, but there are limits due to the depth of our knowledge about practice, policy and administration of the child welfare system. We hope to increase our knowledge in the future to better explain some of the differences between Michigan and Illinois. We intend to use the expertise of practitioners, policymakers and administrators in the states we study and outside of those states to improve the relevance of the work.

Prevention and early intervention

The findings on the percent of children coming into contact with the system and the type of contact they have (often no substantiated cases of abuse or neglect) suggest that points of early intervention might be clearer than previously thought. It is generally believed that even families where there is a report but no substantiation of abuse or neglect may require additional support and in some cases a high level of intervention. Further work could help pinpoint those neighborhoods and types of families which might be good targets for prevention and early intervention services.

Response of the state

Understanding the flow of cases at each juncture can help characterize the service system of the state. The higher placement rates in Illinois with the higher first contact rates obviously result in more children going into placement, per capita in Illinois. However, if one state had a higher placement rate and lower first contact percentage, it may be that the actual per capita placement rate would be very similar in the two states.

Future Work

In this, our first report, we focused primarily on the first complete sequence. In future work, one could look at all sequences, which primarily entails looking at reentry cases and reabuse cases in greater detail. One could also explore adding other services beside casework and out-of-home care, e,g. various counseling, homemaker, family preservation and psychiatric hospitalization.

In addition to additional description of sequences, multivariate analysis should be undertaken to understand the factors that contribute to different outcomes. Such analyses would entail analyzing the characteristics of children, families and services that increase or decrease the probability of making a particular transition.

Small region analysis is necessary if one is to accurately understand the heterogeneity of the child welfare system. We commonly analyze the experiences of children in the child welfare system at the state level and incorrectly treat children as if they are all from the same geographic region. One could analyze the probability of making certain transitions in various types of communities. For example, is it more likely that a child will be substantiated for neglect in a community primarily consisting of unmarried, teenage mothers?


The biggest limitation in studying the sequences of services that children experience is how the data is collected by state agencies. The process that we have gone through in this study is not possible to accomplish in every state. However, the Federal government has given the states an opportunity and resources to address this issue through the State Automated Child Welfare Information System initiative. One of the areas the Federal government addresses in SACWIS is linking the two child welfare system domains.

View full report


"rpt_BuildingFederalWorkforce.pdf" (pdf, 383.19Kb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®