National Study of Child Protective Services Systems and Reform Efforts. Findings on Local CPS Practices. A.3 Data collection


The survey was mailed to all 383 CPS agencies that were identified as serving the 375 sampled counties. Data collection lasted for 19 weeks — beginning February 1, 2002, and ending June 14, 2002. The targeted response rate was achieved — 307 sampled agencies completed the survey, for a response rate of 80 percent.

To facilitate data collection, 6 groups of approximately 60 to 65 agencies were targeted each week for data collection. Four main activities were involved in data collection — State solicitation, agency solicitation, survey distribution and retrieval, and nonresponse followup.

Prior to data collection, State child welfare directors were contacted to request permission to contact the sampled counties. Seven States declined to participate in the survey during this phase, which resulted in a loss of 63 potential responding agencies. Next, an initial letter was mailed to the director of each local CPS agency that introduced the study and requested the agency's participation. In addition to the letter, a "Frequently Asked Questions" sheet and a 5-page overview of the study were sent.

A confirmation call was made to the local CPS agency directors approximately 1 week after the solicitation letters were sent. The purpose of the call was to confirm the agency's participation and to respond to any questions the agency director might have. Generally, the survey was well received and most agencies agreed to consider participating. However, 36 agencies initially declined to participate in the survey.

During the confirmation calls, participating agencies were given preliminary instructions on how to complete the surveys. Multicounty agencies (CPS agencies that provide services to more than one county) and agencies that served multiagency counties (CPS agencies that share responsibility with one or more other CPS agencies for serving a single county) were given instructions on how to complete the survey to accommodate their special circumstances. Multicounty agencies that provided services to more than one county in the sample were instructed to complete only one survey for all the counties they served as long as the same practices and procedures were used. Agencies that used different procedures for different counties were instructed to complete a separate survey for each county. Agencies in multiagency counties were instructed to respond to the survey based solely on the section of the county they served.

Given the number of initial State and agency refusals, special efforts were necessary to boost the number of participating agencies to reach the targeted response rate of 80 percent. Senior members of the research team contacted the seven States that had initially denied permission for sampled counties to be included. This effort was successful and enlisted 3 additional States for the survey, which increased the number of potential respondents by 34 agencies.

An additional effort was made to enlist the support of the County Welfare Director's Association in one large State where, despite State CPS agency approval of the study, some agencies would not participate without the endorsement of the State's County Welfare Director's Association. Contact was made with the director of that association who then agreed to support the survey effort. This increased the number of potential respondents by another 13 agencies.

A receipt control database was developed to assist in administering the LAS, primarily to monitor survey progress and to create mailing lists. The tracking database recorded all contact information and agency participation status. The database was updated after each contact with an agency, as well as whenever a completed survey was received.

Survey packages were sent to those agencies that initially agreed to participate. Survey packages were also sent to any agency that had not been reached after 1 week of confirmation calls. The survey packages consisted of a confirmation letter, the five survey modules, and a return Federal Express envelope. Participating agencies were given 3 weeks to complete and return the survey.

Participating local agencies were given a toll-free telephone number and an email address to contact the survey administration team if they had any questions about the survey. The toll-free telephone number and email address were checked for incoming messages at least twice daily.

A number of efforts were taken to increase the response rate among those agencies that had initially agreed to participate in the survey but had not returned a survey. First, reminder postcards were mailed to agencies that had not returned their completed surveys within 2 weeks of receiving the initial survey package. A second survey package was sent to nonresponding agencies 4 weeks after the initial survey packages were sent. Followup calls were made to agencies that had not returned completed surveys 6 weeks after receiving their initial survey package. In addition, at the end of the survey administration period, State child welfare directors were requested to encourage nonresponding counties (and counties that had originally declined to participate in the survey) to respond.