The design, development, and piloting of a national APS data system was a major undertaking that required the commitment and leadership of the Federal Government in partnership with state representatives. More than 30 representatives from 25 states participated in the early stakeholder meetings; approximately 25 staff from the nine pilot states participated in the detailed activities required to test the design by actually submitting data from their own state systems. Furthermore, ACL continued to conduct outreach activities by presenting at several conferences, including information on NAMRS on its website, and briefing agency leadership.
The time and effort spent by the state agencies reflected their deep interest in collecting national data on the mandated scope, programmatic activities, and client outcomes of APS agencies throughout the United States. Moreover, the development of the NAMRS Pilot included the expertise of many federal and non-federal program administrators, researchers, and practitioners to maximize capturing data that would be useful to APS and to related fields, including caregiving, health, housing, employment, nutrition, financial, and legal services.
In developing a strategy for the full implementation of NAMRS, there are several next steps, which would become part of a multiyear plan for collecting, utilizing, and disseminating national data on adult maltreatment. Prior to even developing such a plan, ACL will need to decide if it will go forward in conducting the steps to fully implement NAMRS. If ACL decides to go forward, there are at least seven key components of a strategic plan to do so, including:
- Reaffirm the Vision
- Confirm Key Principles
- Obtain OMB Clearance to Collect Data through NAMRS
- Establish Operational Practices
- Detail the System Requirements for the Future NAMRS
- Enhance the NAMRS Pilot to Meet the Requirements for the Future NAMRS
- Develop the Communication and Support System for Working with State Agencies
Reaffirm the Vision
Simply stated, the vision underlying the project to design, develop, and pilot a national system was to collect accurate statistics on the maltreatment of older persons and adults with disabilities, based on data collected by all state APS agencies in the United States, including the territories. These data would describe the characteristics of vulnerable adults and the results of APS involvement in responding to their needs.
The vision included collecting detailed case-level data on persons who receive investigation and other services from APS. Case-level data would include demographics, risk factors, maltreatment allegations and findings, services, outcomes of the APS service response, and information on perpetrators. These data would be extracted from data in existing and future state information systems. Such data are often referred to as "administrative data."
Recognizing that not all states would be able to provide case-level data immediately, a short form for collecting key aggregated statistics was developed. A means for collecting policy and programmatic data on each state APS agency was also developed as contextual information.
This vision will need to be reaffirmed. The collection of annual national statistics is an extremely valuable exercise, but does not meet all the potential information needs of policymakers, researchers, advocates, service practitioners, clients, and their families. Administrative data are a foundational set of data, which, due to its periodicity and breadth, can provide many useful insights into social policy and practices.
Confirm Key Principles
The following list includes some of the key principles underlying the pilot:
Reporting would be from state and territorial agencies.
Reporting would be on an annual basis.
Reporting would be voluntary.
States and other stakeholders would have input into the identification of information needs.
States would receive technical support to assist them in participating.
ACL will need to revisit these key principles, determine whether they still apply, and identify other key principles. For example, the pilot did not include a detailed examination of the planned future uses of the data. Will there be an annual report? What form will the report take? Will ACL require specific data outputs that might be pre-programmed for easy accessibility? While this is part of the requirements for the system design, the general assumptions about use of the data could be included as a key principle.
Another set of principles could be related to response rate objectives. While it is known that not all states will submit case-level data in the first year of the future NAMRS, it is not clear at what pace states and territories might decide to submit the Case Component. For example, ACL might set targets of: (A) all states and territories submit the Agency Component in the first year; (B) 15 states submit the Case Component in the first year, 25 states and territories in the second year, and 35 states and territories by the third year; and (C) the remaining states and territories would submit the Key Indicators Component each year (40 in the first year, 30 in the second year, and 20 in the third year).
Obtain Clearance from OMB
The Paperwork Reduction Act requires the approval from OMB on any federal data collection exercise. This process includes several steps, including the first and second notices in the Federal Register, development of a supporting statement by the agency, which is submitted by the department to OMB, and completion of various automated forms required by OMB. The supporting statement includes a description of all data elements to be collected, as well as an estimate of level of effort to provide such data.
The NAMRS Pilot served the crucial function of determining if states could submit case-level data, and provided feedback used to refine the data elements, values, and definitions. The revised version will be submitted to OMB for approval. The NAMRS Pilot also provided sufficient information to estimate the level of effort of states to participate in the future NAMRS.
The OMB package needs to be drafted and reviewed by ACL. If the first Federal Register Announcement is published in October, OMB should complete its review by May or June 2016, which is sufficient time for launching the more specific instructions for reporting FFY 2016 data by September 2016 or earlier. ACL is likely to develop a communication plan, which inform states of plans for the future NAMRS in advance of the first Federal Register Announcement. (See Develop Communications and Support System below.)
Establish Operational Practices
The FSC oversaw the work of the NAMRS Pilot project. The FSC consisted of representatives from ASPE, HHS Administration on Aging, HHS Administration on Disability, and the Office of Information Resources Management. The Office of Performance and Evaluation also attended many of the FSC meetings with the contractor responsible for the technical work.
ACL will need to decide where the future NAMRS initiative is "housed," whether a steering committee will be used, and which key ACL units will be members of the FSC. ACL will need to decide what responsibilities will be allocated to contractor staff and what responsibilities will be allocated to federal staff. The role of the regional offices will also need to be considered.
Since ACL is undertaking the revision of other major data collection efforts, there may need to be more intra-agency coordination that otherwise might be required. Quarterly meetings or even bi-monthly meetings may be useful to identify areas of useful coordination, especially with regards to requesting data from state agencies.
The annual data collection schedule will need to be determined, with consideration of its feasibility and impact upon any required reports or data analyses. The NAMRS Pilot worked with states over a period of 6 months from initial contact through final data submission. If the initial formal request for data is sent in September-October 2016, states are likely to be able to complete their data submissions by March-April 2017. Initial data analyses could be released by June 2017.
Detail the System Requirements for the Future NAMRS
This report includes much of the information for developing the requirements for the future NAMRS. Revisions to the data content are fully explicated; recommendations as to technical enhancements are also identified and prioritized.
ACL will need to review this report and determine which requirements will be operationalized in the future NAMRS. Technical staff will develop detailed specifications. Once approved by ACL, systems developers will implement the requirements and conduct testing.
Enhance the NAMRS Pilot to Meet the Requirements of NAMRS
As discussed in this report, ACL could decide to refine the future NAMRS using the Agile Scrum methodology. This would allow for rapid and efficient development of the system, and an iterative approach to developing requirements.
As part of the work plan for the next phase, ACL would approve the proposed development sprints, and participate in the acceptance of each sprint.
Voluntary (and required) reporting systems work best when there is clear communication between the data suppliers (i.e., the state agencies) and the data recipients (i.e., the Federal Government). This communication must be in both directions, with channels for posing and responding to questions. Clear and open communication is also the best means of building consensus in terms of systems content and systems objectives. A return on investment must be evident to the potential data suppliers.
One of the observations of the pilot test was that face-to-face contact with the states was highly beneficial. The current restrictions on holding meetings funded by the Federal Government may hamper the use of this critical method of communication. The main means of communication may be webinars, conference calls, and individual calls.
Another communication challenge discovered through the pilot test was that APS units did not always have ongoing working relationships with the IT units and often had challenges in terms of resource allocation or prioritization of requests to the IT units.
ACL will need to establish the parameters of communication with the states both in advance of the formal request for data in the fall of 2016, and during the data collection period in FFY 2017 and subsequent years.
ACL will also need to build from the experiences of the pilot phase to help states work more closely and efficiently with their IT units. For example, it may be useful to include the senior leadership of state social services umbrella departments in early communications about the future NAMRS to address resource allocation issues at the highest levels, rather than solely at the APS level. SUAs and social services agencies may need help in examining the resource issues related to providing Case Component or Key Indicator Component data.
The NAMRS Pilot confirmed the feasibility of collecting national data on the maltreatment of vulnerable adults. Implementing the full system will require a strategic plan, which addresses key issues such as vision, key principles, Paperwork Reduction Act requirements, enhancing the system functionality, and communication with the state agencies.