Development of a National Adult Protective Services Data System: Namrs Pilot Final Report (volume 1). 6. Recommendations for Supporting the Future Namrs

09/25/2015

State participation in the development of the content of the NAMRS Pilot was also a critical factor in the successful enrollment of states. All of the states that participated in the pilot had attended at least one meeting on the overall design of the NAMRS Pilot. The actual submission of data to the NAMRS Pilot required extensive technical support to help states understand the data specifications from the perspective of their own systems and practices, and to be able to work closely with the IT units in their departments to prepare the data submissions. This chapter discusses lessons learned from the pilot process in terms of support that was useful to the states in actually submitting data to the NAMRS Pilot, including the topics:

  • Support Provided to the States During the Pilot
  • State Perspectives on the Pilot Process
  • Recommendations Related to Support of States During Full Implementation of NAMRS

     

Support Provided to the States during the Pilot

The NAMRS Pilot Website was developed to provide a central site for states to access resources, user guides, and announcements. The states also submitted their data through the NAMRS Pilot Website. The website was very basic, but effective in providing information. The announcements page kept states abreast of new components added to the site, user guides, and notices of modifications of the NAMRS Pilot during the 6-month period.

State team leaders submitted staff names and emails of persons who needed access to the website. Liaisons set up logins for the staff to access the website. A user guide provided basic instructions on accessing resources available on the website. Some state staff had to update their Internet browser, as the NAMRS Pilot Website required Internet Explorer version 9 or above or Google Chrome. During the pilot, staff names and emails were added or deleted upon request. State staff could send inquiries and receive responses by email to the project team. The website was designed to generate emails to state users when specific events occurred such as when a state submitted the Agency Component data.

The website contained many resources for the state staff. User guides provided instructions on accessing the website and loading and submitting data. Data specifications documents provided details on the data elements such as the definition, type, and values. The Code Values and Definition document listed detailed definitions and data values. The Validation Rules documentation provided a summary of the validation rules for submitting the Case Component data. The XSD file were provided to support the development of the case-level data extraction file from the states.

Case Component Mapping Forms

The Agency Component and the Key Indicators Component in the NAMRS Pilot were largely self-explanatory. Understanding the data specifications of the Case Component was more complex, and required more TA.

The process of matching and documenting states' Case Component data to NAMRS Pilot data was called mapping. The project team provided Microsoft Word forms used to complete the mapping documentation (See Appendix G). The mapping forms included the main page that provides data specifications such as the type, size, requirement, and descriptions. If the NAMRS data element was a coded element, a second page was included. The second page allowed the state to map the values used by the state to the NAMRS Pilot values for that element.

Technical Assistance

Each state APS team was assigned a liaison for TA. Eight of the nine states agreed to regularly scheduled calls to provide updates and discuss any issues. One state opted for regular email updates and requested TA by email or phone, as was needed. When questions were of an IT nature, technical staff provided TA. States valued having a project liaison for quick answers to questions or for longer discussions, including guidance on the mapping forms. A total of approximately 80 hours of TA was provided to the nine pilot states, including the review of their data submissions.

  • Approximately 10 percent of the 80 hours was spent in helping states with the Agency Component; 20 percent was spent in helping with the Key Indicators Component; 50 percent was spent on helping states with the mapping forms; and 20 percent was spent on reviewing data submissions.

State Perspectives on the Pilot Process

States expressed their appreciation for the TA and made many useful suggestions for the improvement of the system or for improving the process. The biggest challenge for some states was competition within large departments for IT staff time to dedicate to the pilot project. One state paid their third party IT contractor to assist with the pilot work. Another state began the mapping process relying on quality assurance/data analyst staff to do IT developer tasks and realized later in the process that they needed an expert in XML to complete the task. States provided the following insight:

  • One state IT staff reported that the lookup lists were helpful in assisting them in rewriting their system. IT staff liked the "submit & resubmit" and "testing & error report" functions of the pilot. The staff in this state acknowledged the challenge of going through the mapping process, but stated that it was necessary work. In one instance (services referred data), the state was able to find an error in their state data system. The mapping form was easy to use, file, and update. One suggestion was to limit/decrease the tester emails sent to everyone especially during the validation phase. Staff was pleased with the responsiveness and coordinating efforts of their liaison. The NAMRS application and their participation in the pilot received positive compliments. The state team provided a wish list for NAMRS.

  • Another state reported that their main lesson learned was that they should have brought in IT staff sooner rather than later especially during the mapping process. They spent many hours writing and rewriting code between programmatic and IT staff. They were not proficient in XML. The state suggested:

    • Place all instructions on the website with no password protection.
    • Do not use encryption and unknown tags.
    • Provide validation utilities to states.

       

  • A third state suggested that the case mapping form should match the XML (i.e., have a path to each note). State staff would like the ability to pull reports from the NAMRS data warehouse. Programmatic staff would prefer to map all services provided to a person in their personal residence as "in-home services." The team appreciated the assistance and being included in the pilot.

  • A fourth state used an MS Excel spreadsheet to complete their mapping process. They found it easier than completing individual forms in MS Word. They thought that overall, the NAMRS Pilot process was easy to follow.

  • A fifth state found the NAMRS Pilot process relatively easy. They had reports developed already that provided the information asked for in the Agency and Key Indicators Components. They only needed to tailor the report period to the FFY and were able to gather the data. They said submission of the data was easy to do with the website. They remarked that they would definitely be able to provide Case Component data in the full implementation version of NAMRS.

  • A sixth state thought that working with the NAMRS Pilot data elements was useful as they reviewed their state data. The reports from the NAMRS Pilot provided useful information. The mapping process was tedious but not difficult and very useful in the development extraction program for the XML file. They thought it would be useful to have a cross reference between the XSD file and the mapping forms.

  • A seventh state used spreadsheets to calculate the data needed for the Key Indicators Component. They indicated that it might have taken less time to prepare the Key Indicators Component data if they had more reporting capabilities. They said they would refer to the NAMRS Pilot data as they reviewed their system data in the future. They found the reports from the NAMRS Pilot very useful.

  • An eighth state indicated that gathering the data for the Agency Component and Key Indicators Component took more time than originally thought. They developed reports to produce the data in the prescribed format. They are looking forward to the future development of NAMRS and the benefits it will provide to the APS field.

Recommendations Related to Support of States During Full Implementation of NAMRS

The overall finding based on providing TA was that the provision of such assistance is crucial to the success of data collection activities. Full implementation will also require an effective communication plan. Communications will be the key to helping all APS programs, regardless of state administrative placement, to stay informed.

Finding 1. For most states, the ability to provide data to the NAMRS Pilot was dependent upon access to TA. While not all states needed the same amount of TA, user documentation, mapping forms, and TA provided by phone were critical to the states. Orientation webinars were also useful.

Action: User documentation, overview webinars, and assignments of particular persons to be the liaison for each state should be included under the full implementation of NAMRS. TA efficiency may be gained if more structured TA tools were used, for example, a tool to log and track detailed questions and issues from the states with resolutions. The tool would assist team members providing TA by allowing them to search for similar issues with resolutions and identify trends in common questions.

Finding 2. There were several new data topics. Specific topical areas often proved more challenging to particular states. For example, many state APS programs do not have a history of collecting data on perpetrators, on clients' legal status, or legal remedies.

Action: Guidance, training, and TA will need to be provided in these key areas. The NAMRS Pilot used subject matter experts to provide TA. The team consisted of one person who was somewhat familiar with aging programs, but very familiar with client tracking systems, and one person who was an expert on aging programs, but with less knowledge of information systems. Both perspectives, content and technical infrastructure, must be available through TA.

Finding 3. Mapping forms were challenging for the states to complete. While recognizing the value of documenting the cross-walk to their own systems, the states found the process burdensome.

Action: Some examples of potential approaches to improving this process are included in the next chapter.

Finding 4. The XML format of the Case Component was challenging for some of the states. The ability to provide the Case Component data in this format was completely dependent upon the skill sets of the IT department or unit.

Action: No change is recommended, as most state IT departments are increasing their skills in using XML.

Finding 5. Submission of data to the NAMRS Pilot required collaboration with other departments. All states recognized that they needed to work closely with their IT departments to submit the Case Component data. States that had strong relationships were able to work smoothly with them. Other states had more difficulty gaining access to the IT departments or even working with them.

Action: TA to support the full implementation of NAMRS could include additional guidance in locating the appropriate technical personnel who will be needed to develop the extracts for the Case Component. Some state IT staff may require specialized TA.

Finding 6. State administrative structures may influence TA needs. Seven of the nine NAMRS Pilot states were located within the SUA. The models for APS program structure include state program administrators and either contracted staff or state employees conducting regional and local APS work. APS programs in SUAs have access or may gain access to client services information if the APS client is also a person receiving other services (e.g., funded by the Older American's Act [OAA], Medicaid, Social Services Block Grant [SSBG], etc.) administered through the SUA. However, the APS programs may not have access to client services data from a sister agency, such as mental health. APS programs in SUAs were more familiar with the OAA definitions of services and taxonomies. All pilot participants were familiar with SSBG definitions of services and taxonomies. The two pilot states not housed in an SUA had some established contacts with the SUA staff.

Action: Some further work may be needed to establish if there are specific TA topics that would be more useful to APS programs located in SUAs and others more useful to those located in state human services agencies. Certainly future TA efforts could create cohorts of states and provide more focused TA based on specific APS administrative structures.

View full report

Preview
Download

"NAMRSpilot-V1.pdf" (pdf, 2.72Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®