The NAMRS Pilot was designed and developed to test the capacity to collect administrative data from state APS agencies, which in turn can be transformed into information. This information will provide knowledge to the field on the extent and underlying features of abuse, neglect, mistreatment, and exploitation of vulnerable adults. While the NAMRS Pilot design and features met the pilot goals and demonstrated the necessary capabilities, a number of technical refinements and enhancements are warranted prior to the full implementation of NAMRS. These suggested enhancements are informed by what was learned during the pilot and by the anticipated new features and requirements needed to support the full implementation. This chapter identifies several areas for enhancement when implementing the system nationwide.
We discuss the following enhancement topics:
- Enrich Data Quality Through Validation
- Ensure Flexibility of the Database
- Enhance Performance and Scaling
- Improve User Experience (UX)
- Strengthen System Security
- Continue to Use a Proven Development Methodology
The enhancement items under each topic are ranked in terms of our sense of priority in meeting the goals and objectives of full implementation of NAMRS, as follows:
Must have for the system to function effectively and meet ACL objectives.
Should have to encourage states to participate and reduce their burden.
Could have, if resources permit, to further improve the UX.
Enrich Data Quality by Incorporating More Validation Rules
The NAMRS Pilot processed data submissions from states to assure that the data format and values submitted were consistent with the NAMRS Pilot requirements. The NAMRS Pilot achieved this by the process of data validation. Data were validated at the time of data entry for the Agency and Key Indicators Components. For the Case Component XML file, the validation was done in two steps. First, the XML file was validated against a schema file, which checked the structure, data type, and data format. Next, an automated validation process applied a number of rules to validate the data elements.
Even though the NAMRS Pilot validation process was broad and detailed, it was still not comprehensive in checking all the possible scenarios that might affect the data quality. More data validation rules must be applied across the following rule categories:
Within a Case--The relationships between the data elements within a case must be checked. For example, clients in a case with a maltreatment of self-neglect must not have perpetrators.
Across Cases--The data elements repeating across cases must be consistent. For example, if a client appears in more than one case, the demographics information must be consistent.
Distribution of Values--The reporting of data elements must match minimum requirements in terms of counts to enforce comprehensiveness. For example, more than 90 percent of the clients must have the client age reported.
Unknown and Blank Values--Clearly identify the distinction between unknown and blank values. For example, if a state collects data for a NAMRS data element, the values must be reported. Unknown must be reported when data are not available for some cases. If a state does not collect any data for a NAMRS data element, then blanks must be reported for all cases.
Missing Data Elements--States must provide dummy data for mandatory NAMRS data elements that they do not collect. For example, some states do not assign IDs to identify perpetrators uniquely. In such cases, the states will be asked to create distinct IDs for each perpetrator.
The NAMRS Pilot produced a number of reports with validation results. While the reports were detailed, they will need to include more information to analyze the completeness and comprehensiveness of the data.
The validation reports must be reviewed by ACL to identify detailed analyses that must be incorporated. These validation reports will support the acceptance criteria that drive the decision making process of accepting data.
The NAMRS Pilot Case Component consisted of 58 data elements, and hence it requires a number of data validation rules.
To achieve completeness of validation and to enhance the performance of the application of the validation rules, the XML structure should be enhanced to include metadata information. The metadata can have information about which data elements are reported by a state, what code values are mapped to the NAMRS values, and what elements contain dummy data. States provided this information in the mapping forms during the pilot. Having the metadata embedded in the XML file or accessible to the file will help with quicker validation as extensive validation rules for data elements identified as not reported might be skipped.
Instead of including metadata as mentioned above, the mapping forms could be integrated into the NAMRS database. The NAMRS Pilot developed mapping forms to help states determine which data elements they had in their systems and which data values could be mapped to NAMRS Pilot data values. The NAMRS Pilot collected mapping form information as paper forms. Reviewing the data against the mapping form was a useful step to identify data quality issues. While some states were enthusiastic about using the mapping forms, some states thought they appeared burdensome and not user friendly. Online screens to collect mapping form information could be added to NAMRS, and the information could be used to validate against the data submitted by the states.
Ensure Flexibility of the Database
The NAMRS Pilot Database stored the submitted data as they were entered and saved via data entry screens or when uploaded as a file. The NAMRS Pilot database also stored the validation rules applied on the Case Component. The NAMRS Loader application read each validation rule from the database and checked the data and the time of loading the data. Adding new validation or modifying the existing rules could be achieved by updating the list of validation rules in the NAMRS Pilot database.
The full implementation of NAMRS must leverage the flexible validation processing based on the accommodating database to implement more validation rules.
As mentioned previously, the NAMRS Pilot was built to collect data only for the pilot data period of FFY 2015.
To accommodate data collection for multiple years, the database must be updated to store and process multiyear, multistate data. This change can be implemented in the data model by adding a new entity called the year/state at the top of the entity hierarchy. The year/state entity will be a combination of a specific year and state. For example, 2015-Pennsylvania. As per the data model in Figure 7.1, NAMRS Pilot Data Model with year/state entity, for each combination of the year and state there will be Agency, Key Indicators, and Case Components.
|FIGURE 7.1. NAMRS Pilot Data Model with Year/State Entity|
The NAMRS Pilot Data Warehouse provided an easy-to-use database that could be queried using any statistical analysis application, data visualization application, or SQL query application. The NAMRS Pilot Database only contained the Case Component data.
The Agency and Key Indicators Components data must be included in the Data Warehouse of the full implementation of NAMRS. This will enable the analysts to query and analyze data for all the components.
Loading other external data related to APS into the Data Warehouse of the full implementation of NAMRS should be considered. External data includes population data, Social Security data, and other datasets. This will enable analysts to link various datasets and compute rates, estimates, and stratified sampling.
The NAMRS Pilot Database was based on a flexible data model that encapsulated the complex relationships between entities. An entity is some unit of data that can be classified and have stated relationships to other entities. For example, client is an entity, which consists of information about the client characteristics. A client entity can have relationships with the perpetrator entity.
New entities and data elements could be added and obsolete ones removed from the data model and subsequently either added to or removed from the physical database without major re-engineering efforts. Changes to the data model would require changes to: (1) the database; (2) the forms on the website or the software that imports data from the case XML files into the database; (3) the validation functions; and (4) any reports or extracts that use the affected data.
Enhance Performance and Scaling by Moving to Platform as a Service
The NAMRS Pilot was designed to collect data from a subset of states, while NAMRS will collect data from 50 states, District of Columbia, Puerto Rico, Guam, Northern Marianas Islands, Virgin Islands, and American Samoa. This implies that the NAMRS Pilot has to scale to accommodate an increased number of users, a higher volume of data, and improved data processing capabilities without forgoing desired levels of system performance.
As explained in an earlier chapter, the NAMRS Pilot Website was built and hosted as an Microsoft Azure Website based on the PaaS model, which provides the hardware architecture and software framework needed to put the application into service, without having to manage and upkeep all the required resources. This means that the existing website is already scalable by assigning more resources (i.e., system memory, storage space, etc.) to the service through the cloud's configuration. Anticipating various modifications during the development cycle of the NAMRS Pilot, the loader and databases are hosted on VMs in Azure based on the IaaS model.
While they can be scaled by assigning more resources to the VMs through the cloud's configuration, the loader and the databases must be moved to a PaaS model such as Azure SQL for the full implementation of NAMRS to leverage the improved performance and system manageability.
Figure 7.2, NAMRS Pilot Using PaaS, provides a diagram of the NAMRS Pilot with all the components built on the Microsoft Azure PaaS model. A consistent cloud infrastructure will provide a number of improvements in terms of system performance, compatibility among various components, software code writing, manageability, and maintainability. It will greatly improve the speed, agility, and flexibility of the development process, as resources are not spent for upfront configuration and continuous maintenance of the infrastructure environment.
|FIGURE 7.2. NAMRS Pilot Using PaaS|
Improve User Experience
The NAMRS Pilot Website provided a simple way for users to access the NAMRS Pilot. It was designed to enable the users to access the features to submit data and retrieve output information specific to the pilot. While it served the purpose of the pilot, the website design must be enhanced to improve the ease of use and efficiency of performing tasks.
For example, in the NAMRS Pilot state users had to navigate to the individual data component screens to see the current data status. The UX must be enhanced by allowing each state to have its own secure private working area that will not be accessed by other states. The home page of the private working areas must list the current status of data collection for the state. This will provide useful information to the user, after the user logs in, and enable quick navigation to the appropriate data collection screens. The private working areas must also include a data repository for the states to store documents pertaining to NAMRS and share with their assigned liaison.
The NAMRS Pilot was built to collect data only for the pilot data period of FFY 2015.
To accommodate data collection for multiple years, the website must be updated to allow users to select the appropriate data period and then submit data. This enhancement will allow states to submit data for the current or most recent data period as well as resubmit data for the previous years, if necessary.
In addition, implementing features to disable data collection functionality after the data collection window closes will avoid losing track of data submitted by states outside of the window.
The NAMRS Pilot was built using a software framework that enforces Section 508 accessibility standards compliance. The website was also tested using available open-source test tools for Section 508 compliance. However, framework tools do not guarantee full compliance. Hence, for the full implementation, thorough accessibility testing using manual and automated processes must be carried out. This will impact the development schedule as more time will be necessary for testing.
The full implementation of NAMRS is expected to request the states to submit Agency and Case Component data or Agency and Key Indicators Component data for each data period. Most of the data submitted in the Agency Component may not change much over the years as these data pertain to state APS policies and practices.
New functionality to clone the Agency Component data for easier data entry and submission across years could be very useful to the state users. Once cloned, state users would have to modify only the appropriate data elements and submit for the new data period.
Strengthen System Security
The NAMRS Pilot enforced security for all components at various levels. It also leveraged the advanced security features provided by the Microsoft Azure Cloud platform. Similar levels of security should be considered for the full implementation.
ACL must be cognizant of all security requirements and make regular security assessments for continuous verification.
All IDs in the data submitted by the states must encrypt to ensure data security. Encryption will add another layer of data security so that individuals cannot be tracked back into the source of the data. Consistent data encryption practice by states, prior to submitting data, will allow tracking of individuals across data periods.
All privileged accounts must be identified and two-factor authentication must be enforced. Privileged accounts are usually the system administrators who are able to add or modify users, make changes to the database and website, initiate backup and restore processes, and install and configure hardware or software.
The common approach for authenticating privilege accounts has been username and password (single factor authentication). The two-factor approach must include another factor for authentication such as a personal identity verification card or another method to obtain something only the privileged user has. The other methods could be a code sent to their phone or email that must be supplied to continue access to administrative functions.
ACL must have internal privileged accounts to gain complete control of the system environment when necessary. Single sign-on features should also be considered by integrating with ACL's user directory either on-premises or in the cloud.
Continue to Use a Proven Development Methodology
The NAMRS Pilot development process used the Agile Scrum methodology for software development. The Agile methodology breaks up development into several sprints where each sprint generally last anywhere from 1-4 weeks. The NAMRS Pilot development cycle consisted of five sprints with each sprint adding system functionality incrementally. Each sprint consisted of development and testing, and resulted in a production-ready version of the system. After each sprint, the system went through a sprint review process where the functionality was reviewed to obtain feedback on the usability of the system and workflow for submitting, reviewing, and accepting data. Following the sprint review, the new functionality was deployed to the production environment. The Agile sprint methodology used for the development of the NAMRS Pilot was very effective in terms of prioritizing features, improving quality by breaking down development into manageable units, and engaging all stakeholders in the decision making process.
Adopting the scrum methodology will enable functional and technical enhancements to be quickly coded, tested, and released as issues can be identified earlier and rectified. This approach will be effective as a system development process for the migration from the NAMRS Pilot to NAMRS, as there will be minimum changes in the system components, environment, and requirements.