Development of a National Adult Protective Services Data System: Namrs Pilot Final Report (volume 2). 7. Namrs Pilot Storage

09/25/2015

The NAMRS Pilot utilized shared storage space where multiple components could access a file. In the Microsoft Azure Cloud, the mechanism for doing this is called Azure Storage.

Function

There were three types of storage mechanisms provided in Azure:

  1. Azure Blob Storage--provided a way to store files.

  2. Azure Table Storage--provided a way to store (unstructured) table rows.

  3. Azure Queues--provided a common message queue--a way to share messages.

Azure Storage is a very economical, infinitely scalable storage service. You only pay for what you use, and there is (practically) no limit to the amount of data that can be stored.

The NAMRS Pilot only used Azure Blob Storage for storing and sharing files. Azure Blob Storage uses the following hierarchy:

  • Storage Account--There must be at least one Storage Accounts.

  • Container--Each storage account must have at least one Container. A container is analogous to a folder (or directory) on a disk.

  • Files--A Container can have zero or more files in it.

The NAMRS Pilot used a single storage account that had the following containers in it:

  • CaseFiles--This container had all the case files. This included the actual XML/zip files that were uploaded by the state users through the website. Once the XML file was processed by the NAMRS Case XML Loader, the following files were added for each case XML file: (1) a text file that had all the error/warning messages produced for the XML file; (2) a second version of the XML file without any invalid data; (3) a PDF file containing the frequency counts which was available on the website; and (4) a PDF file containing the summary counts which was available on the website.

  • ResourceFiles--This container had all the resource files that were uploaded by technical users. These files were available to users on the website.

  • DbBackup--This container had all the database backups created by SQL Server maintenance plans.

  • VHDs--This container had all the "virtual hard disk" files that were used by the Azure VMs.

Azure Storage is a service provided by the Azure Cloud. It is a "geographically redundant," meaning that files are not only backed up, but backed up across different regions. This means that if an entire data center was to ever go offline (or possibly be destroyed), the files will remain intact. (For the NAMRS Pilot, the primary region was in the Eastern United States region, and the secondary region was in Western United States.)

Access

The following components accessed NAMRS Pilot Storage:

  • NAMRS Pilot Website--The website stored Case Component XML files that were uploaded by state users, as well as resource files uploaded by technical/administrative users. It also read PDF and text files that were created by the Case Component XML Loader, as well as resource files.

  • NAMRS Pilot Case Component Loader--The loader retrieved the Case XML files, and stored PDF and text files.

  • NAMRS Pilot Database (not shown on diagram)--The database saved its backup files.

  • NAMRS Pilot Data Warehouse (not shown on diagram)--The data warehouse saved its backup files.

The NAMRS Pilot Storage is a service provided by the Azure Cloud. It did not access any components in the NAMRS Pilot.

Security

Security for NAMRS Pilot Storage was handled by the Microsoft Azure Cloud.

Azure Blob Storage is an application programming interface (API) and it is available from the Internet. It is available by HTTPS. There are many positive aspects to this because Azure Blob Storage can be used for many different purposes. The files were not available on the Internet in the NAMRS Pilot.

Access to Azure Blob Storage API is through an URL and a secret access key. To use the API, one needs the access key. Access keys for Azure Storage are almost 90 characters long, so it was highly unlikely that anyone could guess this key.

All access to the Azure Blob Storage API uses SSL encryption over HTTPS. So, data was always encrypted as it moved between the web server (and the Case XML Loader) and Azure Storage.

View full report

Preview
Download

"NAMRSpilot-V2.pdf" (pdf, 1.83Mb)

Note: Documents in PDF format require the Adobe Acrobat Reader®. If you experience problems with PDF documents, please download the latest version of the Reader®