Spatial Data and the Cloud
Background
We have previously created a SpatialData content-type in Plone for the LCC community. This content-type features a metadata schema that was developed in coordination with resource managers and GIS specialists from the community. It is based around the approach of storing large GIS data assets in the cloud, for durable cost-effective storage and retrieval. The initial implementation of this content-type makes no assumptions about where the GIS data will live, only that each spatial data resource will be accessible via a single, unique URL. As we have further developed this approach, the Appalachian LCC community has gained consensus about storing their GIS files in Amazon’s S3 cloud. When files are loaded up into repositories in Amazon’s cloud, an XML index directory is created for each repository returning the contents of that repository in a simple XML wrapper. These XML indexes are marginally human-readable, but the fact that they are delivered in XML lends them to integration with our applications. We have been developing a javascript-based tool which parses these XML repository indexes and enables autocomplete for filling in the URL field of the SpatialData content-type. Current requirements We understand that it is desirable to improve the process by which spatial data files in the LCC’s S3 cloud are cataloged into the CMS. Once these spatial data files have listings in the CMS, additional metadata can be collected on them, and this metadata can be used for use cases including searching, browsing, and annotations. At the same time, specific LCC stakeholders need to share and archive certain spatial data files in a way that is private or semi-private and that observes a granular and nuanced security and permissions model.
Proposed approach
Based on these need we believe it will be beneficial to have multiple S3 repositories, each with their own security and access policies. We believe it will be beneficial to create a SpatialRepository content-type to support this integration. One instance of the SpatialRepository content-type will be created for each S3 repository. These SpatialRepository content-items can then be shared so as to be visible to only the stakeholders who should have access to them. The SpatialRepository content-type will have a title field and a URL field which points to the top of the S3 repository, enabling us to pick up and parse the appropriate XML index document. We then modify the SpatialData content-type, adding a field which can reference on of the Spatial Repositories. When a stakeholder goes to enter a new SpatialData item in the site, they see a list of possible repositories. Due to Plone’s permissions model this contributor will only see the repositories which they should have access to. After selecting the repository, the appropriate XML index file is parsed, and URL autocomplete is rendered based upon the contents of that repository. As both the SpatialData and SpatialRepository content- types are permissions-aware, we are able to create and maintain the appropriate visibility for both.
