The Oceanum Datamesh
The Oceanum.io Datamesh is an advanced cloud service for interacting with environmental data. It facilitates the discovery, management, and utilisation of all types of environmental datasets. The Datamesh platform is Software As A Service (SaaS), and does not require any installation of software on client or data service systems.
The Datamesh provides core functionality to facilitate the building of environmental data workflows, pipelines and end-user applications:
Searchable catalog - Spatial, temporal and natural language searching of all datasource metadata on the datamesh.
Query capability - Filtering and subsetting datasources to extract only the required portions of large datasets.
Authentication - User and organisation based authorization to allow fine-grained control of datasource access including sharing with other users.
Performance optimisation - Partitioning, caching and lazy execution of operations on datasources allow rapid access and pipelining of massive datasets.
Format conversion - Datasources can be accessed in a wide range of community standard formats regardless of their underlying Datamesh storage formats.
Data access APIs - Community standard REST APIs to connect to datasources for data access or application integration.
Aggregation operators - Calculation of aggregated quantities (for example mean or sum) within the datamesh, next to the data.
Web-based User Interface - A UI allows interactive searching, exploring and management of datasources on the Datamesh.
The Oceanum.io Datamesh is not a standard data warehouse or data lake and has two key differences to many existing data platforms:
First-class support for data structures found in the environmental and earth sciences. This includes multi-dimensional datacubes that vary in space, time and other dimensions and Geographc Information System data such as raster and feature layers.
Data stored on existing data services is left where it is under the direct management of the data owner or provider. The Datamesh makes a connection to these third-party servcies and provides an access layer managing searchability, authentication and connectivity. A storage system is also provided for users wishing to store their data in the Datamesh cloud, but this is not a requirement to utilise the Datamesh.