The volumes, complexity, and diversity of oceanographic data grow rapidly. This highlights a new challenge, a large disparity between the accelerating growth of observational data and the lack of scale-ability of legacy analytical and interpretational workflows. At the same time, accelerating innovation in data technologies, cloud computing, and deep learning offers exciting opportunities to revolutionize legacy routines. This section reviews our ongoing efforts and new initiatives to advance data stewardship, sharing, interpretation, visualization, and analysis while providing the general public with reliable access to oceanographic data.
Partnerships
Rolling Deck to Repository (R2R) curates data collected by sensors embedded on research vessels, such as navigation, fluorometer, raw multibeam, thermosalinograph, conductivity, temperature, depth, and Acoustic Doppler Current Profiler (ADCP) data. By submitting data to R2R, SOI leverages a wide oceanographic data network. R2R links users to the meteorological and current velocity data that is sent to the Shipboard Automated Meteorological and Oceanography System (SAMOS) and the University of Hawai’i Data Acquisition System (UHDAS), and contributes to NOAA’s National Center for Environmental Information and DataOne.
Processed bathymetric, side scan sonar, and echosounder data, vehicle-collected data such as navigation, conductivity, temperature, and depth (CTD), oxygen, magnetometer and other robot-carried instruments are curated by Marine Geoscience Data System (MGDS). Our partnership with MGDS ensures that seafloor bathymetric data reaches more users and contributes to a variety of syntheses such as the Global Multi-Resolution Topography synthesis, Google Earth Basemap, and ESRI Basemap.
Cloud Services
This year, in collaboration with R2R, SOI tested the multibeam data quality assurance workflow in Google Cloud Platform (GCP) Compute Engine. “Google’s support for customizable virtual machines allowed R2R to port the MBQA code and supporting libraries without fundamentally changing the I/O and existing workflow,” said Dr. Suzanne Carbotte, Principal Investigator for R2R, about running the test. “Additionally, R2R realized a performance gain of 35% in runtimes with cloud environments and noted a clear advantage is that the CPU and IO intensive process can be run without competing for compute resources with other R2R tasks.”
SOI continues to support the development of the Open Vessel Data Management (OpenVDM) software and web service. It enables flexible viewing and management of ship’s sensor data in real-time and delayed modes. SOI’s support for the development of this “open-infrastructure” data management software led to its broad adoption by many research vessels, including R/V Helmer Hanssen, R/V Atlantic Explorer, R/V Endeavor and R/V Annie. Expansion of OpenVDM in 2018 will enable the management of data from deployable platforms such as AUVs and ROVs, including ROV SuBastian and ROV Yogi.
Video Annotation
Since 2016, Schmidt Ocean Institute has been supporting the development of Squidle+, an open source scientific image annotation software and web service (squidle.com.au). Squidle+ supports intuitive access, browsing, management, annotation, sharing, and interpretation of georeferenced marine imagery. The program is available continuously as a web service online, where it provides access to about 5 million oceanographic images.
A version of Squidle+ for shipboard image annotation and event logging was developed in 2017 to allow the scientists aboard Falkor to annotate images collected with ROV SuBastian. This application provided a collaborative environment where scientists can log ROV events and annotate the contents of scientific images using a naming convention of their choice. As Squidle+ continues, local annotations will be automatically synchronized with the cloud to allow scientists to access and continue their image and video analysis at any time from anywhere in the world.
In a related effort, SOI has supported the development of Ocean Video Lab (OVL, www.oceanvideolab.org), which provides a lightweight annotation interface for YouTube hosted marine videos. It coordinates the video stream with the platform’s geospatial data, and provides an easy to use annotation search tool to facilitate video analysis. In 2017, OVL is a responsive web application with features that can be called via Application Programming Interfaces (APIs). It provides links to data resources, supports annotation filtering, and allows the users to download geolocated observations in a variety of formats. OVL is now employed by NOAA’s Office of Ocean Exploration and Research to engage citizen scientists.
Data Visualization
Our first research expedition of 2017 hosted a computer scientist and visualization expert providing unique opportunities for data processing and holographic visualization. Ben Knorlein, from Brown University, was onboard Falkor to help scientists use Virtual Reality (VR) to see plankton in 3D space. The resulting immersive visualizations helped the scientists’ interpret how the phytoplankton are congregating in relation to each other.
The high performance computing (HPC) system aboard Falkor helped the team establish a pipeline for processing the holographic images so that they could test different settings, run data analysis in parallel, and test which settings resulted in the best data. The HPC system was also used to host Squidle+, train its new scientific users sailing onboard Falkor, and run MapTracker, an open source software developed to visualize the locations and track multiple simultaneously operating robotic platforms, monitor their status and performance, and interactively control and deconflict their activities.
Advancements in data engineering and software hold promise for dramatic future improvements in oceanographic research, ecosystem management, and conservation in a scalable, efficient, and transferrable manner. While maintaining its focus on the advancements of software and data technologies, SOI remains committed to maintaining state of the art cybersecurity for its infrastructure and collaborators and keeping its research program outcomes open to the public.