Using Mapbox and GDAL to Visualize Trends in

Ocean Phytoplankton from NASA Earth Observations Satellite Images

Presented at FOSS4G 2017 Boston, August 16, 2017

http://RobLabs.com

http://Oceaneos.org

---

Command Line Cartography

eπ Maps — Offline Maps for Mobile

iOS

---

Android

ePi Rational, Inc.

Topics

• Maps tell stories
• Visualizing NASA Earth Observations Satellite Images
• Image Compression with WebP

Oceaneos Marine Research Foundation

"Adding nutrients to a depleted marine ecosystem.""

Icelandic Volcano

NASA Earth Observations (NEO)

"One of the best places to study Earth is from space.""

"NASA satellites continually orbit the globe, collecting information about Earth’s ocean, atmosphere, and land surfaces.""

"Satellites can even monitor the activity of life forms, such as phytoplankton, from their remote vantage points.""

"NEO is part of the EOS Project Science Office located at NASA Goddard Space Flight Center.""

NEO

NEO Data Available

• JPEG, PNG
• GeoTIFF
  • 8-bit index (perfect for color LUT)
  • 32-bit floating point (perfect for computations)
• Pixels on the scale of 1.0° to 0.1°
• Monthly data since June 2002

  • 180 months

• 8-day data since July 2002

  • 650 8-day periods

Architecture - Use gdal2tiles.py to Tile 0.1° GeoTIFFs

gdal_translate -of vrt file.tif out.vrt

gdal2tilesp.py -z 0-6 -f PNG out.vrt out

source — GDAL2Tiles http://wiki.osgeo.org/wiki/GDAL2Tiles_SoC_2007

Geo-Imaging pipeline

Simply

• Download >800 images from NASA
• preprocess
• Apply Color Lookup table
• GDAL2Tiles
• Upload mbtiles to Mapbox.com

Image Healing

G'MIC inpaint

• Rejected my own work — but could make better use of G'MIC
• We used an imaging healing method in Photoshop

gmic kindlmann-MY1DMM_CHLORA_2016-01.tif -to_colormode 3 \
  bathy.3600x1800.threshold.png \
  -blend xor \
  --select_color[0] 0,0,0,0 \
  -inpaint[0] [1],0 \
  -o result.png

http://gmic.eu

Color Lookup Tables (cLUT)

8-bit Index GeoTIFF

8-bit = 256 color levels

Color Lookup Table for 8-bit Index GeoTIFF

---

gdal_translate -of VRT MY1DMM_CHLORA_2016-07-01_rgb_3600x1800.TIFF clut.vrt

# edit the clut.vrt to make blue2red, or any other color LUT

# cp clut.vrt clut.blue2red.vrt
gdal_translate clut.blue2red.vrt MY1DMM_CHLORA_2016-07-01_rgb_3600x1800.clut.blue2red.TIFF

Color Advice

http://KennethMoreland.com/color-advice

gdal2tiles.py

Overlay over Google Mapbox

• Preferred no text nor borders

Chlorophyll over Mapbox Satellite

Done once ... now scale

Upload a MBTile to Mapbox.com

• Initial 5.0 GB on Mapbox.com
• With PNG Pipeline, hit the 5.0 GB limit after 43 months of data

Image Compression

GDAL drivers to the rescue

Solution WebP

• Use the WebP driver in GDAL to compress the data even smaller
• WebP tiles are ~1/3 the size of PNG tiles (~15 MB vs. ~45 MB per month)

WebP

• Limits on Mapbox.com - A single tile as an image file cannot be > 500 KB
  • http:// RobLabs/{z}/{x}/{y}.png
• WebP scales well for Mobile map solutions

gdal_translate -of vrt file.tif out.vrt

gdal2tilesp.py -z 0-6 -f WEBP out.vrt out

WEBP

https://github.com/roblabs/gdal2tilesp

Time Series

NEO Data Available

• JPEG, PNG
• GeoTIFF
  • 8-bit index (perfect for color LUT)
  • 32-bit floating point (perfect for computations)
• Pixels on the scale of 1.0° to 0.1°
• Monthly data since June 2002

  • 180 months

• 8-day data since July 2002

  • 650 8-day periods

Floating Point data

• Units mg/m^3
• 1.0° data shown

NumPy for Data Analysis

float = gdal.Open('MY1DMW_CHLORA_2016-09-05.FLOAT.TIFF')
f = numpy.array(float.GetRasterBand(1).ReadAsArray())
grid = numpy.meshgrid(f)

# Extract chlorophyll less than 1.0
t = numpy.less (grid , 1.0)
choice = numpy.logical_and(t , grid)
lowPlankton = numpy.extract(choice, grid)

Displaying the Time series

• Rejected my own work - Slider would not work with >160 months of satellite data

MetricsGraphics.js

Visualizing and laying out time-series data built on D3.js

• Next try Plot.ly

Plot.ly

D3.js charting for science

The Oceaneos Global Chlorophyll Map

http://Oceaneos.org/chlorophyllmap

Summary — Maps tell stories

• Scale
• Maps as a time series
• Image compression
• Visualization libraries for Javascript

Summary of Tools

• OSGEO — http://osgeo.org
  • gdal2tiles.py
  • gdal2tilesp.py — parallel processing, https://github.com/roblabs/gdal2tilesp
  • WebP driver — http://developers.google.com/speed/webp
    
• Mapbox
  • http://Mapbox.com for hosting of tilesets
  • Mapbox GL for Javascript, for displaying tilesets, https://mapbox.com/mapbox-gl-js
  • mb-util for packing mbtiles, https://github.com/mapbox/mbutil
  • mapbox CLI for uploading mbtiles, https://github.com/mapbox/mapbox-cli-py
    
• d3.js — http://d3js.org
  • MetricsGraphics.js — http://metricsgraphicsjs.org
  • Plot.ly — http://plot.ly
  • Final map was done in http://HighCharts.com

Text & Online Resources

• Chris Garrard, Geoprocessing With Python, https://manning.com/books/geoprocessing-with-python

• Python GDAL/OGR Cookbook, https://pcjericks.github.io/py-gdalogr-cookbook/

Thank You‼️

Questions⁇

Backup

Vector Approach

• Extract raw data from Floating Point GeoTIFF for 1° grid
• Apply color map
• Draw using Mapbox GL JS tools

Scaling WebP

• 7.5 minute US Forest Service Topo Maps (FSTopo)
• http://RobLabs.com/flag