Getting Started

This documentation is intended to explain how to use SlideRule, a C++/Lua framework for on-demand data processing, and its accompanying Python client. SlideRule is a science data processing service at icesat2sliderule.org that responds to REST API calls to process and return science results. This software was developed to support science applications for NASA’s Ice Cloud and land Elevation Satellite-2 (ICESat-2), but also has the goal of demonstrating a new paradigm for providing science data products to researchers.

While SlideRule can be accessed by any http client (e.g. curl) by making GET and POST requests to the SlideRule service, the python packages in this repository provide higher level access by hiding the GET and POST requests inside python function calls that accept and return basic python variable types (e.g. dictionaries, lists, numbers), and GeoDataFrames.

The SlideRule-Python repository includes example Jupyter Notebooks for those who want to jump right in.

Basic Usage

The SlideRule service provides a number of services which allow a user to process ICESat-2 and HDF5 data. For example, the icesat2.atl06p python function makes a request to the atl06 service and returns calculated segment elevations from ATL03 data within a geospatial region.

# import (1)
import pandas as pd
from sliderule import icesat2

# region of interest (2)
grand_mesa = [ {"lon": -108.3435200747503, "lat": 38.89102961045247},
               {"lon": -107.7677425431139, "lat": 38.90611184543033},
               {"lon": -107.7818591266989, "lat": 39.26613714985466},
               {"lon": -108.3605610678553, "lat": 39.25086131372244},
               {"lon": -108.3435200747503, "lat": 38.89102961045247} ]

# initialize (3)
icesat2.init("icesat2sliderule.org", verbose=True, max_errors=3)

# processing parameters (4)
parms = {
    "poly": grand_mesa,
    "srt": icesat2.SRT_LAND,
    "cnf": icesat2.CNF_SURFACE_HIGH,
    "len": 40.0,
    "res": 20.0,
    "maxi": 1
}

# make request (5)
gdf = icesat2.atl06p(parms)

This code snippet performs the following functions with respect to SlideRule:

  1. Imports the icesat2 module from the SlideRule Python packages

  2. Defines a polygon that captures the Grand Mesa region

  3. Initializes the icesat2 module with the address of the SlideRule server and other configuration parameters

  4. Builds an atl06 API request structure that specifies the region of interest and processing parameters

  5. Issues the request to the SlideRule service and collects the results

  6. Builds a pandas GeoDataFrame from the results and returns it to the calling program for further analysis

Python API Reference

SlideRule ICESat-2 Python API:

function

description

init

Initialize the icesat2 package with the URL to the SlideRule service

set_max_resources

Sets maximum allowed resources to process in one request

cmr

Query the NASA Common Metadata Repository (CMR) for a list of data within temporal and spatial parameters

atl06

Perform ATL06-SR processing on a single ATL03 granule and return gridded elevations

atl06p

Perform ATL06-SR processing in parallel on ATL03 data and return gridded elevations

atl03s

Subset a single ATL03 granule and return the photon data

atl03sp

Subset ATL03 granuels in parallel and return the photon data

h5

Read a dataset from an HDF5 file and return the values of the dataset

h5p

Read a list of datasets in parallel from an HDF5 file and return the values of the datasets

toregion

Convert a GeoJSON formatted polygon into the format accepted by SlideRule

get_version

Get version information for the running servers and client

SlideRule Python API:

function

description

source

Perform API call to SlideRule service

set_url

Configure the URL of the SlideRule service

update_available_servers

Cause local client to update the list of available SlideRule servers

set_verbose

Configure the verbose setting in the SlideRule client

set_max_errors

Configure the maximum number of errors a given server can return before being removed from the list

set_rqst_timeout

Set TCP/IP request timeouts for connecting and reading data

gps2utc

Convert a GPS based time returned from SlideRule into a UTC time

get_definition

Get the underlying format specification of a field in a return record

icepyx Python API:

function

description

atl06p

Uses icepyx region to perform ATL06-SR processing in parallel on ATL03 data and return gridded elevations

atl03sp

Uses icepyx region to subset ATL03 granuels in parallel and return the photon data