icesat2
The ICESat-2 Python API icesat2.py
is used to access the services provided by the sliderule-icesat2 plugin for SlideRule. From Python, the module can be imported via:
from sliderule import icesat2
init
- sliderule.icesat2.init(url, verbose=False, max_resources=300, loglevel=50, organization='sliderule')[source]
Initializes the underlying SlideRule module. Must be called before other ICESat-2 API calls. This function is the same as calling the sliderule module functions: set_url, set_verbose, set_max_errors, along with the local set_max_resources function.
- Parameters
url (str) – the IP address or hostname of the SlideRule service (note, there is a special case where the url is provided as a list of strings instead of just a string; when a list is provided, the client hardcodes the set of servers that are used to process requests to the exact set provided; this is used for testing and for local installations and can be ignored by most users)
verbose (bool) – whether or not user level log messages received from SlideRule generate a Python log message (see sliderule.set_verbose)
max_resources (int) – the maximum number of resources that are allowed to be processed in a single request
loglevel (int) – minimum severity of log message to output
organization (str) – SlideRule provisioning system organization user belongs to (see sliderule.authenticate for details)
Examples
>>> from sliderule import icesat2 >>> icesat2.init("my-sliderule-service.my-company.com", True)
set_max_resources
- sliderule.icesat2.set_max_resources(max_resources)[source]
Sets the maximum allowed number of resources to be processed in one request. This is mainly provided as a sanity check for the user.
- Parameters
max_resources (int) – the maximum number of resources that are allowed to be processed in a single request
Examples
>>> from sliderule import icesat2 >>> icesat2.set_max_resources(1000)
cmr
- sliderule.icesat2.cmr(**kwargs)[source]
Query the NASA Common Metadata Repository (CMR) for a list of data within temporal and spatial parameters
- Parameters
polygon (list) – either a single list of longitude,latitude in counter-clockwise order with first and last point matching, defining region of interest (see polygons), or a list of such lists when the region includes more than one polygon
time_start (str) – starting time for query in format
<year>-<month>-<day>T<hour>:<minute>:<second>Z
time_end (str) – ending time for query in format
<year>-<month>-<day>T<hour>:<minute>:<second>Z
version (str) – dataset version as found in the NASA CMR Directory
short_name (str) –
dataset short name as defined in the NASA CMR Directory
- Returns
files (granules) for the dataset fitting the spatial and temporal parameters
- Return type
list
Examples
>>> from sliderule import icesat2 >>> region = [ {"lon": -108.3435200747503, "lat": 38.89102961045247}, ... {"lon": -107.7677425431139, "lat": 38.90611184543033}, ... {"lon": -107.7818591266989, "lat": 39.26613714985466}, ... {"lon": -108.3605610678553, "lat": 39.25086131372244}, ... {"lon": -108.3435200747503, "lat": 38.89102961045247} ] >>> granules = icesat2.cmr(polygon=region) >>> granules ['ATL03_20181017222812_02950102_003_01.h5', 'ATL03_20181110092841_06530106_003_01.h5', ... 'ATL03_20201111102237_07370902_003_01.h5']
atl06
- sliderule.icesat2.atl06(parm, resource, asset='nsidc-s3')[source]
Performs ATL06-SR processing on ATL03 data and returns gridded elevations
- Parameters
parms (dict) – parameters used to configure ATL06-SR algorithm processing (see Parameters)
resource (str) – ATL03 HDF5 filename
asset (str) – data source asset (see Assets)
- Returns
gridded elevations (see Elevations)
- Return type
GeoDataFrame
atl06p
- sliderule.icesat2.atl06p(parm, asset='nsidc-s3', version='005', callbacks={}, resources=None)[source]
Performs ATL06-SR processing in parallel on ATL03 data and returns gridded elevations. This function expects that the parm argument includes a polygon which is used to fetch all available resources from the CMR system automatically. If resources is specified then any polygon or resource filtering options supplied in parm are ignored.
Warning
It is often the case that the list of resources (i.e. granules) returned by the CMR system includes granules that come close, but do not actually intersect the region of interest. This is due to geolocation margin added to all CMR ICESat-2 resources in order to account for the spacecraft off-pointing. The consequence is that SlideRule will return no data for some of the resources and issue a warning statement to that effect; this can be ignored and indicates no issue with the data processing.
- Parameters
parms (dict) –
parameters used to configure ATL06-SR algorithm processing (see Parameters)
asset (str) –
data source asset (see Assets)
version (str) – the version of the ATL03 data to use for processing
callbacks (dictionary) – a callback function that is called for each result record
resources (list) – a list of granules to process (e.g. [“ATL03_20181019065445_03150111_004_01.h5”, …])
- Returns
gridded elevations (see Elevations)
- Return type
GeoDataFrame
Examples
>>> from sliderule import icesat2 >>> icesat2.init("slideruleearth.io", True) >>> parms = { "cnf": 4, "ats": 20.0, "cnt": 10, "len": 40.0, "res": 20.0, "maxi": 1 } >>> resources = ["ATL03_20181019065445_03150111_003_01.h5"] >>> atl03_asset = "atlas-local" >>> rsps = icesat2.atl06p(parms, asset=atl03_asset, resources=resources) >>> rsps dh_fit_dx w_surface_window_final ... time geometry 0 0.000042 61.157661 ... 2018-10-19 06:54:46.104937 POINT (-63.82088 -79.00266) 1 0.002019 61.157683 ... 2018-10-19 06:54:46.467038 POINT (-63.82591 -79.00247) 2 0.001783 61.157678 ... 2018-10-19 06:54:46.107756 POINT (-63.82106 -79.00283) 3 0.000969 61.157666 ... 2018-10-19 06:54:46.469867 POINT (-63.82610 -79.00264) 4 -0.000801 61.157665 ... 2018-10-19 06:54:46.110574 POINT (-63.82124 -79.00301) ... ... ... ... ... ... 622407 -0.000970 61.157666 ... 2018-10-19 07:00:29.606632 POINT (135.57522 -78.98983) 622408 0.004620 61.157775 ... 2018-10-19 07:00:29.250312 POINT (135.57052 -78.98983) 622409 -0.001366 61.157671 ... 2018-10-19 07:00:29.609435 POINT (135.57504 -78.98966) 622410 -0.004041 61.157748 ... 2018-10-19 07:00:29.253123 POINT (135.57034 -78.98966) 622411 -0.000482 61.157663 ... 2018-10-19 07:00:29.612238 POINT (135.57485 -78.98948)
[622412 rows x 16 columns]
atl03s
- sliderule.icesat2.atl03s(parm, resource, asset='nsidc-s3')[source]
Subsets ATL03 data given the polygon and time range provided and returns segments of photons
- Parameters
parms (dict) –
parameters used to configure ATL03 subsetting (see Parameters)
resource (str) – ATL03 HDF5 filename
asset (str) –
data source asset (see Assets)
- Returns
ATL03 extents (see Photon Segments)
- Return type
GeoDataFrame
atl03sp
- sliderule.icesat2.atl03sp(parm, asset='nsidc-s3', version='005', callbacks={}, resources=None)[source]
Performs ATL03 subsetting in parallel on ATL03 data and returns photon segment data. Unlike the atl03s function, this function does not take a resource as a parameter; instead it is expected that the parm argument includes a polygon which is used to fetch all available resources from the CMR system automatically.
Warning
Note, it is often the case that the list of resources (i.e. granules) returned by the CMR system includes granules that come close, but do not actually intersect the region of interest. This is due to geolocation margin added to all CMR ICESat-2 resources in order to account for the spacecraft off-pointing. The consequence is that SlideRule will return no data for some of the resources and issue a warning statement to that effect; this can be ignored and indicates no issue with the data processing.
- Parameters
parms (dict) –
parameters used to configure ATL03 subsetting (see Parameters)
asset (str) –
data source asset (see Assets)
version (str) – the version of the ATL03 data to return
callbacks (dictionary) – a callback function that is called for each result record
resources (list) – a list of granules to process (e.g. [“ATL03_20181019065445_03150111_004_01.h5”, …])
- Returns
ATL03 segments (see Photon Segments)
- Return type
GeoDataFrame
h5
- sliderule.icesat2.h5(dataset, resource, asset='nsidc-s3', datatype=3, col=0, startrow=0, numrows=- 1)[source]
Reads a dataset from an HDF5 file and returns the values of the dataset in a list
This function provides an easy way for locally run scripts to get direct access to HDF5 data stored in a cloud environment. But it should be noted that this method is not the most efficient way to access remote H5 data, as the data is accessed one dataset at a time. The
h5p
api is the preferred solution for reading multiple datasets.One of the difficulties in reading HDF5 data directly from a Python script is converting the format of the data as it is stored in HDF5 to a data format that is easy to use in Python. The compromise that this function takes is that it allows the user to supply the desired data type of the returned data via the datatype parameter, and the function will then return a numpy array of values with that data type.
The data type is supplied as a
sliderule.datatypes
enumeration:sliderule.datatypes["TEXT"]
: return the data as a string of unconverted bytessliderule.datatypes["INTEGER"]
: return the data as an array of integerssliderule.datatypes["REAL"]
: return the data as an array of double precision floating point numberssliderule.datatypes["DYNAMIC"]
: return the data in the numpy data type that is the closest match to the data as it is stored in the HDF5 file
- Parameters
dataset (str) – full path to dataset variable (e.g.
/gt1r/geolocation/segment_ph_cnt
)resource (str) – HDF5 filename
asset (str) –
data source asset (see Assets)
datatype (int) – the type of data the returned dataset list should be in (datasets that are naturally of a different type undergo a best effort conversion to the specified data type before being returned)
col (int) – the column to read from the dataset for a multi-dimensional dataset; if there are more than two dimensions, all remaining dimensions are flattened out when returned.
startrow (int) – the first row to start reading from in a multi-dimensional dataset (or starting element if there is only one dimension)
numrows (int) – the number of rows to read when reading from a multi-dimensional dataset (or number of elements if there is only one dimension); if ALL_ROWS selected, it will read from the startrow to the end of the dataset.
- Returns
dataset values
- Return type
numpy array
Examples
>>> segments = icesat2.h5("/gt1r/land_ice_segments/segment_id", resource, asset) >>> heights = icesat2.h5("/gt1r/land_ice_segments/h_li", resource, asset) >>> latitudes = icesat2.h5("/gt1r/land_ice_segments/latitude", resource, asset) >>> longitudes = icesat2.h5("/gt1r/land_ice_segments/longitude", resource, asset) >>> df = pd.DataFrame(data=list(zip(heights, latitudes, longitudes)), index=segments, columns=["h_mean", "latitude", "longitude"])
h5p
- sliderule.icesat2.h5p(datasets, resource, asset='nsidc-s3')[source]
Reads a list of datasets from an HDF5 file and returns the values of the dataset in a dictionary of lists.
This function is considerably faster than the
icesat2.h5
function in that it not only reads the datasets in parallel on the server side, but also shares a file context between the reads so that portions of the file that need to be read multiple times do not result in multiple requests to S3.For a full discussion of the data type conversion options, see h5.
- Parameters
datasets (dict) – list of full paths to dataset variable (e.g.
/gt1r/geolocation/segment_ph_cnt
); see below for additional parameters that can be added to each datasetresource (str) – HDF5 filename
asset (str) –
data source asset (see Assets)
- Returns
numpy arrays of dataset values, where the keys are the dataset names
The datasets dictionary can optionally contain the following elements per entry:
”valtype” (int): the type of data the returned dataset list should be in (datasets that are naturally of a different type undergo a best effort conversion to the specified data type before being returned)
”col” (int): the column to read from the dataset for a multi-dimensional dataset; if there are more than two dimensions, all remaining dimensions are flattened out when returned.
”startrow” (int): the first row to start reading from in a multi-dimensional dataset (or starting element if there is only one dimension)
”numrows” (int): the number of rows to read when reading from a multi-dimensional dataset (or number of elements if there is only one dimension); if ALL_ROWS selected, it will read from the startrow to the end of the dataset.
- Return type
dict
Examples
>>> from sliderule import icesat2 >>> icesat2.init(["127.0.0.1"], False) >>> datasets = [ ... {"dataset": "/gt1l/land_ice_segments/h_li", "numrows": 5}, ... {"dataset": "/gt1r/land_ice_segments/h_li", "numrows": 5}, ... {"dataset": "/gt2l/land_ice_segments/h_li", "numrows": 5}, ... {"dataset": "/gt2r/land_ice_segments/h_li", "numrows": 5}, ... {"dataset": "/gt3l/land_ice_segments/h_li", "numrows": 5}, ... {"dataset": "/gt3r/land_ice_segments/h_li", "numrows": 5} ... ] >>> rsps = icesat2.h5p(datasets, "ATL06_20181019065445_03150111_003_01.h5", "atlas-local") >>> print(rsps) {'/gt2r/land_ice_segments/h_li': array([45.3146427 , 45.27640582, 45.23608027, 45.21131015, 45.15692304]), '/gt2l/land_ice_segments/h_li': array([45.35118977, 45.33535027, 45.27195617, 45.21816889, 45.18534204]), '/gt1l/land_ice_segments/h_li': array([45.68811156, 45.71368944, 45.74234326, 45.74614113, 45.79866465]), '/gt3l/land_ice_segments/h_li': array([45.29602321, 45.34764226, 45.31430979, 45.31471701, 45.30034622]), '/gt1r/land_ice_segments/h_li': array([45.72632446, 45.76512574, 45.76337375, 45.77102473, 45.81307948]), '/gt3r/land_ice_segments/h_li': array([45.14954134, 45.18970635, 45.16637644, 45.15235916, 45.17135806])}
toregion
- sliderule.icesat2.toregion(source, tolerance=0.0, cellsize=0.01, n_clusters=1)[source]
Convert a GeoJSON representation of a set of geospatial regions into a list of lat,lon coordinates and raster image recognized by SlideRule
- Parameters
filename (str) – file name of GeoJSON formatted regions of interest, file must have name with the .geojson suffix file name of ESRI Shapefile formatted regions of interest, file must have name with the .shp suffix
tolerance (float) – tolerance used to simplify complex shapes so that the number of points is less than the limit (a tolerance of 0.001 typically works for most complex shapes)
cellsize (float) – size of pixel in degrees used to create the raster image of the polygon
clusters (int) – number of clusters of polygons to create when breaking up the request to CMR
- Returns
a list of longitudes and latitudes containing the region of interest that can be used for the poly and raster parameters in a processing request to SlideRule.
region = {“poly”: [{“lat”: <lat1>, “lon”: <lon1>, … }], “clusters”: [{“lat”: <lat1>, “lon”: <lon1>, … }, {“lat”: <lat1>, “lon”: <lon1>, … }, …], “raster”: {“data”: <geojson file as string>, “length”: <length of geojson file>, “cellsize”: <parameter cellsize>}}
- Return type
dict
Examples
>>> from sliderule import icesat2 >>> # Region of Interest # >>> region_filename = sys.argv[1] >>> region = icesat2.toregion(region_filename) >>> # Configure SlideRule # >>> icesat2.init("slideruleearth.io", False) >>> # Build ATL06 Request # >>> parms = { ... "poly": region["poly"], ... "srt": icesat2.SRT_LAND, ... "cnf": icesat2.CNF_SURFACE_HIGH, ... "ats": 10.0, ... "cnt": 10, ... "len": 40.0, ... "res": 20.0, ... "maxi": 1 ... } >>> # Get ATL06 Elevations >>> atl06 = icesat2.atl06p(parms)