Postprocessing Build Slope layer AWS Step Function

This step function postprocessing-build-slope-step-function is used to orchestrate the building of a slope map and uploading the results to a specified Geoserver. The function requires a succesfull DTM, run postprocessing-build-dem-step-function if that is not the case. Processing the slope map is expensive since DEM data is quite large and we need to perform some blurring to smooth out the high resolution DEMs. Therefore this step function leverages an EC2 to run the extensive computations. Slope mapping is essentially the first derivative of elevation. The DTM gets cracked up in tiles with similar dimensions to RGB tiles and the EC2 processes these tiles in parallel batches.

Step Functions

The step functions that are used within this step function:

• A PostprocessingStartECS step function that starts an EC2 within the ECS cluster based on set parameters.
• A PostprocessingRasterOptimization step function that generates the COG dataset for Geoserver.

Lambdas

The lambdas orchestrated by the Step function are the following:

• A UtilityGetPrefixFunction to generate the prefix used to find the input files and where data will be stored in S3.
• A UtilityTaskControllerForTiler function to create the payload jsons for parallel processing inside EC2s.
• A PostprocessingBuildPropertiesFunction function to build properties files necessary for Geoserver coverage configuration.

./datastore.properties  # contains database schema information
./indexer.properties  # contains coverage information (EPSG)
./timeregex.properties  # contains regex string to determine date
./footprint.properties  # contains trigger for COG hosting

• A PostprocessingGetInstanceRequirements function to estimate EC2 resource requirements.
• A PostprocessingRunSsmCommandFunction function to run particular commands against the EC2 running the Geoserver. In this case the lambda runs the upload command moving data from S3 to EC2.
• A PostprocessingCheckSsmCommandFunction function polls the Geoserver data directory for the presence of all data. Basically the checksum.
• A PostprocessingBuildRaster function call the Geoserver REST API to fullfill the ImageMosaic plugin generation. After this step the imagery should be visible online through Geoserver UI and front end App.
• A PostprocessingBuildCache function calls the GeoWebCache API to start creating TileLayers and pre-seeding.

Invocation description

The following keys are required:

job_id: str, job id reference
client_tag: str, client tag to determine costing

Example

{
  "job_id": "20220520151302-1327-0ad42aeec1014698b5efc953b83b5825",
  "client_tag": "Some Client"
}