Postprocessing Build Waterlogging layer AWS Step Function

This step function postprocessing-build-waterlogging-step-function is used to orchestrate the building of a waterlogging map and uploading the results to a specified Geoserver. The function requires a succesfull DTM, run postprocessing-build-dem-step-function if that is not the case. Processing the waterlogging map is a memory expensive operation. Therefore this step function leverages an EC2 to run the computations. The EC2 image is build with tertiary software SAGA GIS. Waterlogging is essentialy the difference between a sink filled DTM and the original DTM. It accentuates a risk of water stagnation in an area with modelled unlimited input of water to the system. The DTM is resampled to 0.5m x 0.5m pixels due hydrological algorithms underperforming at high resolution DEMs.

Step Functions

The step functions that are used within this step function:

• A PostprocessingStartECS step function that starts an EC2 within the ECS cluster based on set parameters.
• A PostprocessingRasterOptimization step function that generates the COG dataset for Geoserver.

Lambdas

The lambdas orchestrated by the Step function are the following:

• A UtilityGetPrefixFunction to generate the prefix used to find the input files and where data will be stored in S3.
• A PostprocessingBuildPropertiesFunction function to build properties files necessary for Geoserver coverage configuration.

./datastore.properties  # contains database schema information
./indexer.properties  # contains coverage information (EPSG)
./timeregex.properties  # contains regex string to determine date
./footprint.properties  # contains trigger for COG hosting

• A PostprocessingGetInstanceRequirements function to estimate EC2 resource requirements.
• A PostprocessingPublishRaster function to generate the publication information for Serverless infrastructure.
• A PostprocessingRunSsmCommandFunction function to run particular commands against the EC2 running the Geoserver. In this case the lambda runs the upload command moving data from S3 to EC2.
• A PostprocessingCheckSsmCommandFunction function polls the Geoserver data directory for the presence of all data. Basically the checksum.
• A PostprocessingBuildRaster function call the Geoserver REST API to fullfill the ImageMosaic plugin generation. After this step the imagery should be visible online through Geoserver UI and front end App.
• A PostprocessingBuildCache function calls the GeoWebCache API to start creating TileLayers and pre-seeding.

Invocation description

The following keys are required:

job_id: str, job id reference
client_tag: str, client tag to determine costing

Example

{
  "job_id": "20220520151302-1327-0ad42aeec1014698b5efc953b83b5825",
  "client_tag": "Some Client"
}