Postprocessing Build Health layer AWS Step Function

This step function postprocessing-build-health-step-function is used to orchestrate the building of a health (vegetation index) map and uploading the results to a specified Geoserver. The function acts on the imagery tiles and can use NIR image collections and multispectral data. Currenlty only accepts multispectral R-Reg-NIR band configuration. In situations where the tiles are not present but a large single tiff aim to run the utility-tiler-step-function to tile the RGB datasets. The function can skip through several stages by checking DynamoDB on the layer status.

Step Functions

The step functions that are used within this step function:

• A HealthLayerAuxiliary step function that pulls the input payload and divides the work over a Map State containing lambdas.
• A PostprocessingRasterOptimization step function that generates the COG dataset for Geoserver.

Lambdas

The lambdas orchestrated by the Step function are the following:

• A UtilityGetPrefixFunction to generate the prefix used to find the input files and where data will be stored in S3.
• A UtilityListS3BucketFunction that retrieves contents of s3 bucket used in a step function to prep for a batcher.
• A UtilityBatcherFunction that seperates a list into a list of lists of batch size. Used in a step function to prepare for a mapper.
• A PostprocessingPublishRaster function to generate the publication information for Serverless infrastructure.
• A PostprocessingBuildPropertiesFunction function to build properties files necessary for Geoserver coverage configuration.

./datastore.properties  # contains database schema information
./indexer.properties  # contains coverage information (EPSG)
./timeregex.properties  # contains regex string to determine date
./footprint.properties  # contains trigger for COG hosting

• A PostprocessingRunSsmCommandFunction function to run particular commands against the EC2 running the Geoserver. In this case the lambda runs the upload command moving data from S3 to EC2.
• A PostprocessingCheckSsmCommandFunction function polls the Geoserver data directory for the presence of all data. Basically the checksum.
• A PostprocessingHealthLayer function that does the heavy lifting. It pulls in a RGB tile, and depending on the source_layer it grabs any other tile (NIR, multispectral), crunches the numbers and outputs a normalized tile.
• A PostprocessingBuildRaster function call the Geoserver REST API to fullfill the ImageMosaic plugin generation. After this step the imagery should be visible online through Geoserver UI and front end App.
• A PostprocessingBuildCache function calls the GeoWebCache API to start creating TileLayers and pre-seeding.
• A UtilityNotificationClickupFunction function to handle ClickUp notifications.

Invocation description

The following keys are required:

job_id: str, job id reference
client_tag: str, client tag to determine costing
source_layer: str, can be ['imagery'||'RGB', 'NIR', "multispectral"],
index: str, vegetation index algorithm to use

upload: bool, switch to upload data, defaults to true
legacy: bool, switch to use the Geoserver workflow, defaults to false
options: object, you can give a set of options to impact the output when publishing
  breaks: list, which bins to use
  colours: list, which colour to use
  nodata: number, overwrite the source nodata value
  si_unit: str, the output legend label

Example

{
  "job_id":"20211213120900-1336-h9pfq384hsjkvdbalksvp3794fhgpq93yh",
  "client_tag": "Some Client",
  "source_layer": "imagery",
  "index": "VARI",
  "upload": true,
  "legacy": false,
  "options": {
    "breaks": [
      -1,
      -0.5,
      0,
      0.5,
      1
    ],
    "colours": [
      "#d7191c",
      "#fdae61",
      "#ffffc0",
      "#a6d96a",
      "#1a9641"
    ],
    "nodata": -2,
    "si_unit": ""
  }
}