Postprocessing Build Imagery AWS Step Function

This step function postprocessing-build-imagery-step-function is used to orchestrate the procedure for instantiating a Geoserver workspace and uploading imagery data to a Geoserver or EFS. Within the core the step function is called by metashape.py, but the step function can be used on other datasets holding RGB information. It is strongly advised to run the utility-tiler-step-function to tile the RGB datasets. The status of the layer and its COG counterpart are updated within the job state and clickup notifications. Publication of the data is by default within the Serverless infrastructure, but an admin can choose to send data to a Geoserver or disable uploading entirely.

Step functions

The step functions that are used within this step function:

• A PostprocessingRasterOptimization step function that generates the COG dataset.

Lambdas

The lambdas orchestrated by the Step function are the following:

• A PostprocessingInstantiate function to set up workspaces and users for a particular dataset.
• A PostprocessingBuildPropertiesFunction function to build properties files necessary for Geoserver coverage configuration.

./datastore.properties  # contains database schema information
./indexer.properties  # contains coverage information (EPSG)
./timeregex.properties  # contains regex string to determine date
./footprint.properties  # contains trigger for COG hosting

• A PostprocessingPublishRaster function to generate the publication information for Serverless infrastructure.
• A PostprocessingRunSsmCommandFunction function to run particular commands against the EC2 running the Geoserver. In this case the lambda runs the upload command moving data from S3 to EC2.
• A PostprocessingCheckSsmCommandFunction function polls the Geoserver data directory for the presence of all data. Basically the checksum.
• A PostprocessingBuildRaster function call the Geoserver REST API to fullfill the ImageMosaic plugin generation. After this step the imagery should be visible online through Geoserver UI and front end App.
• A PostprocessingBuildCache function calls the GeoWebCache API to start creating TileLayers and pre-seeding.
• A UtilityNotificationClickupFunction function to handle ClickUp notifications.

Invocation description

The following keys are required:

job_id: str, job id reference
client_tag: str, client tag to determine costing

The following keys are optional

cog_prefix: str, path to COG source data (usually COG/ appended), generated in PostprocessingInstantiate
geoserver_url: str, url of the geoserver, not used in legacy: false, generated in PostprocessingInstantiate
geoserver_api_id: int, ID of geoserver found in database, not used in legacy: false, generated in PostprocessingInstantiate
layername: str, overwrite the output layername, defaults to imagery
legacy: bool, switch to prepare data for Geoserver of Serverless infrastructure, defaults to false.
prefix: str, path to source data, generated in PostprocessingInstantiate
upload: bool, switch to allow step function to upload to either infrastructure, defaults to true
workspace: str, overwrite the output workspace, generated in PostprocessingInstantiate

Warning: Setting upload false and legacy true will still perform an upload to geoserver.

Example

{
  "job_id": "20211213120900-1336-h9pfq384hsjkvdbalksvp3794fhgpq93yh",
  "geoserver_url": "https://some-geoserver.geoserver/geoserver/web",
  "geoserver_api_id": 21,
  "prefix": "some/object",
  "cog_prefix": "some/object_cog",
  "client_tag": "Some Client",
  "legacy": false,
  "workspace": "org_number_workspace_1",
  "upload": true
}