PubSub Message Schema Airspace
Originally
ADR-0039 PubSub Message Schema - Airspace (v8) · Source on Confluence ↗PubSub Message Schema - Airspace
Context
This document describes the format and metadata of a pubsub message containing Airspace Table data.
Atlas pipeline has a worker attached to Airspaces table which monitors any changes done to that table. Those changes are
then transformed to fit required PubSub message schema and published to the topic: <topic_name>
Any subscriber that listen to the topic is able to use that information to fully (or partially) reproduce Atlas Airspace
table in any storage system and keep it up to date.
PubSub Message Schema:
PubSub message contains 3 fields:
- data
- ordering_key
- attributes
data
Binary string representing row processed in each action as a struct.
It has a different format, depending on an action done.
1. Insert / Update
Airspace Schema
| Name | Values Range | Description |
|---|---|---|
| uuid | - | airspace uuid |
| usage | warning / restricted / prohibited | Information about the operation of a drone within the airspace. |
| ruleset | droneup-delivery | To which ruleset this airspace belongs |
| kind | - | Describes the type of an airspace (eg. class-b, tfr, national_park, stadium) |
| metadata | - | metadata object specific to a kind (see custom_metadata for details)) |
| floor | - | altitude object representing the floor |
| ceiling | - | altitude object representing the ceiling |
| schedule | - | schedule object |
| geometry | - | WKT String representing geometry in EPSG-4326 (WSG84) projection |
Altitude Schema
| Name | Values Range | Description |
|---|---|---|
| value | - | scalar measurement of altitude in unit in referense to ref |
| unit | ft / m | ft (feet), m (meters) |
| ref | AGL / MSL | AGL (Above Ground Level), MSL (Mean Sea Level) |
Schedule Schema
| Name | Values Range | Description |
|---|---|---|
| ??? | ??? | ??? |
Example message:
Binary json is represented with new line for better readability
{
data: b"{
'uuid': '241fsa-521gg-5521aaz',
'usage': 'restricted',
'ruleset': 'droneup-delivery',
'kind': 'class_b_airspace'
'metadata':{
"faa_id":"ROW",
"airport_name":"Roswell Air Center",
"icao":"KROW",
"related_airspace_class":"E",
"airspace_global_id":"A0D4D5ED-2D58-4263-8342-437CB7174D1B"
}
'floor':{
value: '400',
unit: 'ft',
ref: 'AGL'
}
'ceiling': {
value: '400',
unit: 'ft',
ref: 'AGL'
}
'schedule': <airspace_schedule>
'geometry': 'POINT( 11, 12)'
"
ordering_key: '2'
attributes: {
"action": "insert",
"commit_version": 1,
"commit_timestamp": "2023-07-20 09:04:14.331000"
"version": 1,
}
}2. Delete
| Name | Type | Values | Description |
|---|---|---|---|
| uuid | string | - | airspace uuid |
ordering_key
Type: string
String representation of an automatically generated integer ID assigned to a commit. This ID is updated whenever there
is a change in the table. A higher ID value indicates that the data is more up-to-date.
attributes
Struct of cutom atributes, that describes the change
| Name | Type | Values | Description |
|---|---|---|---|
| action | string | insert / update / delete | Action done on Airspaces table. |
| commit_version | integer | Table commit version number (equal to timestamp) | |
| timestamp | string | example: 2023-07-20 09:04:14.331000 | Parameter that indicates when change happen on the table. It is generated along with commit id. |
| version | integer | Version number for message schema |
Notes:
Field metadata is strictly related to a kind. Different kind of object may contain different metadata fields
Consequences
Atlas and it’s Airspace table consumers are coupled via PubSub topic, which allows us to evolve a schema without
interaction on each other work. Currently, Atlas pushes only the subset of row values required by its consumers.
Alternatives Considered
Delta change log contains 4 types of actions: insert/pre-update/post-update and delete, which has full row data.
Insert - new data inserted to the table
Pre-Update - how row data looked before the update
Post-Update - how row data looked after the update
Delete - full row data that has been removed
To lower the message size the decision was made to publish only uuid for delete actions and skip publishing *
pre-update* actions.