Documentation ¶
Overview ¶
Package gcf runs a GCF function that triggers in 2 scenarios:
- completion of prophet-flume job in borg. On triggering it sets up new cloud BT table, scales up BT cluster (if needed) and starts a dataflow job.
- completion of BT cache ingestion dataflow job. It scales BT cluster down (if needed).
There are two set of trigger functions defined: - ProdBTImportController - PrivateBTImportController which targets on production imports and private imports. The folder structure are different for the two scenarios. The environment variables for deployments are stored in (prod|private)/*.yaml
Dataflow launcher helper functions. launchDataflowJob will either start a Dataflow job using a Flex Template or a Classic Template depending on the template path. Flex templates are json based Dataflow templates and MUST end with ".json".
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func CustomBTImportController ¶
CustomBTImportController consumes a GCS event and runs an import state machine.