services/

directory
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 5, 2021 License: Apache-2.0

README

Service packages structure

All service cloud function packages share a consistent structure

Two functions and one type

Initialize function
  • Goal
    • Optimize cloud function performance by reducing the invocation latency
  • Implementation
    • Is executed once per cloud function instance as a cold start.
    • Cache objects expensive to create, like clients
    • Retreive settings once, like environment variables
    • Cached objects and reteived settings are exposed in one global variable named global
Global type
  • A struct to define a global variable carrying cached objects and retreived settings by Initialized function and used by EntryPoint function
EntryPoint function
  • Goal
    • Execute operations to be performed each time the cloud function is invoked
  • Implementation
    • Is executed on every event triggering the cloud function
    • Uses cached objects and retreived settings prepared by the Initialized function and carried by a global variable of type Global
    • Performs the task a given service is targetted to do that is described before the package key word

Automatic retrying

Automatic retrying is consistently implemented in RAM service packages as documented in Google Cloud Function best practice Retrying Background Functions

Impact on cloud functions Stackdriver logs:

  • Errors entry in log only appears for transient errors.
    • As en error is reported the function is retried.
  • Other errors and loged as information to avoid unwanted retries
    • To find errors in such cloud function logs you can use the following Stackdriver logging filter
resource.type="cloud_function"
textPayload:"REDO_ON_TRANSIENT"

The standard exponential backoff algorithm leads to have:

  • 343 retries in one hour
  • 157 retries during the first 100 seconds

Documentation

Directories

Path Synopsis
Package convertlog2feed convert log entry messsage from pubsub into feed compatible format pubusub message Instances Some.
 Package convertlog2feed convert log entry messsage from pubsub into feed compatible format pubusub message Instances Some.
Package dumpinventory request CAI to perform an export Triggered by Cloud Scheduler Job, through PubSub messages.
 Package dumpinventory request CAI to perform an export Triggered by Cloud Scheduler Job, through PubSub messages.
Package getgroupsettings retreives the settings of one group from `Groups Settings API` Triggered by PubSub messages from the GCI groups topic.
 Package getgroupsettings retreives the settings of one group from `Groups Settings API` Triggered by PubSub messages from the GCI groups topic.
Package listgroupmembers extract all members from a group in GCI directory using the Admin SDK API Triggered by PubSub messages from the GCI groups topic.
 Package listgroupmembers extract all members from a group in GCI directory using the Admin SDK API Triggered by PubSub messages from the GCI groups topic.
Package listgroups extract all groups from a GCI directory using the Admin SDK API Triggered by Cloud Scheduler Job, through PubSub messages.
 Package listgroups extract all groups from a GCI directory using the Admin SDK API Triggered by Cloud Scheduler Job, through PubSub messages.
Package monitor check asset compliance Triggered by Resource or IAM policies assets feed messages in PubSub topics.
 Package monitor check asset compliance Triggered by Resource or IAM policies assets feed messages in PubSub topics.
Package publish2fs publish assets resource feeds as FireStore documents It manages creation, updates and delete.
 Package publish2fs publish assets resource feeds as FireStore documents It manages creation, updates and delete.
Package setdashboards set cloud monitoring dashboards for RAM Instances Multiple: one per dashboard.
 Package setdashboards set cloud monitoring dashboards for RAM Instances Multiple: one per dashboard.
Package setfeeds set Cloud Asset Inventory feeds at organization level Instances per targeted GCP organization, per environment - one feed for all iam policies - one feed per asset type for resource metadata.
 Package setfeeds set Cloud Asset Inventory feeds at organization level Instances per targeted GCP organization, per environment - one feed for all iam policies - one feed per asset type for resource metadata.
Package setlogmetrics set cloud logging metrics from logs Instances Multiple: one per metric.
 Package setlogmetrics set cloud logging metrics from logs Instances Multiple: one per metric.
Package setlogsinks set organization log sinks to pubsub Instances Multiple.
 Package setlogsinks set organization log sinks to pubsub Instances Multiple.
Package splitdump nibble large CAI dumps into PubSub asset feed messages One dump line = one PubSub message.
 Package splitdump nibble large CAI dumps into PubSub asset feed messages One dump line = one PubSub message.
Package stream2bq streams PubSub message into BigQuery tables It can stream into 3 RAM tables: 1) assets 2) compliance states 3) violations.
 Package stream2bq streams PubSub message into BigQuery tables It can stream into 3 RAM tables: 1) assets 2) compliance states 3) violations.
Package upload2gcs stores feeds as JSON files in a GCS bucket Manage file creation (with override) and deletion.
 Package upload2gcs stores feeds as JSON files in a GCS bucket Manage file creation (with override) and deletion.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.