Documentation
¶
Overview ¶
Package splitdump nibble large CAI dumps into PubSub asset feed messages
One dump line = one PubSub message.
Triggered by ¶
Google Cloud Storage event when a new dump is delivered.
Instances ¶
Only one.
Output ¶
- PubSub messages formated like Cloud Asset Inventory real-time feed messages.
- Delivered in the same topics as used per CAI real-time.
- Tags to differentiate them from CAI real time feeds.
- Create missing topics en the fly (best effort) in case it does not already exist for real-time.
Cardinality ¶
One-many: one dump is nubbled in many feed messages.
To ensure scallabilty the function is recurssive:
- If dump size > x lines then segments the dumo into child dumps of x lines.
- else nibble the dump, one dump line = one PubSub message.
- x is set through an environment variable, e.g. 1000.
Automatic retrying ¶
Yes.
Is recurssive ¶
Yes.
Required environment variables ¶
- CAIEXPORTBUCKETNAME the name of the GCS bucket where are delivered the CAI dumps.
- IAMTOPICNAME the name of the topic used for all IAM policies feed messages.
Implementation example
package p
import (
"context"
"github.com/BrunoReboul/ram/services/splitdump"
"github.com/BrunoReboul/ram/utilities/ram"
)
var global splitdump.Global
var ctx = context.Background()
// EntryPoint is the function to be executed for each cloud function occurence
func EntryPoint(ctxEvent context.Context, gcsEvent ram.GCSEvent) error {
return splitdump.EntryPoint(ctxEvent, PubSubMessage, &global)
}
func init() {
splitdump.Initialize(ctx, &global)
}
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func EntryPoint ¶
EntryPoint is the function to be executed for each cloud function occurence
func Initialize ¶
Initialize is to be executed in the init() function of the cloud function to optimize the cold start
Source Files