splitdump

Documentation

Overview

Package splitdump nibble large Cloud Asset Inventory dumps into many PubSub asset feed messages - Triggered by: Google Cloud Storage event when a new dump is delivered - Instances: only one - Output:

• PubSub messages formated like Cloud Asset Inventory real-time feed messages
• Delivered in the same topics as used per CAI real-time
• Tags to differentiate them from CAI real time feeds
• Create missing topics en the fly (best effort) in case it does not already exist for real-time

- Cardinality:

• one-many: one dump is nubbled in many feed messages
• To ensure scallabilty the function is recurssive:
• dump size > x lines then segment it in x line child dumps
• else nibble the dump
• x is set through an environment variable

- Automatic retrying: yes - Is recurssive: yes - Required environment variables:

• CAIEXPORTBUCKETNAME the name of the GCS bucket where are delivered the CAI dumps
• IAMTOPICNAME the name of the topic used for all IAM policies feed messages

Index

• func EntryPoint(ctxEvent context.Context, gcsEvent ram.GCSEvent, global *Global) error
• func Initialize(ctx context.Context, global *Global)
• type Asset
• type AssetLegacy
• type FeedMessage
• type Global

Functions

func EntryPoint

func EntryPoint(ctxEvent context.Context, gcsEvent ram.GCSEvent, global *Global) error

EntryPoint is the function to be executed for each cloud function occurence

func Initialize

func Initialize(ctx context.Context, global *Global)

Initialize is to be executed in the init() function of the cloud function to optimize the cold start

Types

type Asset

type Asset struct {
	Name      string          `json:"name"`
	AssetType string          `json:"assetType"`
	Ancestors []string        `json:"ancestors"`
	IamPolicy json.RawMessage `json:"iamPolicy"`
	Resource  json.RawMessage `json:"resource"`
}

Asset uses the new CAI feed format

type AssetLegacy

type AssetLegacy struct {
	Name      string          `json:"name"`
	AssetType string          `json:"asset_type"`
	Ancestors []string        `json:"ancestors"`
	IamPolicy json.RawMessage `json:"iam_policy"`
	Resource  json.RawMessage `json:"resource"`
}

AssetLegacy uses the CAI export legacy format, not the new CAI feed format aka asset_type instead of assetType, iam_policy instead of iamPolicy

type FeedMessage

type FeedMessage struct {
	Asset  Asset      `json:"asset"`
	Window ram.Window `json:"window"`
	Origin string     `json:"origin"`
}

FeedMessage Cloud Asset Inventory feed message

type Global

type Global struct {
	// contains filtered or unexported fields
}

Global structure for global variables to optimize the cloud function performances