xfirestore

Documentation

Index

• type Extractor
  • func NewExtractor(spec model.Spec, id string, client FirestoreClient, defaultNamespace string) (*Extractor, error)
  • func (e *Extractor) Extract(ctx context.Context, query model.ExtractorQuery, result any) (error, bool)
  • func (e *Extractor) ExtractFromSink(ctx context.Context, query model.ExtractorQuery, result *[]*model.Transformed) (error, bool)
  • func (e *Extractor) SendToSource(ctx context.Context, eventData any) (string, error)
  • func (e *Extractor) StreamExtract(ctx context.Context, reportEvent model.ProcessEventFunc, err *error, ...)
• type FirestoreClient
• type Loader
  • func NewLoader(spec *model.Spec, id string, client FirestoreClient, defaultNamespace string) (*Loader, error)
  • func (l *Loader) Shutdown()
  • func (l *Loader) StreamLoad(ctx context.Context, data []*model.Transformed) (string, error, bool)
• type Query
• type QueryType

Types

type Extractor

type Extractor struct {
	// contains filtered or unexported fields
}

func NewExtractor

func NewExtractor(
	spec model.Spec,
	id string,
	client FirestoreClient,
	defaultNamespace string) (*Extractor, error)

func (*Extractor) Extract

func (e *Extractor) Extract(ctx context.Context, query model.ExtractorQuery, result any) (error, bool)

Currently, this function requires the caller to know how to handle Datastore specific storage structures. Instead, for generic repository usage, hiding away Datastore specifics, use ExtractFromSink(). TODO: Add support for q := datastore.NewQuery("Entity").Filter("A =", 12).Limit(1) (added partly in ExtractFromSink())

func (*Extractor) ExtractFromSink

func (e *Extractor) ExtractFromSink(ctx context.Context, query model.ExtractorQuery, result *[]*model.Transformed) (error, bool)

Currently only supporting Sink specs with a single Datastore "table" (kind)

func (*Extractor) SendToSource

func (e *Extractor) SendToSource(ctx context.Context, eventData any) (string, error)

func (*Extractor) StreamExtract

func (e *Extractor) StreamExtract(
	ctx context.Context,
	reportEvent model.ProcessEventFunc,
	err *error,
	retryable *bool)

type FirestoreClient

type FirestoreClient interface {
	Put(ctx context.Context, key *datastore.Key, src any) (*datastore.Key, error)
	Get(ctx context.Context, key *datastore.Key, dst any) (err error)
	GetAll(ctx context.Context, q *datastore.Query, dst any) (keys []*datastore.Key, err error)
}

type Loader

type Loader struct {
	// contains filtered or unexported fields
}

The Firestore Loader supports arbitrary ingestion of data into Firestore in Datastore mode. It currently does not support auto generated entity IDs, but require entity name syntax to be provided in the GEIST spec.

func NewLoader

func NewLoader(
	spec *model.Spec,
	id string,
	client FirestoreClient,
	defaultNamespace string) (*Loader, error)

func (*Loader) Shutdown

func (l *Loader) Shutdown()

func (*Loader) StreamLoad

func (l *Loader) StreamLoad(ctx context.Context, data []*model.Transformed) (string, error, bool)

The GEIST Firestore Loader implementation currently only supports a single input Transformed object, TODO: Check if eventSplit transforms causing multiple Transformed, should be supported as input --> Probably NO! TODO: Remove Kind as array and keep single

type Query

type Query struct {
	Type         QueryType
	Namespace    string
	Kind         string
	EntityName   string
	CompositeKey []model.KeyValueFilter
}

CompositeKey works as a SQL 'WHERE key1 = value1 and key2 = value2 ...' for all props in a stream spec with index=true. Due to the simple nature of GEIST GET /streams/.../events?... API, only string values are supported. A more full-fledged query API should be provided by a separate query service, so no need to support complex queries in GEIST.

type QueryType

type QueryType int

const (
	Unknown QueryType = iota
	KeyValue
	CompositeKeyValue
	All
)