oomstore

package
v0.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 14, 2022 License: Apache-2.0 Imports: 23 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	ImportStreamBatchSize = 100
)

Variables

This section is empty.

Functions

func GetEntityRowsFromInputFile added in v0.0.4 

func GetEntityRowsFromInputFile(ctx context.Context, inputFilePath string) (<-chan types.EntityRow, []string, error)

Types

type GroupBuffer added in v0.0.4 

type GroupBuffer struct {
	// contains filtered or unexported fields
}

type OomStore 

type OomStore struct {
	// contains filtered or unexported fields
}

OomStore combines online store, offline store, and metadata store instances in one place.

func Create 

func Create(ctx context.Context, opt types.OomStoreConfig) (*OomStore, error)

Create a new OomStore instance.

func Open 

func Open(ctx context.Context, opt types.OomStoreConfig) (*OomStore, error)

Open returns an OomStore instance given the configuration. Under the hood, it setups connections to the underlying databases. You should always use this method to create a new OomStore instance in code.

func TEST__New added in v0.0.3 

func TEST__New(online online.Store, offline offline.Store, metadata metadata.Store) *OomStore

TEST__New returns an OomStore instance for internal test purpose ONLY. You should NOT use this method directly in any of your code.

func (*OomStore) Apply added in v0.0.4 

func (s *OomStore) Apply(ctx context.Context, opt apply.ApplyOpt) error

Apply registers entity, feature group, feature to metadata store.

func (*OomStore) ChannelExport added in v0.0.4 

func (s *OomStore) ChannelExport(ctx context.Context, opt types.ChannelExportOpt) (*types.ExportResult, error)

ChannelExport exports the latest feature value up to the given timestamp. Usage Example: 

	exportResult, err := store.ChannelExport(ctx, opt)
	if err != nil {
		return err
	}
	for row := range exportResult.Data {
   		if row.Error != nil {
			return err
    		}
		fmt.Println(cast.ToStringSlice([]interface{}(row.Record)))
	}
    return nil

func (*OomStore) ChannelJoin added in v0.0.4 

func (s *OomStore) ChannelJoin(ctx context.Context, opt types.ChannelJoinOpt) (*types.JoinResult, error)

ChannelJoin gets point-in-time correct feature values for each entity row.

func (*OomStore) Close 

func (s *OomStore) Close() error

Close the connections to underlying databases.

func (*OomStore) CreateEntity 

func (s *OomStore) CreateEntity(ctx context.Context, opt types.CreateEntityOpt) (int, error)

CreateEntity creates metadata for an entity.

func (*OomStore) CreateFeature added in v0.0.4 

func (s *OomStore) CreateFeature(ctx context.Context, opt types.CreateFeatureOpt) (int, error)

CreateFeature creates metadata of a feature.

func (*OomStore) CreateGroup added in v0.0.3 

func (s *OomStore) CreateGroup(ctx context.Context, opt types.CreateGroupOpt) (int, error)

CreateGroup registers a feature group.

func (*OomStore) Export added in v0.0.4 

func (s *OomStore) Export(ctx context.Context, opt types.ExportOpt) error

Export exports the latest feature value up to the given timestamp, it outputs feature values to the given file path.

func (*OomStore) GetEntity 

func (s *OomStore) GetEntity(ctx context.Context, id int) (*types.Entity, error)

GetEntity gets metadata of an entity by ID.

func (*OomStore) GetEntityByName added in v0.0.2 

func (s *OomStore) GetEntityByName(ctx context.Context, name string) (*types.Entity, error)

GetEntityByName gets metadata of an entity by name.

func (*OomStore) GetFeature 

func (s *OomStore) GetFeature(ctx context.Context, id int) (*types.Feature, error)

GetFeature gets metadata of a feature by ID.

func (*OomStore) GetFeatureByFullName added in v0.1.0 

func (s *OomStore) GetFeatureByFullName(ctx context.Context, fullName string) (*types.Feature, error)

GetFeatureByFullName gets metadata of a feature by full name.

func (*OomStore) GetFeatureByName added in v0.0.2 

func (s *OomStore) GetFeatureByName(ctx context.Context, groupName string, featureName string) (*types.Feature, error)

GetFeatureByName gets metadata of a feature by group name and feature name.

func (*OomStore) GetGroup added in v0.0.3 

func (s *OomStore) GetGroup(ctx context.Context, id int) (*types.Group, error)

GetGroup gets metadata of a feature group by ID.

func (*OomStore) GetGroupByName added in v0.0.3 

func (s *OomStore) GetGroupByName(ctx context.Context, name string) (*types.Group, error)

GetGroupByName gets metadata of a feature group by name.

func (*OomStore) GetRevision 

func (s *OomStore) GetRevision(ctx context.Context, id int) (*types.Revision, error)

GetRevision gets metadata of a revision by ID.

func (*OomStore) GetRevisionBy added in v0.0.2 

func (s *OomStore) GetRevisionBy(ctx context.Context, groupID int, revision int64) (*types.Revision, error)

GetRevisionBy gets metadata of a revision by group ID and revision.

func (*OomStore) Import added in v0.0.3 

func (s *OomStore) Import(ctx context.Context, opt types.ImportOpt) (int, error)

Import API imports data from external data source to offline store.

func (*OomStore) InitPushProcessor added in v0.1.0 

func (s *OomStore) InitPushProcessor(ctx context.Context, cfg *types.PushProcessorConfig)

InitPushProcessor inits a push processor.

func (*OomStore) Join added in v0.0.3 

func (s *OomStore) Join(ctx context.Context, opt types.JoinOpt) error

Join gets point-in-time correct feature values for each entity row. The method is similar to Join, except that both input and output are files on disk. Input File should contain header, the first two columns of Input File should be entity_key, unix_milli, then followed by other real-time feature values.

func (*OomStore) ListEntity 

func (s *OomStore) ListEntity(ctx context.Context, opt types.ListEntityOpt) (types.EntityList, error)

ListEntity lists metadata of entities given entity names.

func (*OomStore) ListFeature 

func (s *OomStore) ListFeature(ctx context.Context, opt types.ListFeatureOpt) (types.FeatureList, error)

ListFeature lists metadata of features meeting particular criteria.

func (*OomStore) ListGroup added in v0.0.3 

func (s *OomStore) ListGroup(ctx context.Context, opt types.ListGroupOpt) (types.GroupList, error)

ListGroup lists metadata of feature groups meeting particular criteria.

func (*OomStore) ListRevision 

func (s *OomStore) ListRevision(ctx context.Context, groupID *int) (types.RevisionList, error)

ListRevision lists metadata of revisions of a same group.

func (*OomStore) OnlineGet added in v0.0.3 

func (s *OomStore) OnlineGet(ctx context.Context, opt types.OnlineGetOpt) (*types.FeatureValues, error)

OnlineGet gets online features of a particular entity instance.

func (*OomStore) OnlineMultiGet added in v0.0.3 

func (s *OomStore) OnlineMultiGet(ctx context.Context, opt types.OnlineMultiGetOpt) (map[string]*types.FeatureValues, error)

OnlineMultiGet gets online features of multiple entity instances.

func (*OomStore) Ping added in v0.0.4 

func (s *OomStore) Ping(ctx context.Context) error

Ping verifies the connections to the backend stores are still alive

func (*OomStore) Push added in v0.0.4 

func (s *OomStore) Push(ctx context.Context, opt types.PushOpt) error

Push inserts stream feature values to online store and offline store

func (*OomStore) Snapshot added in v0.0.4 

func (s *OomStore) Snapshot(ctx context.Context, groupName string) error

Snapshot generates snapshots for a specified group based on user-defined snapshot interval. This API only applies to stream features.

func (*OomStore) Sync added in v0.0.2 

func (s *OomStore) Sync(ctx context.Context, opt types.SyncOpt) error

Sync a particular revision of a feature group from offline to online store. It is a streaming process - it writes to online store while reading from offline store. This helps get rid of unwanted out-of-memory errors, where size of the particular revision outgrows memory limit of your machine.

func (*OomStore) UpdateEntity 

func (s *OomStore) UpdateEntity(ctx context.Context, opt types.UpdateEntityOpt) error

UpdateEntity updates metadata for an entity.

func (*OomStore) UpdateFeature 

func (s *OomStore) UpdateFeature(ctx context.Context, opt types.UpdateFeatureOpt) error

UpdateFeature updates metadata of a feature.

func (*OomStore) UpdateGroup added in v0.0.3 

func (s *OomStore) UpdateGroup(ctx context.Context, opt types.UpdateGroupOpt) error

UpdateGroup updates metadata of a feature group.

type PushProcessor added in v0.1.0 

type PushProcessor struct {
	// contains filtered or unexported fields
}

func (*PushProcessor) Close added in v0.1.0 

func (p *PushProcessor) Close() error

Close push processor.

func (*PushProcessor) Push added in v0.1.0 

func (p *PushProcessor) Push(record types.StreamRecord)

Push stream record to push processor.

Source Files

View all Source files

Directories

Path Synopsis
apply

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.