vertigo

README

vertigo

An alternative Vertex AI (AI Platform) Featurestore Online Serving Client.

Goals

The main goal of this project is to provide an alternative way of interacting with the Online Featurestore Service. Specifically, several semantics are introduced in this API:

• Query
• Config
• Entity

The Query type contains parameterized inputs for the ReadFeatureValues RPC, containing the entity ID, features to return (projection), and the entity type.

The Config type contains project level configuration, around the GCP Region, GCP Project ID, and the Featurestore you are querying from.

Lastly, the Entity type is a wrapper struct around the Header and EntityView.Data fields in the ReadFeatureValuesResponse. The Entity has a receiver function that can scan the header and data into a user-provided struct pointer.

You can provide an interface{} (pointer to a struct) type to the entity.ScanStruct function, and as long as your struct has a vertex tag with the corresponding feature name, it will load the values into the struct.

Example

The following is an example of using vertigo for a "customer" entity, which has the following features defined in Vertex AI Featurestore:

entity_type: my_customer
features:
  - name: segment
    type: STRING
  - name: market_audiences
    type: STRING_ARRAY
  - name: six_month_spend
    type: DOUBLE
  - name: another_numeric_feature
    type: INT64 

Note that the ${features[i].name} directly maps to the vertex:"${feature_name}" struct tag.

package main

import (
	"context"
	"github.com/bradleybonitatibus/vertigo"
	"log"
)

// MyCustomer is a user provided struct that contain `"vertex"` tags that map to the entity feature ID in
// the Vertex Feature Store.
type MyCustomer struct {
	Segment               string   `json:"segment" vertex:"segment"`
	MarketAudiences       []string `json:"market_audiences" vertex:"market_audiences"`
	SixMonthSpend         *float64 `json:"six_month_spend" vertex:"six_month_spend"`
	AnotherNumericFeature int64    `json:"another_numeric_feature" vertex:"another_numeric_feature"`
}

func main() {
	var region string
	var projectID string
	var featurestoreName string
	// region = "my-gcp-region"
	// projectID = "my-project-id"
	// featurestoreName = "my_featurestore_name"
	ctx := context.Background()
	client, err := vertigo.NewClient(ctx, &vertigo.Config{
		Region:           region,
		ProjectID:        projectID,
		FeatureStoreName: featurestoreName,
	})
	defer client.Close()
	if err != nil {
		log.Fatalf("vertigo.NewClient: %v", err)
	}
	myCust := MyCustomer{}

	entity, err := client.GetEntity(ctx, &vertigo.Query{
		EntityType: "my_customer",
		EntityID:   "123abc",
		Features:   []string{"*"},
	})
	if err != nil {
		log.Fatalf("client.GetEntity: %v", err)
	}
	err = entity.ScanStruct(&myCust)
	if err != nil {
		log.Fatalf("entity.ScanStruct: %v", err)
	}
	// continue using MyCustomer as you wish.
}

Documentation

Overview

Package vertigo is an opinionated approach to interacting with the Vertex AI Online Feature Store gRPC API.

Index

• Constants
• Variables
• type Client
  • func NewClient(ctx context.Context, cfg *Config) (*Client, error)
  • func (c *Client) Close() error
  • func (c *Client) GetEntity(ctx context.Context, query *Query) (*Entity, error)
• type Config
  • func (c *Config) APIEndpoint() string
  • func (c *Config) ParentPath() string
• type ConfigBuilder
  • func NewConfigBuilder() ConfigBuilder
• type Entity
  • func (e *Entity) ScanStruct(dst interface{}) error
• type Query
  • func (q *Query) BuildRequest(cfg *Config) *aiplatformpb.ReadFeatureValuesRequest

Constants

const DefaultRegion = "us-central1"

DefaultRegion is the region

const VertexEndpoint = "aiplatform.googleapis.com:443"

VertexEndpoint is the URL:PORT for the AI Platform API. This is used in providing non-default regional endpoints for Vertex AI.

Variables

var ErrInvalidFeatureStoreName = errors.New("feature store name is not valid")

var ErrInvalidProjectID = errors.New("project id is not valid")

Types

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client is the Vertigo client, which uses the aiplatformv1beta1 gRPC API to communicate with the FeaturestoreOnlineServingClient.

func NewClient

func NewClient(ctx context.Context, cfg *Config) (*Client, error)

NewClient creates a Client using the provided Config.

func (*Client) Close

func (c *Client) Close() error

Close closes the underlying vertex AI gRPC client.

func (*Client) GetEntity

func (c *Client) GetEntity(ctx context.Context, query *Query) (*Entity, error)

GetEntity calls the Vertex AI Online Serving API and retrieves the response in the form of an Entity and error if one occurs.

type Config

type Config struct {
	// ProjectID is the GCP Project ID your Feature Store resides in.
	ProjectID string `json:"project_id" yaml:"project_id"`

	// Region is the GCP Region (sometimes referred to as Location) the Feature Store is running in.
	// Please see `https://cloud.google.com/vertex-ai/docs/general/locations` for a list
	// of supported regions.
	Region string `json:"region" yaml:"region"`

	// FeatureStoreName is the name of the feature store.
	FeatureStoreName string `json:"feature_store_name" yaml:"feature_store_name"`
}

Config is the struct the contains configuration that is used in the Vertex AI API.

func (*Config) APIEndpoint

func (c *Config) APIEndpoint() string

APIEndpoint is the Vertex AI API Endpoint, specific to the Region you have deployed your feature store in.

func (*Config) ParentPath

func (c *Config) ParentPath() string

ParentPath is the resource hierarchy for the feature store that we are interacting with.

type ConfigBuilder

type ConfigBuilder interface {
	WithRegion(region string) ConfigBuilder
	WithProjectID(projectID string) ConfigBuilder
	WithFeatureStoreName(featureStore string) ConfigBuilder
	Apply() (*Config, error)
}

ConfigBuilder provides a fluent interface for building the Vertigo Config. If Region is not set, it will fall back to the DefaultRegion value.

func NewConfigBuilder

func NewConfigBuilder() ConfigBuilder

NewConfigBuilder returns a fluent API to build the Config struct using the ConfigBuilder interface.

type Entity

type Entity struct {
	ID string
	// contains filtered or unexported fields
}

Entity contains the header and data from the aiplatform.ReadFeatureValuesResponse to be used to scan the response into a user provided struct.

func (*Entity) ScanStruct

func (e *Entity) ScanStruct(dst interface{}) error

ScanStruct will parse the ReadFeatureValues response from the online serving client and load the features into dst. DST must be a pointer to a struct and have valid `vertex` tags that map to the feature IDs of the entity being parsed.

type Query

type Query struct {
	EntityType string
	EntityID   string
	Features   []string
}

Query represents a query to the Vertex AI Online Feature Store API for getting an Entity's Feature Values.

func (*Query) BuildRequest

func (q *Query) BuildRequest(cfg *Config) *aiplatformpb.ReadFeatureValuesRequest

BuildRequest translates the Query struct into an AI Platform ReadFeatureValuesRequest, which is submitted to the Vertex AI Online Feature Store API to retrieve the Feature Values for an entity.