bigtable

package
v1.35.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 27, 2022 License: Apache-2.0 Imports: 14 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// BtPlaceStatsVarPrefix for place to statsvar list cache.
	BtPlaceStatsVarPrefix = "d/0/"
	// BtStatVarGroup is the key for statvar group cache.
	BtStatVarGroup = "d/1"
	// BtSVAndSVGExistence is the key for stat var and stat var group existence cache.
	BtSVAndSVGExistence = "d/2/"
	// BtObsTimeSeries is the key for obs time series cache.
	BtObsTimeSeries = "d/3/"
	// BtPlacePagePrefix for place page cache.
	BtPlacePagePrefix = "d/4/"
	// BtProteinPagePrefix for protein page cache.
	BtProteinPagePrefix = "d/6/"
	// BtTriplesPrefix for internal GetTriples cache.
	BtTriplesPrefix = "d/7/"
	// BtObsCollectionDateFrequency for obs collection cache that contains the frequency of each
	// date across places.
	BtObsCollectionDateFrequency = "d/8/"
	// BtArcsPrefix for internal arcs cache.
	BtArcsPrefix = "d/9/"
	// BtStatVarSummary for stat var summary cache.
	BtStatVarSummary = "d/a/"
	// BtPlacesInPrefix for GetPlacesIn cache.
	BtPlacesInPrefix = "d/c/"
	// BtPlacesMetadataPrefix for GetPlaceMetadata cache.
	BtPlacesMetadataPrefix = "d/d/"
	// BtObsCollection for obs collection cache.
	BtObsCollection = "d/e/"
	// BtInPropValPrefix for in-arc prop value.
	BtInPropValPrefix = "d/l/"
	// BtOutPropValPrefix for out-arc prop value.
	BtOutPropValPrefix = "d/m/"
	// BtRelatedLocationsSameTypePrefix for related places with same type.
	BtRelatedLocationsSameTypePrefix = "d/o/"
	// BtRelatedLocationsSameTypeAndAncestorPrefix for related places with same type and ancestor.
	BtRelatedLocationsSameTypeAndAncestorPrefix = "d/q/"
	// BtRelatedLocationsSameTypePCPrefix for related places with same type, per capita.
	BtRelatedLocationsSameTypePCPrefix = "d/o0/"
	// BtRelatedLocationsSameTypeAndAncestorPCPrefix for related places with same type and ancestor,
	// per capita.
	BtRelatedLocationsSameTypeAndAncestorPCPrefix = "d/q0/"

	// BtReconIDMapPrefix for ID mapping for ID-based recon. The key excludes DCID.
	BtReconIDMapPrefix = "d/5/"
	// BtCoordinateReconPrefix for coordinate recon.
	BtCoordinateReconPrefix = "d/b/"

	// BtFamily is the key for the row.
	BtFamily = "csv"
	// BtCacheLimit is the cache limit. The limit is per predicate and neighbor type.
	BtCacheLimit = 500
	// BtBatchQuerySize is the size of BigTable batch query.
	BtBatchQuerySize = 1000
)

Variables

This section is empty.

Functions

func BuildObsCollectionDateFrequencyKey added in v1.33.0 

func BuildObsCollectionDateFrequencyKey(parentPlace, childType string, statVars []string) (
	bigtable.RowList, map[string]string)

BuildObsCollectionDateFrequencyKey builds bigtable key for obs collection cache that contains the frequency of each date across places.

func BuildObsCollectionKey added in v1.33.0 

func BuildObsCollectionKey(parentPlace, childType, date string, statVars []string) (
	bigtable.RowList, map[string]string)

BuildObsCollectionKey builds bigtable key for obs collection cache.

func BuildObsTimeSeriesKey added in v1.33.0 

func BuildObsTimeSeriesKey(
	places []string, statVars []string) (
	bigtable.RowList, map[string]*util.PlaceStatVar)

BuildObsTimeSeriesKey builds bigtable key for obs time series cache.

func BuildPlaceMetadataKey added in v1.34.0 

func BuildPlaceMetadataKey(places []string) bigtable.RowList

BuildPlaceMetadataKey builds Bigtable key for place metadata cache

func BuildPlaceStatsVarKey 

func BuildPlaceStatsVarKey(dcids []string) bigtable.RowList

BuildPlaceStatsVarKey builds bigtable key for place stat vars cache

func BuildPlacesInKey added in v1.33.0 

func BuildPlacesInKey(dcids []string, placeType string) bigtable.RowList

BuildPlacesInKey builds bigtable key for place in cache

func BuildPropertyLabelKey 

func BuildPropertyLabelKey(dcids []string) bigtable.RowList

BuildPropertyLabelKey builds bigtable key for property label cache

func BuildPropertyValuesKey 

func BuildPropertyValuesKey(
	dcids []string, prop string, arcOut bool) bigtable.RowList

BuildPropertyValuesKey builds bigtable key for property value cache

func BuildStatExistenceKey 

func BuildStatExistenceKey(
	places []string, statVars []string) (
	bigtable.RowList, map[string]*util.PlaceStatVar)

BuildStatExistenceKey builds bigtable key for stat existence cache

func BuildStatVarSummaryKey 

func BuildStatVarSummaryKey(statVars []string) bigtable.RowList

BuildStatVarSummaryKey builds bigtable key for place stat var summary cache

func BuildTriplesKey 

func BuildTriplesKey(dcids []string) bigtable.RowList

BuildTriplesKey builds bigtable key for triples cache

func NewBtTable added in v1.33.0 

func NewBtTable(ctx context.Context, projectID, instanceID, tableID string) (
	*cbt.Table, error)

NewTable creates a new cbt.Table instance.

func Read 

func Read(
	ctx context.Context,
	btGroup *Group,
	rowSet cbt.RowSet,
	action func(string, []byte) (interface{}, error),
	getToken func(string) (string, error),
) ([]map[string]interface{}, error)

Read reads BigTable rows from multiple Bigtable in parallel.

Reading multiple rows is chunked as the size limit for RowSet is 500KB.

Args: baseBt: The bigtable that holds the base cache rowSet: BigTable rowSet containing the row keys. action: A callback function that converts the raw bytes into appropriate 

go struct based on the cache content.

getToken: A function to get back the indexed token (like place dcid) from 

bigtable row key.

func SetupBigtable 

func SetupBigtable(
	ctx context.Context, data map[string]string) (*bigtable.Table, error)

SetupBigtable creates a testing bigtable instance used for testing

func SortTables added in v1.33.0 

func SortTables(tables []*Table)

SortTables sorts the bigtable by import group preferences

  • frequent should always be the highest rank
  • infrequent should always be the lowest rank
  • if a group is not in ranking list, put it right before "infrequent" and after other groups with ranking.

Types

type Group 

type Group struct {
	// contains filtered or unexported fields
}

Group represents all the cloud bigtables that mixer talks to.

func GetFrequentGroup added in v1.35.0 

func GetFrequentGroup(g *Group) *Group

GetFrequentGroup creates a group that only has frequent import group table.

func NewGroup added in v1.33.0 

func NewGroup(
	tables []*Table,
	branchTableName string,
) *Group

NewGroup creates a BigtableGroup

func (*Group) TableNames added in v1.33.0 

func (g *Group) TableNames() []string

TableNames is the accesser to get all the Bigtable names.

func (*Group) Tables added in v1.33.0 

func (g *Group) Tables() []*cbt.Table

Tables is the accessor for all the Bigtable client stubs.

func (*Group) UpdateBranchTable added in v1.33.0 

func (g *Group) UpdateBranchTable(branchTable *Table)

UpdateBranchTable updates the branch Bigtable.

type Table added in v1.33.0 

type Table struct {
	// contains filtered or unexported fields
}

Table holds the bigtable name and client stub.

func NewTable added in v1.33.0 

func NewTable(name string, table *cbt.Table) *Table

NewTable creates a new Table struct.

func (*Table) Name added in v1.33.0 

func (t *Table) Name() string

Name access the name of a table

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.