dosa

package module
v3.4.30+incompatible Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 4, 2020 License: MIT Imports: 19 Imported by: 30

README

DOSA - Declarative Object Storage Abstraction

GoDoc Coverage Status Build Status

Abstract

DOSA is a storage framework that provides a declarative object storage abstraction for applications in Golang and (soon) Java. DOSA is designed to relieve common headaches developers face while building stateful, database-dependent services.

If you'd like to start by writing a small DOSA-enabled program, check out the getting started guide.

Overview

DOSA is a storage library that supports:

  • methods to store and retrieve go structs
  • struct annotations to describe queries against data
  • tools to create and/or migrate database schemas
  • implementations that serialize requests to remote stateless servers

Annotations

This project is released under the MIT License.

Documentation

Overview

Package dosa is the DOSA - Declarative Object Storage Abstraction.

Abstract

DOSA (https://github.com/uber-go/dosa/wiki) is a storage framework that provides a declarative object storage abstraction for applications in Golang and (soon) Java. DOSA is designed to relieve common headaches developers face while building stateful, database-dependent services.

If you'd like to start by writing a small DOSA-enabled program, check out the getting started guide (https://github.com/uber-go/dosa/wiki/Getting-Started-Guide).

Overview

DOSA is a storage library that supports:

• methods to store and retrieve go structs

• struct annotations to describe queries against data

• tools to create and/or migrate database schemas

• implementations that serialize requests to remote stateless servers

Annotations

This project is released under the MIT License (LICENSE.txt).

Index

Examples

Constants

View Source 
const (
	// AdaptiveRangeLimit is a sentinel value that is used to indicate an intent
	// to range over data in a partition as fast as possible. The server will
	// determine an appropriate limit to use to range over the partition as fast
	// as possible while ensuring the server remains healthy.
	AdaptiveRangeLimit = -1
)
View Source 
const DosaNamingRule = "DOSA valid names must start with a letter or underscore, and may contain letters, " +
	"digits, and underscores."

DosaNamingRule is the error message for invalid names.

View Source 
const InvalidVersion = -1

InvalidVersion is an invalid value for a schema version.

View Source 
const MetadataSchemaVersion = 1

MetadataSchemaVersion is the version of the schema of the scope metadata

View Source 
const VERSION = "3.4.29"

VERSION indicates the dosa client version

Variables

View Source 
var ErrNullValue = errors.New("Value is null")

ErrNullValue is returned if a caller tries to call Get() on a nullable primitive value.

Functions

func All 

func All() []string

All is used for "fields []string" to read/update all fields. It's a convenience function for code readability.

func ConditionsString 

func ConditionsString(columnConditions map[string][]*Condition) string

func ConvertConditions 

func ConvertConditions(conditions map[string][]*Condition, t *Table) (map[string][]*Condition, error)

ConvertConditions converts a list of client field names to server side field names

func EnsureValidRangeConditions 

func EnsureValidRangeConditions(ed *EntityDefinition, pk *PrimaryKey, columnConditions map[string][]*Condition, transform func(string) string) error

EnsureValidRangeConditions checks the conditions for a PK Range(). "transform" is a name-prettifying function.

func ErrorIsAlreadyExists 

func ErrorIsAlreadyExists(err error) bool

ErrorIsAlreadyExists checks if the error is caused by "ErrAlreadyExists"

func ErrorIsNotFound 

func ErrorIsNotFound(err error) bool

ErrorIsNotFound checks if the error is a "ErrNotFound" (possibly wrapped)

func ErrorIsNotInitialized 

func ErrorIsNotInitialized(err error) bool

ErrorIsNotInitialized checks if the error is a "ErrNotInitialized" (possibly wrapped)

func ErrorIsRateLimited 

func ErrorIsRateLimited(err error) bool

ErrorIsRateLimited is an error returned when the rate limit is exceeded.

func GetUsername 

func GetUsername() *string

GetUsername returns the username of the current user.

func IsValidName 

func IsValidName(name string) error

IsValidName checks if a string corresponds to DOSA naming rules.

func IsValidNamePrefix 

func IsValidNamePrefix(namePrefix string) error

IsValidNamePrefix checks if a name prefix is valid.

func NoTTL 

func NoTTL() time.Duration

NoTTL returns predefined identifier for not setting TTL

func NormalizeName 

func NormalizeName(name string) (string, error)

NormalizeName normalizes a name to a canonical representation.

func NormalizeNamePrefix 

func NormalizeNamePrefix(name string) (string, error)

NormalizeNamePrefix normalizes a name-prefix to a canonical representation.

func ValidateTTL 

func ValidateTTL(ttl time.Duration) error

ValidateTTL returns whether the TTL is validated or not Any TTL with value of TTL less than 1 second is not allowed except value 0. The TTL (0) means to clear previous TTL associated with the data record and make the record stay permanently.

Types

type AdminClient 

type AdminClient interface {
	// Directories sets admin client search path
	Directories(dirs []string) AdminClient
	// Excludes sets patters to exclude when searching for entities
	Excludes(excludes []string) AdminClient
	// Scope sets the admin client scope
	Scope(scope string) AdminClient
	// CanUpsertSchema checks the compatibility of to-be-upserted schemas
	CanUpsertSchema(ctx context.Context, namePrefix string) (*SchemaStatus, error)
	// CheckSchemaStatus checks the status of schema application
	CheckSchemaStatus(ctx context.Context, namePrefix string, version int32) (*SchemaStatus, error)
	// UpsertSchema upserts the schemas
	UpsertSchema(ctx context.Context, namePrefix string) (*SchemaStatus, error)
	// GetSchema finds entity definitions
	GetSchema() ([]*EntityDefinition, error)
	// CreateScope creates a new scope
	CreateScope(ctx context.Context, md *ScopeMetadata) error
	// TruncateScope keeps the scope and the schemas, but drops the data associated with the scope
	TruncateScope(ctx context.Context, s string) error
	// DropScope drops the scope and the data and schemas in the scope
	DropScope(ctx context.Context, s string) error
	// Shutdown gracefully shuts down the client, cleaning up any resources it may have
	// allocated during its usage. Shutdown should be called whenever the client
	// is no longer needed. After calling shutdown there should be no further usage
	// of the client.
	Shutdown() error
}

AdminClient has methods to manage schemas and scopes

func NewAdminClient 

func NewAdminClient(conn Connector) AdminClient

NewAdminClient returns a new DOSA admin client for the connector provided.

type Client 

type Client interface {
	// GetRegistrar returns the registrar
	GetRegistrar() Registrar

	// Initialize must be called before any data operation
	Initialize(ctx context.Context) error

	// Create creates an entity; it fails if the entity already exists.
	// You must fill in all of the fields of the DomainObject before
	// calling this method, or they will be inserted with the zero value
	// This is a relatively expensive operation. Use Upsert whenever possible.
	CreateIfNotExists(ctx context.Context, objectToCreate DomainObject) error

	// Read fetches a row by primary key. A list of fields to read can be
	// specified. Use All() or nil for all fields.
	// Before calling this method, fill in the DomainObject with ALL
	// of the primary key fields; the other field values will be populated
	// as a result of the read
	Read(ctx context.Context, fieldsToRead []string, objectToRead DomainObject) error

	// MultiRead fetches several rows by primary key. A list of fields can be
	// specified. Use All() or nil for all fields. All entities MUST be on the same partition.
	// The domainObject will be filled by corresponding values if the object is fetched successfully.
	// Otherwise the DomainObject as key and an error message as value will be saved into
	// MultiResult map.
	// NOTE: This API only fetches objects of same entity type from same scope.
	//
	// Range should be preferred over MultiRead. If you are not sure which endpoint to use,
	// use Range instead of MultiRead.
	MultiRead(context.Context, []string, ...DomainObject) (MultiResult, error)

	// Upsert creates or update a row. A list of fields to update can be
	// specified. Use All() or nil for all fields.
	// Before calling this method, fill in the DomainObject with ALL
	// of the primary key fields, along with whatever fields you specify
	// to update in fieldsToUpdate (or all the fields if you use dosa.All())
	Upsert(ctx context.Context, fieldsToUpdate []string, objectToUpdate DomainObject) error

	// Remove removes a row by primary key. The passed-in entity should contain
	// the primary key field values, all other fields are ignored.
	Remove(ctx context.Context, objectToRemove DomainObject) error

	// RemoveRange removes all of the rows that fall within the range specified by the
	// given RemoveRangeOp. Note that only Primary Key queries may be used here.
	RemoveRange(ctx context.Context, removeRangeOp *RemoveRangeOp) error

	// Range fetches entities within a range
	// Before calling range, create a RangeOp and fill in the table
	// along with the partition key information. You will get back
	// an array of DomainObjects, which will be of the type you requested
	// in the rangeOp.
	//
	// Range only fetches a portion of the range at a time (the size of that portion is defined
	// by the Limit parameter of the RangeOp). A continuation token is returned so subsequent portions
	// of the range can be fetched with additional calls to the range function.
	Range(ctx context.Context, rangeOp *RangeOp) ([]DomainObject, string, error)

	// WalkRange starts at the offset specified by the RangeOp and walks the entire
	// range of values that fall within the RangeOp conditions. It will make multiple, sequential
	// range requests, fetching values until there are no more left in the range.
	//
	// For each value fetched, the provided onNext function is called with the value as it's argument.
	WalkRange(ctx context.Context, r *RangeOp, onNext func(value DomainObject) error) error

	// ScanEverything fetches all entities of a type
	// Before calling ScanEverything, create a scanOp to specify the
	// table to scan. The return values are an array of objects, that
	// you can type-assert to the appropriate dosa.Entity, a string
	// that contains the continuation token, and any error.
	// To scan the next set of rows, modify the scanOp to provide
	// the string returned as an Offset()
	ScanEverything(ctx context.Context, scanOp *ScanOp) ([]DomainObject, string, error)

	// Shutdown gracefully shuts down the client, cleaning up any resources it may have
	// allocated during its usage. Shutdown should be called whenever the client
	// is no longer needed. After calling shutdown there should be no further usage
	// of the client.
	Shutdown() error
}

Client defines the methods to operate with DOSA entities

func NewClient 

func NewClient(reg Registrar, conn Connector) Client

NewClient returns a new DOSA client for the registrar and connector provided. This is currently only a partial implementation to demonstrate basic CRUD functionality.

Example

ExampleNewClient initializes a client using the devnull connector, which discards all the data you send it and always returns no rows. It's only useful for testing dosa. 

package main

import (
	"context"
	"fmt"

	dosaRenamed "github.com/uber-go/dosa"
	"github.com/uber-go/dosa/connectors/devnull"
)

type ClientTestEntity1 struct {
	dosaRenamed.Entity `dosa:"primaryKey=(ID)"`
	dosaRenamed.Index  `dosa:"key=Name, name=username"`
	ID                 int64
	Name               string
	Email              string
}

type ClientTestEntity2 struct {
	dosaRenamed.Entity `dosa:"primaryKey=(UUID,Color)"`
	SearchByColor      dosaRenamed.Index `dosa:"key=Color"`
	UUID               string
	Color              string
	IsActive           bool
	ignoreme           int32
}

var cte1 = &ClientTestEntity1{ID: int64(1), Name: "foo", Email: "foo@uber.com"}

func main() {
	// initialize registrar
	reg, err := dosaRenamed.NewRegistrar("test", "myteam.myservice", cte1)
	if err != nil {
		// registration will fail if the object is tagged incorrectly
		fmt.Printf("NewRegistrar error: %s", err)
		return
	}

	// use a devnull connector for example purposes
	conn := devnull.NewConnector()

	// create the client using the registrar and connector
	client := dosaRenamed.NewClient(reg, conn)

	err = client.Initialize(context.Background())
	if err != nil {
		fmt.Printf("Initialize error: %s", err)
		return
	}
}

Output:

type ClusteringKey 

type ClusteringKey struct {
	Name       string
	Descending bool
}

ClusteringKey stores name and ordering of a clustering key

func (ClusteringKey) String 

func (ck ClusteringKey) String() string

String takes a ClusteringKey and returns "column-name ASC|DESC"

type ColumnCondition 

type ColumnCondition struct {
	Name      string
	Condition *Condition
}

ColumnCondition represents the condition of each column

func NormalizeConditions 

func NormalizeConditions(columnConditions map[string][]*Condition) []*ColumnCondition

NormalizeConditions takes a set of conditions for columns and returns a sorted, denormalized view of the conditions.

func (*ColumnCondition) String 

func (cc *ColumnCondition) String() string

type ColumnDefinition 

type ColumnDefinition struct {
	Name      string // normalized column name
	Type      Type
	IsPointer bool // used by client only to indicate whether this field is pointer
	Tags      map[string]string
}

ColumnDefinition stores information about a column

func (*ColumnDefinition) Clone 

func (cd *ColumnDefinition) Clone() *ColumnDefinition

Clone returns a deep copy of ColumnDefinition

func (*ColumnDefinition) String 

func (cd *ColumnDefinition) String() string

type Condition 

type Condition struct {
	Op    Operator
	Value FieldValue
}

Condition holds an operator and a value for a condition on a field.

type Connector 

type Connector interface {
	// DML operations (CRUD + range + scan)
	// CreateIfNotExists creates a row, but only if it does not exist.
	CreateIfNotExists(ctx context.Context, ei *EntityInfo, values map[string]FieldValue) error
	// Read fetches a row by primary key
	// If minimumFields is empty or nil, all non-key fields would be fetched.
	Read(ctx context.Context, ei *EntityInfo, keys map[string]FieldValue, minimumFields []string) (values map[string]FieldValue, err error)
	// MultiRead fetches several rows by primary key
	// If minimumFields is empty or nil, all non-key fields would be fetched.
	MultiRead(ctx context.Context, ei *EntityInfo, keys []map[string]FieldValue, minimumFields []string) (results []*FieldValuesOrError, err error)
	// Upsert updates some columns of a row, or creates a new one if it doesn't exist yet.
	Upsert(ctx context.Context, ei *EntityInfo, values map[string]FieldValue) error
	// MultiUpsert updates some columns of several rows, or creates a new ones if they doesn't exist yet
	MultiUpsert(ctx context.Context, ei *EntityInfo, multiValues []map[string]FieldValue) (result []error, err error)
	// Remove deletes a row
	Remove(ctx context.Context, ei *EntityInfo, keys map[string]FieldValue) error
	// RemoveRange removes all entities in a particular range, only Primary Key queries are allowed.
	RemoveRange(ctx context.Context, ei *EntityInfo, columnConditions map[string][]*Condition) error
	// MultiRemove removes multiple rows
	MultiRemove(ctx context.Context, ei *EntityInfo, multiKeys []map[string]FieldValue) (result []error, err error)
	// Range does a range scan using a set of conditions.
	// If minimumFields is empty or nil, all fields (including key fields) would be fetched.
	Range(ctx context.Context, ei *EntityInfo, columnConditions map[string][]*Condition, minimumFields []string, token string, limit int) ([]map[string]FieldValue, string, error)
	// Scan reads the whole table, for doing a sequential search or dump/load use cases
	// If minimumFields is empty or nil, all fields (including key fields) would be fetched.
	Scan(ctx context.Context, ei *EntityInfo, minimumFields []string, token string, limit int) (multiValues []map[string]FieldValue, nextToken string, err error)

	// DDL (schema) operations
	// CheckSchema makes sure that the schema provided is compatible with the version on the database.
	// It returns the latest schema version for use with later DML operations.
	// This method should have been called something like 'IsSchemaCompatibleWithDB'.
	CheckSchema(ctx context.Context, scope string, namePrefix string, eds []*EntityDefinition) (version int32, err error)
	// CanUpsertSchema checks whether the new schema is backwards-compatible with the schema on the DB.
	// If compatible, the latest schema version is returned, which can be "invalid" (i.e. -1) if no schema
	// has been applied yet.
	CanUpsertSchema(ctx context.Context, scope string, namePrefix string, eds []*EntityDefinition) (version int32, err error)
	// UpsertSchema updates the schema to the one provided.
	UpsertSchema(ctx context.Context, scope string, namePrefix string, ed []*EntityDefinition) (status *SchemaStatus, err error)
	// CheckSchemaStatus checks the status of a schema upsert: whether it is accepted or in progress of application.
	CheckSchemaStatus(ctx context.Context, scope string, namePrefix string, version int32) (*SchemaStatus, error)
	// GetEntitySchema returns the entity info for a given entity in a given scope and prefix.
	GetEntitySchema(ctx context.Context, scope, namePrefix, entityName string, version int32) (*EntityDefinition, error)

	// Datastore management
	// CreateScope creates a scope for storage of data, usually implemented by a keyspace for this data
	// This is usually followed by UpsertSchema
	CreateScope(ctx context.Context, md *ScopeMetadata) error
	// TruncateScope keeps the scope around, but removes all the data
	TruncateScope(ctx context.Context, scope string) error
	// DropScope removes the scope and all of the data
	DropScope(ctx context.Context, scope string) error
	// ScopeExists checks whether a scope exists or not
	ScopeExists(ctx context.Context, scope string) (bool, error)

	// Shutdown finishes the connector to do clean up work
	Shutdown() error
}

Connector is the interface that must be implemented for a backend service It can also be implemented using an RPC such as thrift (dosa-idl) When fields are returned from read/range/scan methods, it's legal for the connector to return more fields than originally requested. The caller of the connector should never mutate the returned columns either, in case they are from a cache

type DomainIndex 

type DomainIndex interface {
	// contains filtered or unexported methods
}

DomainIndex is a marker interface method for an Index

type DomainObject 

type DomainObject interface {
	// contains filtered or unexported methods
}

DomainObject is a marker interface method for an Entity

type ETLState 

type ETLState string

ETLState is a custom type based on the entity ETL tag 

const (
	// EtlOn means ETL on the entity is turn ON
	EtlOn ETLState = "on"
	// EtlOff means ETL on the entity is turn OFF
	EtlOff ETLState = "off"
)

func ToETLState 

func ToETLState(s string) (ETLState, error)

ToETLState converts the etl tag value (in string) to the corresponding ETLState

type Entity 

type Entity struct {
	// contains filtered or unexported fields
}

Entity represents any object that can be persisted by DOSA

func (*Entity) TTL 

func (e *Entity) TTL(t *time.Duration)

TTL sets dynamic ttl to an entity

type EntityDefinition 

type EntityDefinition struct {
	Name    string // normalized entity name
	Key     *PrimaryKey
	Columns []*ColumnDefinition
	Indexes map[string]*IndexDefinition
	ETL     ETLState
}

EntityDefinition stores information about a DOSA entity

func (*EntityDefinition) CanBeUpsertedOn 

func (e *EntityDefinition) CanBeUpsertedOn(older *EntityDefinition) error

CanBeUpsertedOn checks upsertability: Can I be upserted on top of the prior definition?

func (*EntityDefinition) Clone 

func (e *EntityDefinition) Clone() *EntityDefinition

Clone returns a deep copy of EntityDefinition

func (*EntityDefinition) ColumnMap 

func (e *EntityDefinition) ColumnMap() map[string]*ColumnDefinition

ColumnMap returns a map of column name to column definition for all columns.

func (*EntityDefinition) ColumnTypes 

func (e *EntityDefinition) ColumnTypes() map[string]Type

ColumnTypes returns a map of column name to column type for all columns.

func (*EntityDefinition) EnsureValid 

func (e *EntityDefinition) EnsureValid() error

EnsureValid ensures the entity definition is valid. All the names used (entity name, column name) must be valid. No duplicate names can be used in column names or key names. The primary key must not be nil and must contain at least one partition key.

func (*EntityDefinition) FindColumnDefinition 

func (e *EntityDefinition) FindColumnDefinition(name string) *ColumnDefinition

FindColumnDefinition finds the column definition by the column name

func (*EntityDefinition) KeySet 

func (e *EntityDefinition) KeySet() map[string]struct{}

KeySet returns a set of all keys, including partition keys and clustering keys.

func (*EntityDefinition) PartitionKeySet 

func (e *EntityDefinition) PartitionKeySet() map[string]struct{}

PartitionKeySet returns a set of all partition keys.

func (*EntityDefinition) String 

func (e *EntityDefinition) String() string

func (*EntityDefinition) UniqueKey 

func (e *EntityDefinition) UniqueKey(oldKey *PrimaryKey) *PrimaryKey

UniqueKey adds any missing keys from the entity's primary key to the keys specified in the index, to guarantee that the returned key is unique This method is used to create materialized views

type EntityErrors 

type EntityErrors struct {
	// contains filtered or unexported fields
}

EntityErrors is a container for parse errors/warning.

func NewEntityErrors 

func NewEntityErrors(warns []error) *EntityErrors

NewEntityErrors returns a wrapper for errors encountered while parsing entity struct tags.

func (*EntityErrors) Error 

func (ee *EntityErrors) Error() string

Error makes parse errors discernable to end-user.

type EntityInfo 

type EntityInfo struct {
	Ref *SchemaRef
	Def *EntityDefinition
	TTL *time.Duration
}

EntityInfo is all the information about an entity, including the schema reference as well as the entity definition

func (*EntityInfo) IndexFromConditions 

func (ei *EntityInfo) IndexFromConditions(conditions map[string][]*Condition, searchIndexes bool) (name string, key *PrimaryKey, err error)

IndexFromConditions returns the name of the index or the base table to use, along with the key info for that index. If no suitable index could be found, an error is returned

func (*EntityInfo) String 

func (ei *EntityInfo) String() string

type ErrAlreadyExists 

type ErrAlreadyExists struct{}

ErrAlreadyExists is an error returned when CreateIfNotExists but a row already exists

func (*ErrAlreadyExists) Error 

func (*ErrAlreadyExists) Error() string

type ErrNotFound 

type ErrNotFound struct{}

ErrNotFound is an error when a row is not found (single or multiple)

func (*ErrNotFound) Error 

func (*ErrNotFound) Error() string

Error returns a constant string "Not found" for this error

type ErrNotInitialized 

type ErrNotInitialized struct{}

ErrNotInitialized is returned when a user didn't call Initialize

func (*ErrNotInitialized) Error 

func (*ErrNotInitialized) Error() string

Error returns a constant string "client not initialized"

type ErrRateLimited 

type ErrRateLimited struct{}

ErrRateLimited is an error returned when the rate limit is exceeded.

func (*ErrRateLimited) Error 

func (*ErrRateLimited) Error() string

type FieldNameValuePair 

type FieldNameValuePair struct {
	Name  string
	Value FieldValue
}

FieldNameValuePair is a field name and value

type FieldValue 

type FieldValue interface{}

FieldValue holds a field value. It's just a marker.

type FieldValuesOrError 

type FieldValuesOrError struct {
	Values map[string]FieldValue
	Error  error
}

FieldValuesOrError either holds a slice of field values for a row, or an error

type Index 

type Index struct{}

Index represents any object that can be indexed by by DOSA

type IndexDefinition 

type IndexDefinition struct {
	Key       *PrimaryKey
	Columns   []string
	TableName string // May be empty
}

IndexDefinition stores information about a DOSA entity's index

func (*IndexDefinition) Clone 

func (id *IndexDefinition) Clone() *IndexDefinition

Clone returns a deep copy of IndexDefinition

func (*IndexDefinition) String 

func (id *IndexDefinition) String() string

type MultiResult 

type MultiResult map[DomainObject]error

MultiResult contains the result for each entity operation in the case of MultiRead. If the operation succeeded for an entity, the value for in the map will be nil; otherwise, the entity is untouched and error is not nil.

type Operator 

type Operator int

Operator defines an operator against some data for range scans 

const (

	// Eq is the equals operator
	Eq Operator

	// Lt is the less than operator
	Lt

	// LtOrEq is the less than or equal operator
	LtOrEq

	// Gt is the greater than operator
	Gt

	// GtOrEq is the greater than or equal operator
	GtOrEq
)

func (Operator) String 

func (op Operator) String() string

func (Operator) Symbol 

func (op Operator) Symbol() string

type PrimaryKey 

type PrimaryKey struct {
	PartitionKeys  []string
	ClusteringKeys []*ClusteringKey
}

PrimaryKey stores information about partition keys and clustering keys

func (PrimaryKey) Clone 

func (pk PrimaryKey) Clone() *PrimaryKey

Clone returns a deep copy of PrimaryKey

func (PrimaryKey) ClusteringKeySet 

func (pk PrimaryKey) ClusteringKeySet() map[string]struct{}

ClusteringKeySet returns a set of all clustering keys.

func (PrimaryKey) PartitionKeySet 

func (pk PrimaryKey) PartitionKeySet() map[string]struct{}

PartitionKeySet returns the set of partition keys

func (PrimaryKey) PrimaryKeySet 

func (pk PrimaryKey) PrimaryKeySet() map[string]struct{}

PrimaryKeySet returns the union of the set of partition keys and clustering keys

func (PrimaryKey) String 

func (pk PrimaryKey) String() string

type RangeOp 

type RangeOp struct {
	// contains filtered or unexported fields
}

RangeOp is used to specify constraints to Range calls

func NewRangeOp 

func NewRangeOp(object DomainObject) *RangeOp

NewRangeOp returns a new RangeOp instance

func (*RangeOp) Conditions 

func (r *RangeOp) Conditions() map[string][]*Condition

Conditions returns all conditions embedded in the range operator

func (*RangeOp) Eq 

func (r *RangeOp) Eq(fieldName string, value interface{}) *RangeOp

Eq is used to express an equality constraint for a range query

func (*RangeOp) Fields 

func (r *RangeOp) Fields(fields []string) *RangeOp

Fields list the non-key fields users want to fetch. PrimaryKey fields are always fetched.

func (*RangeOp) Gt 

func (r *RangeOp) Gt(fieldName string, value interface{}) *RangeOp

Gt is used to express an "greater than" constraint for a range query

func (*RangeOp) GtOrEq 

func (r *RangeOp) GtOrEq(fieldName string, value interface{}) *RangeOp

GtOrEq is used to express an "greater than or equal" constraint for a range query

func (*RangeOp) Limit 

func (r *RangeOp) Limit(n int) *RangeOp

Limit sets the number of rows returned per call. Default is 100

func (*RangeOp) LimitRows 

func (r *RangeOp) LimitRows() int

LimitRows returns number of rows to return per call

func (*RangeOp) Lt 

func (r *RangeOp) Lt(fieldName string, value interface{}) *RangeOp

Lt is used to express a "less than" constraint for a range query

func (*RangeOp) LtOrEq 

func (r *RangeOp) LtOrEq(fieldName string, value interface{}) *RangeOp

LtOrEq is used to express a "less than or equal" constraint for a range query

func (*RangeOp) Offset 

func (r *RangeOp) Offset(token string) *RangeOp

Offset sets the pagination token. If not set, an empty token would be used.

func (*RangeOp) String 

func (r *RangeOp) String() string

String satisfies the Stringer interface

type RegisteredEntity 

type RegisteredEntity struct {
	// contains filtered or unexported fields
}

RegisteredEntity is the structure that holds all information necessary for performing operations on an entity as well as helper methods for accessing type data so that reflection can be minimized.

func NewRegisteredEntity 

func NewRegisteredEntity(scope, namePrefix string, table *Table) *RegisteredEntity

NewRegisteredEntity is a constructor for creating a RegisteredEntity

func (*RegisteredEntity) ColumnNames 

func (e *RegisteredEntity) ColumnNames(fieldNames []string) ([]string, error)

ColumnNames translates field names to column names.

func (*RegisteredEntity) EntityDefinition 

func (e *RegisteredEntity) EntityDefinition() *EntityDefinition

EntityDefinition is a helper for accessing the registered entity's EntityDefinition instance.

func (*RegisteredEntity) EntityInfo 

func (e *RegisteredEntity) EntityInfo() *EntityInfo

EntityInfo is a helper for accessing the registered entity's EntityInfo instance which is required by clients to call connector methods.

func (*RegisteredEntity) KeyFieldValues 

func (e *RegisteredEntity) KeyFieldValues(entity DomainObject) map[string]FieldValue

KeyFieldValues is a helper for generating a map of field values to be used in a query.

func (*RegisteredEntity) OnlyFieldValues 

func (e *RegisteredEntity) OnlyFieldValues(entity DomainObject, fieldNames []string) (map[string]FieldValue, error)

OnlyFieldValues is a helper for generating a map of field values for a a subset of fields. If a field name provided does not map to an entity field, an error will be returned.

func (*RegisteredEntity) SchemaRef 

func (e *RegisteredEntity) SchemaRef() *SchemaRef

SchemaRef is a helper for accessing the registered entity's SchemaRef instance.

func (*RegisteredEntity) SetFieldValues 

func (e *RegisteredEntity) SetFieldValues(entity DomainObject, fieldValues map[string]FieldValue, fieldsToRead []string)

SetFieldValues is a helper for populating a DOSA entity with the given fieldName->value map

func (*RegisteredEntity) SetVersion 

func (e *RegisteredEntity) SetVersion(v int32)

SetVersion sets the current schema version on the registered entity.

func (*RegisteredEntity) Table 

func (e *RegisteredEntity) Table() *Table

Table is a helper for accessing the registered entity's Table instance.

type Registrar 

type Registrar interface {
	Scope() string
	NamePrefix() string
	Find(DomainObject) (*RegisteredEntity, error)
	FindAll() []*RegisteredEntity
}

Registrar is the interface to register DOSA entities.

func NewRegistrar 

func NewRegistrar(scope, namePrefix string, entities ...DomainObject) (Registrar, error)

NewRegistrar returns a new Registrar for the scope, name prefix and entities provided. `dosa.Client` implementations are intended to use scope and prefix to uniquely identify where entities should live but the registrar itself is only responsible for basic accounting of entities.

type RemoveRangeOp 

type RemoveRangeOp struct {
	// contains filtered or unexported fields
}

RemoveRangeOp is used to specify contraints on RemoveRange calls

func NewRemoveRangeOp 

func NewRemoveRangeOp(object DomainObject) *RemoveRangeOp

NewRemoveRangeOp returns a new RangeOp instance

func (*RemoveRangeOp) Eq 

func (r *RemoveRangeOp) Eq(fieldName string, value interface{}) *RemoveRangeOp

Eq is used to express an equality constraint for a remove range operation

func (*RemoveRangeOp) Gt 

func (r *RemoveRangeOp) Gt(fieldName string, value interface{}) *RemoveRangeOp

Gt is used to express an "greater than" constraint for a remove range operation

func (*RemoveRangeOp) GtOrEq 

func (r *RemoveRangeOp) GtOrEq(fieldName string, value interface{}) *RemoveRangeOp

GtOrEq is used to express an "greater than or equal" constraint for a remove range operation

func (*RemoveRangeOp) Lt 

func (r *RemoveRangeOp) Lt(fieldName string, value interface{}) *RemoveRangeOp

Lt is used to express a "less than" constraint for a remove range operation

func (*RemoveRangeOp) LtOrEq 

func (r *RemoveRangeOp) LtOrEq(fieldName string, value interface{}) *RemoveRangeOp

LtOrEq is used to express a "less than or equal" constraint for a remove range operation

type ScanOp 

type ScanOp struct {
	// contains filtered or unexported fields
}

ScanOp represents the scan query

func NewScanOp 

func NewScanOp(obj DomainObject) *ScanOp

NewScanOp returns a new ScanOp instance

func (*ScanOp) Fields 

func (s *ScanOp) Fields(fields []string) *ScanOp

Fields list the non-key fields users want to fetch. PrimaryKey fields are always fetched.

func (*ScanOp) Limit 

func (s *ScanOp) Limit(n int) *ScanOp

Limit sets the number of rows returned per call. Default is 100

func (*ScanOp) Offset 

func (s *ScanOp) Offset(token string) *ScanOp

Offset sets the pagination token. If not set, an empty token would be used.

func (*ScanOp) String 

func (s *ScanOp) String() string

String satisfies the Stringer interface

type SchemaRef 

type SchemaRef struct {
	Scope      string
	NamePrefix string
	EntityName string
	IndexName  string
	Version    int32
}

SchemaRef is a reference to the table and schema version of an object

type SchemaStatus 

type SchemaStatus struct {
	// the version of the schema
	Version int32
	// the application status of the schema
	Status string
}

SchemaStatus saves the version and application status of a schema

type ScopeFlagType 

type ScopeFlagType int64

ScopeFlagType is a set of scope flags 

const (
	// AccessFromProd means access is allowed from production (only relevant for Dev scopes)
	AccessFromProd ScopeFlagType = 1 << iota
)

Scope flags (remember to update ScopeFlagType.String)

func (ScopeFlagType) String 

func (f ScopeFlagType) String() string

type ScopeMetadata 

type ScopeMetadata struct {
	Entity      `dosa:"primaryKey=(Name)" json:"-"`
	Name        string     `json:"name"`
	Owner       string     `json:"owner"` // Owning group name (or the same as Creator)
	Type        int32      `json:"type"`  // Production, Staging, or Development
	Flags       int64      `json:"flags"`
	Version     int32      `json:"version"`
	PrefixStr   string     `json:"prefix_str,omitempty"` // With ":" separators
	Cluster     string     `json:"cluster,omitempty"`    // Host DB cluster
	Creator     string     `json:"creator"`
	CreatedOn   time.Time  `json:"created_on"`
	ExpiresOn   *time.Time `json:"expires_on,omitempty"`
	ExtendCount int32      `json:"extend_count,omitempty"`
	NotifyCount int32      `json:"notify_count,omitempty"`
	ReadMaxRPS  int32      `json:"read_max_rps"`
	WriteMaxRPS int32      `json:"write_max_rps"`
	// This is for convenience only, not stored in the DB:
	Prefixes StringSet `dosa:"-" json:"-"`
}

ScopeMetadata is metadata about a scope. (JSON tags to support MD setting CLI tools.) The scope may be qualified by a prefix, as in "production.vsoffers".

func (*ScopeMetadata) String 

func (md *ScopeMetadata) String() string

type ScopeType 

type ScopeType int32

ScopeType is the type of a scope 

const (
	// Development scope (also the default, so should be 0)
	Development ScopeType = iota
	// Staging doesn't really exist yet, but may in the future
	Staging
	// Production scope
	Production
)

Scope types

func (ScopeType) String 

func (t ScopeType) String() string

type StringSet 

type StringSet map[string]struct{}

StringSet is a set of strings.

func (StringSet) String 

func (s StringSet) String() string

type Table 

type Table struct {
	EntityDefinition
	StructName string
	ColToField map[string]string // map from column name -> field name
	FieldToCol map[string]string // map from field name -> column name
	TTL        time.Duration
}

Table represents a parsed entity format on the client side In addition to shared EntityDefinition, it records struct name and field names.

func FindEntityByName 

func FindEntityByName(path string, structName string) (*Table, error)

FindEntityByName returns the entity with given name in the path.

func TableFromInstance 

func TableFromInstance(object DomainObject) (*Table, error)

TableFromInstance creates a dosa.Table from a dosa.DomainObject instance. Note: this method is not cheap as it does a lot of reflection to build the Table instances. It is recommended to only be called once and cache results.

func (Table) String 

func (d Table) String() string

type Type 

type Type int

Type defines a data type for an entity field 

const (
	// Invalid type to be used as zero value for Type
	Invalid Type = iota

	// TUUID is different from dosa.UUID
	TUUID

	// String represents a string
	String

	// Int32 represents an int32
	Int32

	// Int64 represents either an int64 or a regular int
	Int64

	// Double is a float64
	Double

	// Blob is a byte slice
	Blob

	// Timestamp is a time.Time
	Timestamp

	// Bool is a bool type
	Bool
)

func FromString 

func FromString(s string) Type

FromString converts string to dosa Type

func (Type) String 

func (i Type) String() string

type UUID 

type UUID string

UUID stores a string format of uuid. Validation is done before saving to datastore. The format of uuid used in datastore is orthogonal to the string format here.

func BytesToUUID 

func BytesToUUID(bs []byte) (UUID, error)

BytesToUUID creates a UUID from a byte slice

func NewUUID 

func NewUUID() UUID

NewUUID is a helper for returning a new dosa.UUID value

func (UUID) Bytes 

func (u UUID) Bytes() ([]byte, error)

Bytes gets the bytes from a UUID

Source Files

View all Source files

Directories

Path Synopsis
cmd
Package cmd contains all standalone binaries
 Package cmd contains all standalone binaries
dosa
Package main is the entrypoint for the dosa CLI.
 Package main is the entrypoint for the dosa CLI.
base
cache
devnull
memory
random
redis
routing
yarpc
Package examples is the Runnable examples.
 Package examples is the Runnable examples.
testing
Package testingexamples is the Examples: Testing.
 Package testingexamples is the Examples: Testing.
Package mocks is a generated GoMock package.
 Package mocks is a generated GoMock package.
schema
avro
cql
uql

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.