model

package
v0.0.0-...-1643519 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 16, 2024 License: Apache-2.0 Imports: 26 Imported by: 0

Documentation

Overview

Package model contains core CIPD datastore entities.

Index

Constants

View Source 
const (
	// Hidden can be used in place of 'true' when working with Package.Hidden flag.
	Hidden = true
	// Visible can be used in place of 'false' when working with Package.Hidden flag.
	Visible = false
)

Variables

This section is empty.

Functions

func AttachTags 

func AttachTags(c context.Context, inst *Instance, tags []*api.Tag, who identity.Identity) error

AttachTags transactionally attaches a bunch of tags to an instance.

Assumes inputs are already validated. Launches a transaction inside (and thus can't be a part of a transaction itself). Updates 'inst' in-place with the most recent instance state.

Returns gRPC-tagged errors: 

NotFound if there's no such instance or package.
FailedPrecondition if some processors are still running.
Aborted if some processors have failed.
Internal on tag ID collision.

func CheckInstanceExists 

func CheckInstanceExists(c context.Context, inst *Instance) error

CheckInstanceExists fetches the instance and verifies it exists.

Can be called as part of a transaction. Updates 'inst' in place.

Returns gRPC-tagged NotFound error if there's no such instance or package.

func CheckInstanceReady 

func CheckInstanceReady(c context.Context, inst *Instance) error

CheckInstanceReady fetches the instance and verifies it exists and has zero pending or failed processors.

Can be called as part of a transaction. Updates 'inst' in place.

Returns gRPC-tagged errors: 

NotFound if there's no such instance or package.
FailedPrecondition if some processors are still running.
Aborted if some processors have failed.

func CheckPackageExists 

func CheckPackageExists(c context.Context, pkg string) error

CheckPackageExists verifies the package exists.

Returns gRPC-tagged NotFound error if there's no such package.

func CheckPackages 

func CheckPackages(c context.Context, names []string, includeHidden bool) ([]string, error)

CheckPackages given a list of package names returns packages that exist, in the order they are listed in the list.

If includeHidden is false, omits hidden packages from the result.

Returns only transient errors.

func DeletePackage 

func DeletePackage(c context.Context, pkg string) error

DeletePackage deletes all entities associated with a package.

Returns grpc-tagged errors (in particular NotFound if there's no such package).

func DeleteRef 

func DeleteRef(c context.Context, pkg, ref string) error

DeleteRef removes the ref if it exists.

Does nothing if there's no such ref or package.

func DetachTags 

func DetachTags(c context.Context, inst *Instance, tags []*api.Tag) error

DetachTags detaches a bunch of tags from an instance.

Assumes inputs are already validated. Launches a transaction inside (and thus can't be a part of a transaction itself).

'inst' is used only for its key.

func FetchProcessors 

func FetchProcessors(c context.Context, inst *Instance) ([]*api.Processor, error)

FetchProcessors fetches results of all processors assigned to the instance and returns them as cipd.Processor proto messages (sorted by processor ID).

func ListPackages 

func ListPackages(c context.Context, prefix string, includeHidden bool) (out []string, err error)

ListPackages returns a list of names of packages under the given prefix.

Lists all packages recursively. If there's package named as 'prefix' it is NOT included in the result. Only packaged under the prefix are included.

The result is sorted by the package name. Returns only transient errors.

func PackageKey 

func PackageKey(c context.Context, pkg string) *datastore.Key

PackageKey returns a datastore key of some package, given its name.

func ResolveTag 

func ResolveTag(c context.Context, pkg string, tag *api.Tag) (string, error)

ResolveTag searches for a given tag among all instances of the package and returns an ID of the instance the tag is attached to.

Assumes inputs are already validated. Doesn't double check the instance exists.

Returns gRPC-tagged errors: 

NotFound if there's no such tag at all.
FailedPrecondition if the tag resolves to multiple instances.

func SetPackageHidden 

func SetPackageHidden(c context.Context, pkg string, hidden bool) error

SetPackageHidden updates Hidden field of the package.

If the package is missing returns datastore.ErrNoSuchEntity. All other errors are transient.

func SetRef 

func SetRef(c context.Context, ref string, inst *Instance, who identity.Identity) error

SetRef moves or creates a ref.

Assumes inputs are already validated. Launches a transaction inside (and thus can't be a part of a transaction itself). Updates 'inst' in-place with the most recent instance state.

Returns gRPC-tagged errors: 

NotFound if there's no such instance or package.
FailedPrecondition if some processors are still running.
Aborted if some processors have failed.

func TagID 

func TagID(t *api.Tag) string

TagID calculates Tag entity ID (SHA1 digest) from the given tag.

Panics if the tag is invalid.

func Txn 

func Txn(c context.Context, name string, cb func(context.Context) error) error

Txn runs the callback in a datastore transaction, marking commit errors with transient tag.

The given name will be used for logging and error messages.

Types

type Instance 

type Instance struct {
	InstanceID string         `gae:"$id"`     // see common.ObjectRefToInstanceID()
	Package    *datastore.Key `gae:"$parent"` // see PackageKey()

	RegisteredBy string    `gae:"registered_by"` // who registered it
	RegisteredTs time.Time `gae:"registered_ts"` // when it was registered

	// Names of currently running or scheduled processors.
	ProcessorsPending []string `gae:"processors_pending"`
	// Names of processors that successfully finished the processing.
	ProcessorsSuccess []string `gae:"processors_success"`
	// Names of processors that returned fatal errors.
	ProcessorsFailure []string `gae:"processors_failure"`
	// contains filtered or unexported fields
}

Instance represents a package instance as it is stored in the datastore.

Contains some instance metadata, in particular a list of processors that scanned the instance (see below).

The parent entity is the corresponding package entity. ID is derived from package instance file digest, see common.ObjectRefToInstanceID().

Compatible with the python version of the backend.

func ListInstances 

func ListInstances(c context.Context, pkg string, pageSize int32, cursor datastore.Cursor) (out []*Instance, nextCur datastore.Cursor, err error)

ListInstances lists instances of a package, more recent first.

Only does a query over Instances entities. Doesn't check whether the Package entity exists. Returns up to pageSize entities, plus non-nil cursor (if there are more results). pageSize must be positive.

func RegisterInstance 

func RegisterInstance(c context.Context, inst *Instance, cb func(context.Context, *Instance) error) (reg bool, out *Instance, err error)

RegisterInstance transactionally registers an instance (and the corresponding package), if it isn't registered already.

Calls the given callback (inside the transaction) if it is indeed registering a new instance. The callback may mutate the instance entity before it is stored. The callback may be called multiple times in case of retries (each time it will be given a fresh instance to be mutated).

Returns (true, entity, nil) if the instance was registered just now or (false, entity, nil) if it existed before.

In either case, it returns the entity that is stored now in the datastore. It is either the new instance, or something that existed there before.

func ResolveVersion 

func ResolveVersion(c context.Context, pkg, version string) (*Instance, error)

ResolveVersion takes a version identifier (an instance ID, a ref or a tag) and resolves it into a concrete package instance, ensuring it exists.

Returns gRPC-tagged errors: 

InvalidArgument if the version string format is invalid.
NotFound if there's no such package or such version.
FailedPrecondition if the tag resolves to multiple instances.

func SearchInstances 

func SearchInstances(c context.Context, pkg string, tags []*api.Tag, pageSize int32, cursor datastore.Cursor) (out []*Instance, nextCur datastore.Cursor, err error)

SearchInstances lists instances of a package with all given tags attached.

Only does a query over Instances entities. Doesn't check whether the Package entity exists. Returns up to pageSize entities, plus non-nil cursor (if there are more results). 'pageSize' must be positive.

func (*Instance) FromProto 

func (e *Instance) FromProto(c context.Context, p *api.Instance) *Instance

FromProto fills in the entity based on the proto message.

Returns the entity itself for easier chaining.

Doesn't touch output-only fields at all.

func (*Instance) Proto 

func (e *Instance) Proto() *api.Instance

Proto returns cipd.Instance proto with information from this entity.

type Package 

type Package struct {
	Name string `gae:"$id"` // e.g. "a/b/c"

	RegisteredBy string    `gae:"registered_by"` // who registered it
	RegisteredTs time.Time `gae:"registered_ts"` // when it was registered

	Hidden bool `gae:"hidden"` // if true, hide from the listings
	// contains filtered or unexported fields
}

Package represents a package as it is stored in the datastore.

It is mostly a marker that the package exists plus some minimal metadata about this specific package. Metadata for the package prefix is stored separately elsewhere (see 'metadata' package). Package instances, tags and refs are stored as child entities (see below).

Root entity. ID is the package name.

Compatible with the python version of the backend.

type ProcessingResult 

type ProcessingResult struct {
	ProcID   string         `gae:"$id"`     // processor that generated the result
	Instance *datastore.Key `gae:"$parent"` // instance it was generated from

	CreatedTs time.Time `gae:"created_ts"`     // when it was generated
	Success   bool      `gae:"success"`        // mostly for for indexing
	Error     string    `gae:"error,noindex"`  // for Success == false
	ResultRaw []byte    `gae:"result,noindex"` // for Success == true
	// contains filtered or unexported fields
}

ProcessingResult holds information extracted from the package instance file.

It is obtained during an asynchronous post processing step triggered after the instance is uploaded. Immutable.

Entity ID is a processor name used to extract it. Parent entity is PackageInstance the information was extracted from.

func (*ProcessingResult) Proto 

func (p *ProcessingResult) Proto() (*api.Processor, error)

Proto returns cipd.Processor proto with information from this entity.

func (*ProcessingResult) ReadResult 

func (p *ProcessingResult) ReadResult(r interface{}) error

ReadResult deserializes the result into the given variable.

Does nothing if there's no results stored.

func (*ProcessingResult) ReadResultIntoStruct 

func (p *ProcessingResult) ReadResultIntoStruct(s *structpb.Struct) error

ReadResultIntoStruct deserializes the result into the protobuf.Struct.

Does nothing if there's no results stored.

func (*ProcessingResult) WriteResult 

func (p *ProcessingResult) WriteResult(r interface{}) error

WriteResult overwrites ResultRaw field with compressed JSON-serialized 'r'.

'r' should serialize into a JSON object, e.g '{...}'.

type Ref 

type Ref struct {
	Name    string         `gae:"$id"`     // e.g. "latest"
	Package *datastore.Key `gae:"$parent"` // see PackageKey()

	InstanceID string    `gae:"instance_id"` // see common.ObjectRefToInstanceID()
	ModifiedBy string    `gae:"modified_by"` // who moved it the last time
	ModifiedTs time.Time `gae:"modified_ts"` // when it was moved the last time
	// contains filtered or unexported fields
}

Ref represents a named pointer to some package instance.

ID is a ref name, the parent entity is the corresponding Package.

Compatible with the python version of the backend.

func GetRef 

func GetRef(c context.Context, pkg, ref string) (*Ref, error)

GetRef fetches the given ref.

Returns gRPC-tagged NotFound error if there's no such ref.

func ListInstanceRefs 

func ListInstanceRefs(c context.Context, inst *Instance) (out []*Ref, err error)

ListInstanceRefs returns all refs that point to a particular instance, most recently modified first.

This is a subset of the output of ListPackageRefs for the corresponding package.

Assumes 'inst' is a valid Instance, panics otherwise.

Returns an empty list if there's no such instance at all.

func ListPackageRefs 

func ListPackageRefs(c context.Context, pkg string) (out []*Ref, err error)

ListPackageRefs returns all refs in a package, most recently modified first.

Returns an empty list if there's no such package at all.

func (*Ref) Proto 

func (e *Ref) Proto() *api.Ref

Proto returns cipd.Ref proto with information from this entity.

type Tag 

type Tag struct {
	ID       string         `gae:"$id"`     // see TagID()
	Instance *datastore.Key `gae:"$parent"` // key of corresponding Instance entity

	Tag          string    `gae:"tag"`           // the tag itself, as "k:v" pair
	RegisteredBy string    `gae:"registered_by"` // who added this tag
	RegisteredTs time.Time `gae:"registered_ts"` // when it was added
	// contains filtered or unexported fields
}

Tag represents a "key:value" pair attached to some package instance.

Tags exist as separate entities (as opposed to being a repeated property of Instance entity) to allow essentially unlimited number of them. Also we often do not want to fetch all tags when fetching Instance entity.

ID is hex-encoded SHA1 of the tag. The parent entity is the corresponding Instance. The tag string can't be made a part of the key because the total key length (including entity kind names and all parent keys) is limited to 500 bytes and tags are allowed to be pretty long (up to 400 bytes).

There's a possibility someone tries to abuse a collision in SHA1 hash used by TagID() to attach or detach some wrong tag. The chance is minuscule, and it's not a big deal if it happens now, but it may become important in the future once tags have their own ACLs. For the sake of paranoia, we double check 'Tag' field in all entities we touch.

Compatible with the python version of the backend.

func ListInstanceTags 

func ListInstanceTags(c context.Context, inst *Instance) (out []*Tag, err error)

ListInstanceTags returns all tags attached to an instance, sorting them by the tag key first, and then by the timestamp (most recent first).

Returns an empty list if there's no such instance at all.

func (*Tag) Proto 

func (t *Tag) Proto() *api.Tag

Proto returns cipd.Tag proto with information from this entity.

Assumes the tag is valid.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.