model

package
v0.0.0-...-ff9afc2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 25, 2024 License: Apache-2.0 Imports: 34 Imported by: 1

Documentation

Overview

Package model contains datastore model implementation.

Index

Constants

View Source 
const (
	// BuildKind is a Build entity's kind in the datastore.
	BuildKind = "Build"

	// BuildStatusKind is a BuildStatus entity's kind in the datastore.
	BuildStatusKind = "BuildStatus"

	// BuildStorageDuration is the maximum lifetime of a Build.
	//
	// Lifetime is the time elapsed since the Build creation time.
	// Cron runs periodically to scan and remove all the Builds of which
	// lifetime exceeded this duration.
	BuildStorageDuration = time.Hour * 24 * 30 * 18 // ~18 months
	// BuildMaxCompletionTime defines the maximum duration that a Build must be
	// completed within, from the build creation time.
	BuildMaxCompletionTime = time.Hour * 24 * 5 // 5 days

)
View Source 
const (
	// BuildStepsKind is a BuildSteps entity's kind in the datastore.
	BuildStepsKind = "BuildSteps"

	// BuildInfraKind is a BuildInfra entity's kind in the datastore.
	BuildInfraKind = "BuildInfra"

	// BuildInputPropertiesKind is a BuildInputProperties entity's kind in the datastore.
	BuildInputPropertiesKind = "BuildInputProperties"

	// BuildOutputPropertiesKind is a BuildOutputProperties entity's kind in the datastore.
	BuildOutputPropertiesKind = "BuildOutputProperties"
)
View Source 
const (
	// BucketKind is a bucket entity's kind in the datastore.
	BucketKind = "BucketV2"
)
View Source 
const BuildStepsMaxBytes = 1e6

BuildStepsMaxBytes is the maximum length of BuildSteps.Bytes. If Bytes exceeds this maximum, this package will try to compress it, setting IsZipped accordingly, but if this length is still exceeded it's an error to write such entities to the datastore. Use FromProto to ensure this maximum is respected.

View Source 
const BuilderExpirationDuration = 4 * 7 * 24 * time.Hour // 4 weeks

BuilderExpirationDuration is the maximum duration a builder can go without having a build scheduled before its BuilderStat may be deleted.

View Source 
const BuilderKind = "Bucket.Builder"

BuilderKind is the kind of the Builder entity.

View Source 
const BuilderStatKind = "Builder"

BuilderStatKind is the kind of the BuilderStat entity.

View Source 
const BuilderStatZombieDuration = 6 * time.Hour

BuilderStatZombieDuration is the maximum duration for which a zombie BuilderStat that can exist without having a build scheduled before it may be deleted.

Zombie BuilderStat is a BuilderStat entity of which Builder entity doesn't exist.

View Source 
const MaxTagIndexEntries = 1000

MaxTagIndexEntries is the maximum number of entries that may be associated with a single TagIndex entity.

View Source 
const (
	// ProjectKind is a project entity's kind in the datastore.
	ProjectKind = "Project"
)
View Source 
const TagIndexShardCount = 16

TagIndexShardCount is the number of shards used by the TagIndex.

Variables

View Source 
var DefaultBuildMask = HardcodedBuildMask(defaultFieldMask.Paths...)

DefaultBuildMask is the default mask to use for read requests.

View Source 
var GetLimitedBuildMask = HardcodedBuildMask(BuildFieldsWithVisibility(pb.BuildFieldVisibility_BUILDS_GET_LIMITED_PERMISSION)...)

GetLimitedBuildMask is an extra mask to hide fields from callers who have the BuildsGetLimited permission but not BuildsGet. These callers should only be able to see fields specified in this mask.

View Source 
var LegacyCancelationReason_name = []string{
	0:                  "UNSET",
	ExplicitlyCanceled: "CANCELED_EXPLICITLY",
	TimeoutCanceled:    "TIMEOUT",
}
View Source 
var LegacyFailureReason_name = []string{
	0:                      "UNSET",
	BuildFailure:           "BUILD_FAILURE",
	BuildbucketFailure:     "BUILDBUCKET_FAILURE",
	InfraFailure:           "INFRA_FAILURE",
	InvalidBuildDefinition: "INVALID_BUILD_DEFINITION",
}
View Source 
var LegacyResult_name = []string{
	0:        "UNSET",
	Success:  "SUCCESS",
	Failure:  "FAILURE",
	Canceled: "CANCELED",
}
View Source 
var LegacyStatus_name = []string{
	0:         "UNSET",
	Scheduled: "SCHEDULED",
	Started:   "STARTED",
	Completed: "COMPLETED",
}
View Source 
var ListOnlyBuildMask = HardcodedBuildMask(BuildFieldsWithVisibility(pb.BuildFieldVisibility_BUILDS_LIST_PERMISSION)...)

ListOnlyBuildMask is an extra mask to hide fields from callers who have the BuildsList permission but not BuildsGet or BuildsGetLimited. These callers should only be able to see fields specified in this mask.

View Source 
var NoopBuildMask = &BuildMask{m: mask.All(&buildPrototype)}

NoopBuildMask selects all fields.

View Source 
var TagIndexIncomplete = errors.BoolTag{Key: errors.NewTagKey("tag index incomplete")}

TagIndexIncomplete means the tag index is incomplete and thus cannot be searched.

Functions

func BucketKey 

func BucketKey(ctx context.Context, project, bucket string) *datastore.Key

BucketKey returns a datastore key of a bucket.

func BuildFieldsWithVisibility 

func BuildFieldsWithVisibility(visibility pb.BuildFieldVisibility) []string

BuildFieldsWithVisibility returns a list of Build fields that are visible with the specified level of read permission. For example, the following: 

BuildFieldsWithVisibility(pb.BuildFieldVisibility_BUILDS_GET_LIMITED_PERMISSION)

will return a list of Build fields (including nested fields) that have been annotated with either of the following field options: 

[(visible_with) = BUILDS_GET_LIMITED_PERMISSION]
[(visible_with) = BUILDS_LIST_PERMISSION]

Note that visibility permissions are strictly ordered: if a user has the GetLimited permission, that implies they also have the List permission.

func BuilderKey 

func BuilderKey(ctx context.Context, project, bucket, builder string) *datastore.Key

BuilderKey returns a datastore key of a builder.

func BuilderStatKey 

func BuilderStatKey(ctx context.Context, project, bucket, builder string) *datastore.Key

BuilderStatKey returns a datastore key for a given Builder.

func ConstructNextSyncTime 

func ConstructNextSyncTime(backend, project string, shardID int, syncTime time.Time) string

func CustomBuilderMetricsKey 

func CustomBuilderMetricsKey(ctx context.Context) *datastore.Key

CustomBuilderMetricsKey is CustomBuilderMetrics entity key.

func EvaluateBuildForCustomBuilderMetrics 

func EvaluateBuildForCustomBuilderMetrics(ctx context.Context, bld *Build, loadDetails bool) error

EvaluateBuildForCustomBuilderMetrics finds the custom builder metrics from bld.CustomMetrics and evaluates the build against those metrics' predicates. Update bld.CustomBuilderXXMetrics to add names of the matching metrics.

func GenerateSequenceNumbers 

func GenerateSequenceNumbers(ctx context.Context, name string, n int) (int32, error)

GenerateSequenceNumbers generates n numbers for the given sequence name, returning the smallest number the caller may use. For a returned number i, the caller may use [i, i+n).

func GetIgnoreMissing 

func GetIgnoreMissing(ctx context.Context, dst ...any) error

GetIgnoreMissing fetches the given entities from the datastore, ignoring datastore.ErrNoSuchEntity errors. All other errors are returned. For valid values of dst, see datastore.Get.

func GetMultiOutputProperties 

func GetMultiOutputProperties(c context.Context, props ...*BuildOutputProperties) error

GetMultiOutputProperties fetches multiple BuildOutputProperties in parallel.

func LoadBuildDetails 

func LoadBuildDetails(ctx context.Context, m *BuildMask, redact func(*pb.Build) error, builds ...*pb.Build) error

LoadBuildDetails loads the details of the given builds, trimming them according to the specified mask and redaction function.

func ProjectKey 

func ProjectKey(ctx context.Context, project string) *datastore.Key

ProjectKey returns a datastore key of a project.

func UpdateBuilderStat 

func UpdateBuilderStat(ctx context.Context, builds []*Build, scheduledTime time.Time) error

UpdateBuilderStat updates or creates datastore BuilderStat entities.

func UpdateTagIndex 

func UpdateTagIndex(ctx context.Context, tag string, ents []TagIndexEntry) error

UpdateTagIndex updates the tag index for the given tag.

Types

type Bucket 

type Bucket struct {

	// ID is the bucket in v2 format.
	// e.g. try (never luci.chromium.try).
	ID     string         `gae:"$id"`
	Parent *datastore.Key `gae:"$parent"`

	// Bucket is the bucket in v2 format.
	// e.g. try (never luci.chromium.try).
	//
	// Note: This field is "computed" in *Bucket.Save phase and exists because v1
	// APIs are allowed to search without a luci project context.
	Bucket string `gae:"bucket_name"`
	// Proto is the pb.Bucket proto representation of the bucket.
	//
	// swarming.builders is zeroed and stored in separate Builder datastore
	// entities due to potentially large size.
	Proto *pb.Bucket `gae:"config,legacy"`
	// Revision is the config revision this entity was created from.
	// TODO(crbug/1042991): Switch to noindex.
	Revision string `gae:"revision"`
	// Schema is this entity's schema version.
	// TODO(crbug/1042991): Switch to noindex.
	Schema int32 `gae:"entity_schema_version"`
	// Shadows is the list of buckets this bucket shadows.
	// It will be empty if this bucket is not a shadow bucket.
	Shadows []string `gae:"shadows,noindex"`
	// contains filtered or unexported fields
}

Bucket is a representation of a bucket in the datastore.

func (*Bucket) Load 

func (b *Bucket) Load(p datastore.PropertyMap) error

Load implements datastore.PropertyLoadSaver

func (*Bucket) Save 

func (b *Bucket) Save(withMeta bool) (datastore.PropertyMap, error)

Save implements datastore.PropertyLoadSaver. It mutates this bucket entity by always copying its ID value into the bucket_name field.

type Build 

type Build struct {
	ID int64 `gae:"$id"`

	// LegacyProperties are properties set for v1 legacy builds.
	LegacyProperties
	// UnusedProperties are properties set previously but currently unused.
	UnusedProperties

	// Proto is the pb.Build proto representation of the build.
	//
	// infra, input.properties, output.properties, and steps
	// are zeroed and stored in separate datastore entities
	// due to their potentially large size (see details.go).
	// tags are given their own field so they can be indexed.
	//
	// noindex is not respected here, it's set in pb.Build.ToProperty.
	Proto *pb.Build `gae:"proto,legacy"`

	Project string `gae:"project"`
	// <project>/<bucket>. Bucket is in v2 format.
	// e.g. chromium/try (never chromium/luci.chromium.try).
	BucketID string `gae:"bucket_id"`
	// <project>/<bucket>/<builder>. Bucket is in v2 format.
	// e.g. chromium/try/linux-rel.
	BuilderID string `gae:"builder_id"`

	Canary bool `gae:"canary"`

	CreatedBy identity.Identity `gae:"created_by"`
	// TODO(nodir): Replace reliance on create_time indices with id.
	CreateTime time.Time `gae:"create_time"`
	// Experimental, if true, means to exclude from monitoring and search results
	// (unless specifically requested in search results).
	Experimental bool `gae:"experimental"`
	// Experiments is a slice of experiments enabled or disabled on this build.
	// Each element should look like "[-+]$experiment_name".
	//
	// Special case:
	//   "-luci.non_production" is not kept here as a storage/index
	//   optimization.
	//
	//   Notably, all search/query implementations on the Build model
	//   apply this filter in post by checking that
	//   `b.ExperimentStatus("luci.non_production") == pb.Trinary_YES`.
	//
	//   This is because directly including this value in the datastore query
	//   results in bad performance due to excessive zig-zag join overhead
	//   in the datastore, since 99%+ of the builds in Buildbucket are production
	//   builds.
	Experiments []string `gae:"experiments"`
	Incomplete  bool     `gae:"incomplete"`

	// Deprecated; remove after v1 api turndown
	IsLuci bool `gae:"is_luci"`

	ResultDBUpdateToken string    `gae:"resultdb_update_token,noindex"`
	Status              pb.Status `gae:"status_v2"`
	StatusChangedTime   time.Time `gae:"status_changed_time"`
	// Tags is a slice of "<key>:<value>" strings taken from Proto.Tags.
	// Stored separately in order to index.
	Tags []string `gae:"tags"`

	// UpdateToken is set at the build creation time, and UpdateBuild requests are required
	// to have it in the header.
	UpdateToken string `gae:"update_token,noindex"`

	// StartBuildToken is set when a backend task starts, and StartBuild requests are required
	// to have it in the header.
	StartBuildToken string `gae:"start_build_token,noindex"`

	// PubSubCallback, if set, creates notifications for build status changes.
	PubSubCallback PubSubCallback `gae:"pubsub_callback,noindex"`

	// ParentID is the build's immediate parent build id.
	// Stored separately from AncestorIds in order to index this special case.
	ParentID int64 `gae:"parent_id"`

	// Ids of the build’s ancestors. This includes all parents/grandparents/etc.
	// This is ordered from top-to-bottom so `ancestor_ids[0]` is the root of
	// the builds tree, and `ancestor_ids[-1]` is this build's immediate parent.
	// This does not include any "siblings" at higher levels of the tree, just
	// the direct chain of ancestors from root to this build.
	AncestorIds []int64 `gae:"ancestor_ids"`

	// Id of the first StartBuildTask call Buildbucket receives for the build.
	// Buildbucket uses this to deduplicate the other StartBuildTask calls.
	StartBuildTaskRequestID string `gae:"start_task_request_id,noindex"`

	// Id of the first StartBuild call Buildbucket receives for the build.
	// Buildbucket uses this to deduplicate the other StartBuild calls.
	StartBuildRequestID string `gae:"start_build_request_id,noindex"`

	// Computed field to be used by a cron job to get the builds that have not
	// been updated for a while.
	//
	// It has a format like "<backend>--<project>--<shard>-=<next-sync-time>", where
	//   * backend is the backend target.
	//	 * project is the luci project of the build.
	//   * shard is the added prefix to make sure the index on this property is
	//     sharded to avoid hot spotting.
	//   * next-sync-time is the unix time of the next time the build is supposed
	//     to be synced with its backend task, truncated in minute.
	NextBackendSyncTime string `gae:"next_backend_sync_time"`

	// Backend target for builds on TaskBackend.
	BackendTarget string `gae:"backend_target"`

	// How far into the future should NextBackendSyncTime be set after a build update.
	BackendSyncInterval time.Duration `gae:"backend_sync_interval,noindex"`

	// The list of custom metrics that the build may report to.
	// Copied from the builder config when the build is created.
	CustomMetrics []CustomMetric `gae:"custome_metrics,noindex"`

	// Names of the custom builder metrics this build could report to.
	// Each base has a separate field for custom builder metrics based on it.
	// We report builder metrics by querying datastore, so doing predicate checks
	// on query results is infeasible or expensive. Instead we should do the
	// predicate check when an event happens then saves the metric name if the
	// check passes. So later we could run a query for that specific custom metric.
	CustomBuilderCountMetrics               []string `gae:"custom_builder_count_metrics"`
	CustomBuilderMaxAgeMetrics              []string `gae:"custom_builder_max_age_metrics"`
	CustomBuilderConsecutiveFailuresMetrics []string `gae:"custom_builder_consecutive_failures_metrics"`
	// contains filtered or unexported fields
}

Build is a representation of a build in the datastore. Implements datastore.PropertyLoadSaver.

func (*Build) ClearLease 

func (b *Build) ClearLease()

ClearLease clears the lease by resetting the LeaseProperties.

NeverLeased is kept unchanged.

func (*Build) ExperimentStatus 

func (b *Build) ExperimentStatus(expname string) (ret pb.Trinary)

ExperimentStatus scans the experiments attached to this Build and returns:

  • YES - The experiment was known at schedule time and enabled.
  • NO - The experiment was known at schedule time and disabled.
  • UNSET - The experiment was unknown at schedule time.

Malformed Experiment filters are treated as UNSET.

func (*Build) ExperimentsString 

func (b *Build) ExperimentsString() string

ExperimentsString sorts, joins, and returns the enabled experiments with "|".

Returns "None" if no experiments were enabled in the build.

func (*Build) GenerateNextBackendSyncTime 

func (b *Build) GenerateNextBackendSyncTime(ctx context.Context, shards int32)

GenerateNextBackendSyncTime generates the build's NextBackendSyncTime if the build runs on a backend.

func (*Build) GetParentID 

func (b *Build) GetParentID() int64

func (*Build) IterExperiments 

func (b *Build) IterExperiments(cb func(enabled bool, exp string) bool)

IterExperiments parses all experiments and calls `cb` for each.

This will always include a call with bb.ExperimentNonProduction, even if '-'+bb.ExperimentNonProduction isn't recorded in the underlying Experiments field.

func (*Build) Load 

func (b *Build) Load(p datastore.PropertyMap) error

Load overwrites this representation of a build by reading the given datastore.PropertyMap. Mutates this entity.

func (*Build) MustParseNextBackendSyncTime 

func (b *Build) MustParseNextBackendSyncTime() (backend, project, shardID, syncTime string)

func (*Build) Realm 

func (b *Build) Realm() string

Realm returns this build's auth realm, or an empty string if not opted into the realms experiment.

func (*Build) Save 

func (b *Build) Save(withMeta bool) (datastore.PropertyMap, error)

Save returns the datastore.PropertyMap representation of this build. Mutates this entity to reflect computed datastore fields in the returned PropertyMap.

func (*Build) ToProto 

func (b *Build) ToProto(ctx context.Context, m *BuildMask, redact func(*pb.Build) error) (*pb.Build, error)

ToProto returns the *pb.Build representation of this build after applying the provided mask and redaction function.

func (*Build) ToSimpleBuildProto 

func (b *Build) ToSimpleBuildProto(ctx context.Context) *pb.Build

ToSimpleBuildProto returns the *pb.Build without loading steps, infra, input/output properties. Unlike ToProto, does not support redaction of fields.

type BuildInfra 

type BuildInfra struct {

	// ID is always 1 because only one such entity exists.
	ID int `gae:"$id,1"`
	// Build is the key for the build this entity belongs to.
	Build *datastore.Key `gae:"$parent"`
	// Proto is the pb.BuildInfra proto representation of the infra field.
	Proto *pb.BuildInfra `gae:"infra,legacy"`
	// contains filtered or unexported fields
}

BuildInfra is a representation of a build proto's infra field in the datastore.

func (*BuildInfra) Load 

func (bi *BuildInfra) Load(pm datastore.PropertyMap) error

Load implements datastore.PropertyLoadSaver in order to apply defaultStructValues to bi.Proto.

func (*BuildInfra) Save 

func (bi *BuildInfra) Save(withMeta bool) (datastore.PropertyMap, error)

Save implements datastore.PropertyLoadSaver

type BuildInputProperties 

type BuildInputProperties struct {

	// ID is always 1 because only one such entity exists.
	ID int `gae:"$id,1"`
	// Build is the key for the build this entity belongs to.
	Build *datastore.Key `gae:"$parent"`
	// Proto is the structpb.Struct representation of the properties field.
	Proto *structpb.Struct `gae:"properties,legacy"`
	// contains filtered or unexported fields
}

BuildInputProperties is a representation of a build proto's input field's properties field in the datastore.

type BuildMask 

type BuildMask struct {
	// contains filtered or unexported fields
}

BuildMask knows how to filter pb.Build proto messages.

func HardcodedBuildMask 

func HardcodedBuildMask(fields ...string) *BuildMask

HardcodedBuildMask returns a build mask with given fields.

Panics if some of them are invalid. Intended to be used to initialize constants or in tests.

func NewBuildMask 

func NewBuildMask(legacyPrefix string, legacy *fieldmaskpb.FieldMask, bm *pb.BuildMask) (*BuildMask, error)

NewBuildMask constructs a build mask either using a legacy `fields` FieldMask or new `mask` BuildMask (but not both at the same time, pick one).

legacyPrefix is usually "", but can be "builds" to trim "builds." from the legacy field mask (used by SearchBuilds API).

If the mask is empty, returns DefaultBuildMask.

func (*BuildMask) Includes 

func (m *BuildMask) Includes(fieldPath string) bool

Includes returns true if the given field path is included in the mask (either partially or entirely), or the mask includes all fields.

Panics if the fieldPath is invalid.

func (*BuildMask) Trim 

func (m *BuildMask) Trim(b *pb.Build) error

Trim applies the mask to the build in-place.

type BuildOutputProperties 

type BuildOutputProperties struct {

	// Build is the key for the build this entity belongs to.
	Build *datastore.Key `gae:"$parent"`
	// Proto is the structpb.Struct representation of the properties field.
	Proto *structpb.Struct `gae:"properties,legacy"`

	// ChunkCount indicates how many chunks this Proto is splitted into.
	ChunkCount int `gae:"chunk_count,noindex"`
	// contains filtered or unexported fields
}

BuildOutputProperties is a representation of a build proto's output field's properties field in the datastore.

Note: avoid directly access to BuildOutputProperties via datastore.Get and datastore.Put, as it may be chunked if it exceeds maxPropertySize. Please always use *BuildOutputProperties.Get and *BuildOutputProperties.Put.

func (*BuildOutputProperties) Get 

func (bo *BuildOutputProperties) Get(c context.Context) error

Get is a wrapper of `datastore.Get` to properly handle large properties.

func (*BuildOutputProperties) Put 

func (bo *BuildOutputProperties) Put(c context.Context) error

Put is a wrapper of `datastore.Put` to properly handle large properties. Suggest calling it in a transaction to correctly handle partial failures when putting PropertyChunk and BuildOutputProperties.

type BuildStatus 

type BuildStatus struct {

	// ID is always 1 because only one such entity exists.
	ID int `gae:"$id,1"`

	// Build is the key for the build this entity belongs to.
	Build *datastore.Key `gae:"$parent"`

	// Address of a build.
	// * If build number is enabled for the build, the address would be
	// <project>/<bucket>/<builder>/<build_number>;
	// * otherwise the address would be <project>/<bucket>/<builder>/b<build_id> (
	// to easily differentiate build number and build id).
	BuildAddress string `gae:"build_address"`

	Status pb.Status `gae:"status,noindex"`
	// contains filtered or unexported fields
}

BuildStatus stores build ids and their statuses.

type BuildSteps 

type BuildSteps struct {

	// ID is always 1 because only one such entity exists.
	ID int `gae:"$id,1"`
	// Build is the key for the build this entity belongs to.
	Build *datastore.Key `gae:"$parent"`
	// IsZipped indicates whether or not Bytes is zlib compressed.
	// Use ToProto to ensure this compression is respected.
	IsZipped bool `gae:"step_container_bytes_zipped,noindex"`
	// Bytes is the pb.Build proto representation of the build proto where only steps is set.
	// IsZipped determines whether this value is compressed or not.
	Bytes []byte `gae:"steps,noindex"`
	// contains filtered or unexported fields
}

BuildSteps is a representation of a build proto's steps field in the datastore.

func (*BuildSteps) CancelIncomplete 

func (s *BuildSteps) CancelIncomplete(ctx context.Context, now *timestamppb.Timestamp) (bool, error)

CancelIncomplete marks any incomplete steps as cancelled, returning whether at least one step was cancelled. The caller is responsible for writing the entity to the datastore if any steps were cancelled. This entity will not be mutated if an error occurs.

func (*BuildSteps) FromProto 

func (s *BuildSteps) FromProto(stp []*pb.Step) error

FromProto overwrites the current []*pb.Step representation of these steps. The caller is responsible for writing the entity to the datastore. This entity will not be mutated if an error occurs.

func (*BuildSteps) ToProto 

func (s *BuildSteps) ToProto(ctx context.Context) ([]*pb.Step, error)

ToProto returns the []*pb.Step representation of these steps.

type Builder 

type Builder struct {

	// ID is the builder name, e.g. "linux-rel".
	ID string `gae:"$id"`

	// Parent is the key of the parent Bucket.
	Parent *datastore.Key `gae:"$parent"`

	// Config is the builder configuration feched from luci-config.
	Config *pb.BuilderConfig `gae:"config,legacy"`

	// ConfigHash is used for fast deduplication of configs.
	ConfigHash string `gae:"config_hash"`

	// Metadata is the builder owner and health information.
	Metadata *pb.BuilderMetadata `gae:"builder_metadata,legacy"`
	// contains filtered or unexported fields
}

Builder is a Datastore entity that stores builder configuration. It is a child of Bucket entity.

Builder entities are updated together with their parents, in a cron job.

func (*Builder) FullBuilderName 

func (b *Builder) FullBuilderName() string

FullBuilderName return the builder name in the format of "<project>.<bucket>.<builder>".

type BuilderQueue 

type BuilderQueue struct {

	// ID is a string representation of the Bucket.Builder entity key,
	// e.g. "angle/ci/linux-test".
	// Only one BuilderQueue exists per Builder.
	ID string `gae:"$id"`

	// TriggeredBuilds represents the set of builds dispatched to task backend
	// but not yet terminated (i.e. builds in Cloud Task queue
	// + builds with created Backend tasks
	// + the builds actually running).
	TriggeredBuilds []int64 `gae:"triggered_builds,noindex"`

	// PendingBuilds represents the queue of builds not yet triggered.
	// (i.e. builds that were scheduled but not yet dispatched)
	PendingBuilds []int64 `gae:"pending_builds,noindex"`
	// contains filtered or unexported fields
}

BuilderQueue is a Datastore entity that stores the builder's current workload. It is used in conjunction with the max_concurrent_builds field defined in lucicfg builder definition. This entity only exists for builders with the max_concurrent_builds feature enabled.

type BuilderStat 

type BuilderStat struct {

	// ID is a string with format "{project}:{bucket}:{builder}".
	ID string `gae:"$id"`

	// LastScheduled is the last time we received a valid build scheduling request
	// for this builder. Probabilistically update when scheduling a build.
	LastScheduled time.Time `gae:"last_scheduled,noindex"`
	// contains filtered or unexported fields
}

BuilderStat represents a builder Datastore entity which is used internally for metrics.

The builder will be registered automatically by scheduling a build, and unregistered automatically by not scheduling builds for BuilderExpirationDuration.

Note: due to the historical reason, the entity kind is Builder.

func (*BuilderStat) BuilderKey 

func (s *BuilderStat) BuilderKey(ctx context.Context) *datastore.Key

BuilderKey returns a datastore key for the Builder that a given BuilderStat references.

Panics if the ID of the BuilderStat is invalid.

type CustomBuilderMetrics 

type CustomBuilderMetrics struct {

	// Key is CustomBuilderMetricsKey.
	Key *datastore.Key `gae:"$key"`
	// LastUpdate is when this entity changed the last time.
	LastUpdate time.Time `gae:",noindex"`

	Metrics *modeldefs.CustomBuilderMetrics
	// contains filtered or unexported fields
}

CustomBuilderMetrics is a Datastore entity that stores custom builder metrics and builders report to them.

type CustomMetric 

type CustomMetric struct {
	Base   pb.CustomMetricBase        `gae:"base,noindex"`
	Metric *pb.CustomMetricDefinition `gae:"metric"`
}

CustomMetric encapsulates information of one custom metric this build may report to.

type LeaseProperties 

type LeaseProperties struct {
	IsLeased bool `gae:"is_leased"`
	// TODO(crbug/1042991): Create datastore.PropertyConverter in server/auth.
	Leasee              []byte    `gae:"leasee"`
	LeaseExpirationDate time.Time `gae:"lease_expiration_date"`
	// LeaseKey is a random value used to verify the leaseholder's identity.
	LeaseKey    int  `gae:"lease_key"`
	NeverLeased bool `gae:"never_leased"`
}

LeaseProperties are properties associated with the legacy leasing API.

type LegacyCancelationReason 

type LegacyCancelationReason int

LegacyCancelationReason is the reason for a canceled legacy build. 

const (

	// ExplicitlyCanceled means the build was canceled (likely via API call).
	ExplicitlyCanceled LegacyCancelationReason
	// TimeoutCanceled means Buildbucket timed the build out.
	TimeoutCanceled
)

func (LegacyCancelationReason) String 

func (r LegacyCancelationReason) String() string

type LegacyFailureReason 

type LegacyFailureReason int

LegacyFailureReason is the reason for a legacy build failure. 

const (

	// BuildFailure means the build itself failed.
	BuildFailure LegacyFailureReason
	// BuildbucketFailure means something went wrong within Buildbucket.
	BuildbucketFailure
	// InfraFailure means something went wrong outside the build and Buildbucket.
	InfraFailure
	// InvalidBuildDefinition means the build system rejected the build definition.
	InvalidBuildDefinition
)

func (LegacyFailureReason) String 

func (r LegacyFailureReason) String() string

type LegacyProperties 

type LegacyProperties struct {
	LeaseProperties

	CancelationReason LegacyCancelationReason `gae:"cancelation_reason"`
	FailureReason     LegacyFailureReason     `gae:"failure_reason"`
	Parameters        []byte                  `gae:"parameters"`
	Result            LegacyResult            `gae:"result"`
	ResultDetails     []byte                  `gae:"result_details"`
	// ID of the Build this is a retry of.
	RetryOf int          `gae:"retry_of"`
	Status  LegacyStatus `gae:"status"`
	URL     string       `gae:"url,noindex"`
}

LegacyProperties are properties of legacy builds.

Parameters and ResultDetails are byte slices interpretable as JSON. TODO(crbug/1042991): Create datastore.PropertyConverter for JSON properties.

type LegacyResult 

type LegacyResult int

LegacyResult is the result of a completed legacy build. 

const (

	// Success means the build completed successfully.
	Success LegacyResult
	// Failure means the build failed and has an associated LegacyFailureReason.
	Failure
	// Canceled means the build was canceled
	// and has an associated LegacyCancelationReason.
	Canceled
)

func (LegacyResult) String 

func (r LegacyResult) String() string

type LegacyStatus 

type LegacyStatus int

LegacyStatus is the status of a legacy build request. 

const (

	// Scheduled builds may be leased and started.
	Scheduled LegacyStatus
	// Started builds are leased and marked as started.
	Started
	// Completed builds are finished and have an associated LegacyResult.
	Completed
)

func (LegacyStatus) String 

func (r LegacyStatus) String() string

type NumberSequence 

type NumberSequence struct {
	ID string `gae:"$id"`

	Next int32 `gae:"next_number,noindex"`
	// contains filtered or unexported fields
}

NumberSequence stores the next number in a named sequence of numbers.

type Project 

type Project struct {
	ID string `gae:"$id"`

	// CommonConfig is the share config among all buckets and builders in this
	// project.
	CommonConfig *pb.BuildbucketCfg_CommonConfig `gae:"common_config"`
	// contains filtered or unexported fields
}

Project is the parent entity of buckets in the datastore. Entities of this kind may not exist in the datastore if they don't have global_config.

type PropertyChunk 

type PropertyChunk struct {

	// ID starts from 1 to N where N is BuildOutputProperties.ChunkCount.
	ID int `gae:"$id"`
	// The BuildOutputProperties entity that this entity belongs to.
	Parent *datastore.Key `gae:"$parent"`

	// chunked bytes
	Bytes []byte `gae:"chunk,noindex"`
	// contains filtered or unexported fields
}

PropertyChunk stores a chunk of serialized and compressed BuildOutputProperties.Proto bytes. In the future, it may expand to buildInputProperties.

type PubSubCallback 

type PubSubCallback struct {
	AuthToken string `gae:"auth_token,noindex"`
	Topic     string `gae:"topic,noindex"`
	UserData  []byte `gae:"user_data,noindex"`
}

PubSubCallback encapsulates parameters for a Pub/Sub callback.

type RequestID 

type RequestID struct {

	// ID is a string of the form "<auth.Identity>:<request ID string>" encoded
	// as a hex string using SHA-256 for a well-distributed key space.
	ID string `gae:"$id"`

	// BuildID is the ID of the Build entity this entity refers to.
	BuildID    int64             `gae:"build_id,noindex"`
	CreatedBy  identity.Identity `gae:"created_by,noindex"`
	CreateTime time.Time         `gae:"create_time,noindex"`
	// RequestID is the original request ID string this entity was created from.
	RequestID string `gae:"request_id,noindex"`
	// contains filtered or unexported fields
}

RequestID stores request IDs for request deduplication.

func NewRequestID 

func NewRequestID(ctx context.Context, buildID int64, now time.Time, requestID string) *RequestID

NewRequestID returns a request ID with the entity ID filled in.

type TagIndex 

type TagIndex struct {

	// ID is a "<key>:<value>" or ":<index>:<key>:<value>" string for index > 0.
	ID string `gae:"$id"`
	// Incomplete means there are more than MaxTagIndexEntries entities
	// with the same ID, and therefore the index is incomplete and cannot be
	// searched.
	Incomplete bool `gae:"permanently_incomplete,noindex"`
	// Entries is a slice of TagIndexEntries matching this ID.
	Entries []TagIndexEntry `gae:"entries,noindex"`
	// contains filtered or unexported fields
}

TagIndex is an index used to search Build entities by tag.

type TagIndexEntry 

type TagIndexEntry struct {
	// BuildID is the ID of the Build entity this entry refers to.
	BuildID int64 `json:"build_id"`
	// <project>/<bucket>. Bucket is in v2 format.
	// e.g. chromium/try (never chromium/luci.chromium.try).
	BucketID string `json:"bucket_id"`
	// CreatedTime is the time this entry was created.
	CreatedTime time.Time `json:"created_time"`
}

TagIndexEntry refers to a particular Build entity.

func SearchTagIndex 

func SearchTagIndex(ctx context.Context, key, val string) ([]*TagIndexEntry, error)

SearchTagIndex searches the tag index for the given tag. Returns an error tagged with TagIndexIncomplete if the tag index is incomplete and thus cannot be searched.

func (*TagIndexEntry) FromProperty 

func (e *TagIndexEntry) FromProperty(p datastore.Property) error

FromProperty deserializes TagIndexEntries from the datastore. Implements datastore.PropertyConverter.

func (*TagIndexEntry) ToProperty 

func (e *TagIndexEntry) ToProperty() (datastore.Property, error)

ToProperty serializes TagIndexEntries to datastore format. Implements datastore.PropertyConverter.

type UnusedProperties 

type UnusedProperties struct {
	// PubSubCallback is normally a struct (see build.go), which translates into datastore
	// fields pubsub_callback.auth_token, pubsub_callback.topic, pubsub_callback.user_data
	// with no actual field called pubsub_callback. However, nil values in the datastore
	// may exist for pubsub_callback (should instead be represented by having all three
	// pubsub_callback.* fields nil, but isn't). Capture such nil values here.
	// TODO(crbug/1042991): Support this case properly in gae datastore package.
	PubSubCallback []byte `gae:"pubsub_callback,noindex"`
}

UnusedProperties are properties previously set but currently unused.

Notes

Bugs

  • - Setting this to 0 to see if it tamps down suprious Lookup costs in datastore. Should evaluate re-enabling after we turn entity caching back on.

Source Files

View all Source files

Directories

Path Synopsis
Package modeldefs contains messages used by Buildbucket data model.
 Package modeldefs contains messages used by Buildbucket data model.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.