common

package
v0.0.0-...-ddee7fb Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Nov 9, 2021 License: Apache-2.0 Imports: 40 Imported by: 0

Documentation

Index

Constants

View Source 
const ServiceConfigID = "service_config"

ServiceConfigID is the key for the service config entity in datastore.

Variables

View Source 
var ErrConsoleNotFound = errors.New("console not found")

ErrConsoleNotFound is returned from GetConsole if the requested console isn't known to exist.

View Source 
var ErrInvalidBuilderID = errors.New("the string is not a valid builder ID")
View Source 
var ErrInvalidLegacyBuildID = errors.New("the string is not a valid legacy build ID")
View Source 
var ErrInvalidLegacyBuilderID = errors.New("the string is not a valid legacy builder ID")

Functions

func BucketResourceID 

func BucketResourceID(project, bucket string) string

BucketResourceID returns a string identifying the bucket resource. It is used when checking bucket permission.

func CheckACL 

func CheckACL(c context.Context, acl ACL) (bool, error)

CheckACL returns true if the caller is in the ACL.

Returns an internal error if the check itself fails.

func Duration 

func Duration(start *timestamppb.Timestamp, ends ...*timestamppb.Timestamp) string

Duration returns duration between start and the earliest of ends. Ignores nil ends. Fallbacks to "N/A" on insufficient data.

func GetJSONData 

func GetJSONData(client *http.Client, url string, out interface{}) (err error)

GetJSONData fetches data from the given URL, parses the response body to `out`. It follows redirection and returns an error if the status code is 4xx or 5xx.

func GetSettings 

func GetSettings(c context.Context) *config.Settings

GetSettings returns the service (aka global) config for the current instance of Milo from the datastore. Returns an empty config and warn heavily if none is found. TODO(hinoka): Use process cache to cache configs.

func HumanDuration 

func HumanDuration(d time.Duration) string

HumanDuration translates d into a human readable string of x units y units, where x and y could be in days, hours, minutes, or seconds, whichever is the largest.

func IsAllowed 

func IsAllowed(c context.Context, project string) (bool, error)

IsAllowed checks to see if the user in the context is allowed to access the given project.

Returns false for unknown projects. Returns an internal error if the check itself fails.

func JSONMarshalCompressed 

func JSONMarshalCompressed(message interface{}) ([]byte, error)

JSONMarshalCompressed converts a message into compressed JSON form, suitable for storing in memcache.

func JSONUnmarshalCompressed 

func JSONUnmarshalCompressed(serialized []byte, out interface{}) error

JSONUnmarshalCompressed converts a message back from compressed JSON form.

func LegacyBuilderIDString 

func LegacyBuilderIDString(bid *buildbucketpb.BuilderID) string

LegacyBuilderIDString returns a legacy string identifying the builder. It is used in the Milo datastore.

func MergeStrings 

func MergeStrings(sss ...[]string) []string

MergeStrings merges multiple string slices together into a single slice, removing duplicates.

func NewShouldResemberMatcher 

func NewShouldResemberMatcher(expected interface{}) shouldResembleMatcher

NewShouldResemberMatcher constrcuts a gomock.Matcher that performs convey.ShouldResemble(actual, expected).

func ObfuscateEmail 

func ObfuscateEmail(email string) template.HTML

ObfuscateEmail converts a string containing email address email@address.com into email<junk>@address.com.

func ParseBuilderID 

func ParseBuilderID(bid string) (*buildbucketpb.BuilderID, error)

ParseBuilderID parses the canonical builder ID (e.g. `<project>/<bucket>/<builder>`) and returns the BuilderID struct.

func ParseIntFromForm 

func ParseIntFromForm(form url.Values, key string, base int, bitSize int) (int64, error)

ParseIntFromForm parses an integer from a form.

func ParseLegacyBuildID 

func ParseLegacyBuildID(bid string) (builderID *buildbucketpb.BuilderID, number int32, err error)

ParseLegacyBuildID parses the legacy build ID (e.g. `buildbucket/luci.<project>.<bucket>/<builder>/<number>`)

func ParseLegacyBuilderID 

func ParseLegacyBuilderID(bid string) (*buildbucketpb.BuilderID, error)

ParseLegacyBuilderID parses the legacy builder ID (e.g. `buildbucket/luci.<project>.<bucket>/<builder>`) and returns the BuilderID struct.

func ReadExactOneFromForm 

func ReadExactOneFromForm(form url.Values, key string) (string, error)

ReadExactOneFromForm read a string from a form. There must be exactly one and non-empty entry of the given key in the form.

func ReplaceNSEWith 

func ReplaceNSEWith(err errors.MultiError, replacement error) error

ReplaceNSEWith takes an errors.MultiError returned by a datastore.Get() on a slice (which is always a MultiError), filters out all datastore.ErrNoSuchEntity or replaces it with replacement instances, and returns an error generated by errors.LazyMultiError.

func ShortenEmail 

func ShortenEmail(email string) string

ShortenEmail shortens Google emails.

func TagGRPC 

func TagGRPC(c context.Context, err error) error

TagGRPC annotates some gRPC with Milo specific semantics, specifically: * Marks the error as Unauthorized if the user is not logged in, and the underlying error was a 403 or 404. * Otherwise, tag the error with the original error code.

func UpdateProjects 

func UpdateProjects(c context.Context) error

UpdateProjects reads project configs from LUCI Config and updates entities.

Visits all LUCI projects (not only ones that have Milo config) to grab their visibility ACL from project.cfg file.

func UpdateServiceConfig 

func UpdateServiceConfig(c context.Context) (*config.Settings, error)

UpdateServiceConfig fetches the service config from luci-config and then stores a snapshot of the configuration in datastore.

func WithCachedAccessClient 

func WithCachedAccessClient(c context.Context, a *CachedAccessClient) context.Context

WithCachedAccessClient attaches an AccessClient to the given context.

Types

type ACL 

type ACL struct {
	Groups     []string
	Identities []identity.Identity
}

ACL lists groups and identities that are allowed to see consoles in a project.

Fetched from project.cfg config file in fetchProjectACL.

type CachedAccessClient 

type CachedAccessClient struct {
	accessProto.AccessClient
	// contains filtered or unexported fields
}

CachedAccessClient wraps an accessProto.AccessClient and caches its response.

func GetCachedAccessClient 

func GetCachedAccessClient(c context.Context) *CachedAccessClient

GetCachedAccessClient retrieves an AccessClient from the given context.

func NewCachedAccessClient 

func NewCachedAccessClient(c context.Context, buildbucketHost string, cache caching.BlobCache) (*CachedAccessClient, error)

NewCachedAccessClient creates a new AccessClient for talking to this milo instance's buildbucket instance.

func NewTestCachedAccessClient 

func NewTestCachedAccessClient(client *bbAccess.TestClient, cache caching.BlobCache) *CachedAccessClient

NewTestCachedAccessClient creates a cached access client for testing purpose.

func (*CachedAccessClient) BucketPermissions 

func (ac *CachedAccessClient) BucketPermissions(c context.Context, buckets ...string) (bbAccess.Permissions, error)

BucketPermissions gets permissions for the current identity for all given buckets.

TODO(mknyszek): If a cache entry expires, then there could be QPS issues if all instances query buildbucket for an update simultaneously. Evaluate whether there's an issue in practice, and if so, consider expiring cache entries randomly.

type Console 

type Console struct {
	// Parent is a key to the parent Project entity where this console was
	// defined in.
	Parent *datastore.Key `gae:"$parent"`

	// ID is the ID of the console.
	// This ID is local to the console, i.e. it should equal .Def.Id and not
	// Def.ExternalId.
	ID string `gae:"$id"`

	// Ordinal specifies the console's ordering in its project's consoles list.
	Ordinal int

	// The URL to the luci-config definition of this console.
	ConfigURL string

	// The luci-config revision from when this Console was retrieved.
	ConfigRevision string `gae:",noindex"`

	// (indexed) All builder IDs mentioned by this console config.
	Builders []string

	// Def is the actual underlying proto Console definition.
	// If this console is external (i.e. a reference to a console from
	// another project), this will contain the resolved Console definition,
	// but with ExternalId and ExternalProject also set.
	Def config.Console `gae:",noindex"`

	// Realm that the console exists under.
	Realm string
	// contains filtered or unexported fields
}

Console is a datastore entity representing a single console.

func GetAllConsoles 

func GetAllConsoles(c context.Context, builderID string) ([]*Console, error)

GetAllConsoles returns all Consoles (across all projects) which contain the builder ID. If builderID is empty, then this retrieves all Consoles.

TODO-perf(iannucci): Maybe memcache this too.

func GetConsole 

func GetConsole(c context.Context, proj, id string) (*Console, error)

GetConsole returns the requested console.

TODO-perf(iannucci,hinoka): Memcache this.

func GetConsoles 

func GetConsoles(c context.Context, consoles []ConsoleID) ([]*Console, error)

GetConsoles returns the requested consoles. Excludes references to external consoles that the user does not have access to.

TODO-perf(iannucci,hinoka): Memcache this.

func GetProjectConsoles 

func GetProjectConsoles(c context.Context, projectID string) ([]*Console, error)

GetProjectConsoles returns all consoles for the given project ordered as in config. Excludes references to external consoles that the user does not have access to.

func (*Console) Buckets 

func (c *Console) Buckets() stringset.Set

Buckets returns all buckets referenced by this Console's Builders.

func (*Console) ConsoleID 

func (c *Console) ConsoleID() ConsoleID

ConsoleID returns a struct containing ID strings for the console and its parent project.

func (*Console) FilterBuilders 

func (c *Console) FilterBuilders(perms access.Permissions)

FilterBuilders uses an access.Permissions to filter out builder IDs and builders from the definition, and builders in the definition's header, which are not allowed by the permissions.

func (*Console) IsExternal 

func (c *Console) IsExternal() bool

IsExternal returns whether the console is a reference to a console from another project. If this is the case, the console will have Def.ExternalProject and Def.ExternalId set.

func (*Console) ProjectID 

func (c *Console) ProjectID() string

ProjectID retrieves the project ID string of the console out of the Console's parent key.

type ConsoleID 

type ConsoleID struct {
	Project string
	ID      string
}

ConsoleID is a reference to a console.

func ParseConsoleID 

func ParseConsoleID(id string) (cid ConsoleID, err error)

ParseConsoleID reformats the string into a ConsoleID.

func (*ConsoleID) SetID 

func (id *ConsoleID) SetID(c context.Context, console *Console) *Console

SetID returns an empty Console datastore entity keyed with itself.

func (*ConsoleID) String 

func (id *ConsoleID) String() string

type Interval 

type Interval struct {
	Start time.Time `json:"start,omitempty"`
	End   time.Time `json:"end,omitempty"`
	Now   time.Time `json:"now,omitempty"`
}

func ToInterval 

func ToInterval(start, end, now *timestamppb.Timestamp) (result Interval)

func (Interval) Duration 

func (in Interval) Duration() time.Duration

func (Interval) Ended 

func (in Interval) Ended() bool

func (Interval) Started 

func (in Interval) Started() bool

type Project 

type Project struct {
	ID        string `gae:"$id"`
	HasConfig bool
	ACL       ACL `gae:",noindex"`

	LogoURL        string
	BugURLTemplate string

	// IgnoredBuilderIds is a list of builder IDs to be ignored by pubsub handler.
	// ID format: <bucket>/<builder>
	IgnoredBuilderIDs []string

	// ExternalBuilderIDs is a list of builder IDs that are not in this project
	// but are referenced by this project.
	// ID format: <project>/<bucket>/<builder>
	ExternalBuilderIDs []string
	// contains filtered or unexported fields
}

Project is a datastore entity representing a single project.

Its children are consoles. This entity exists even if the project doesn't define milo.cfg config file. It has HasConfig == false in this case. We still need the entity to be able to check project's ACLs when accessing individual build pages.

func GetProject 

func GetProject(c context.Context, project string) (*Project, error)

GetProject loads the project from the datastore.

func GetVisibleProjects 

func GetVisibleProjects(c context.Context) ([]*Project, error)

GetVisibleProjects returns all projects with consoles the current user has access to.

Skips projects that do not have Milo config file.

func (*Project) BuilderIsIgnored 

func (p *Project) BuilderIsIgnored(builderID *buildbucketpb.BuilderID) bool

BuilderIsIgnored checks if the builder is marked as ignored in this project.

type PubSubMessage 

type PubSubMessage struct {
	Attributes map[string]interface{} `json:"attributes"`
	Data       string                 `json:"data"`
	MessageID  string                 `json:"message_id"`
}

type PubSubSubscription 

type PubSubSubscription struct {
	Message PubSubMessage `json:"message"`
}

func (*PubSubSubscription) GetData 

func (m *PubSubSubscription) GetData() ([]byte, error)

GetData returns the expanded form of Data (decoded from base64).

type ServiceConfig 

type ServiceConfig struct {
	// ID is the datastore key.  This should be static, as there should only be
	// one service config.
	ID string `gae:"$id"`
	// Revision is the revision of the config, taken from luci-config.  This is used
	// to determine if the entry needs to be refreshed.
	Revision string
	// Data is the binary proto of the config.
	Data []byte `gae:",noindex"`
	// Text is the text format of the config.  For human consumption only.
	Text string `gae:",noindex"`
	// LastUpdated is the time this config was last updated.
	LastUpdated time.Time
}

ServiceConfig is a container for the instance's service config.

func GetCurrentServiceConfig 

func GetCurrentServiceConfig(c context.Context) (*ServiceConfig, error)

GetCurrentServiceConfig gets the service config for the instance from either process cache or datastore cache.

Source Files

View all Source files

Directories

Path Synopsis
milostatus

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.